The Wiert Corner – irregular stream of stuff

Jeroen W. Pluimers on .NET, C#, Delphi, databases, and personal interests

  • My badges

  • Twitter Updates

  • My Flickr Stream

  • Pages

  • All categories

  • Enter your email address to subscribe to this blog and receive notifications of new posts by email.

    Join 1,649 other followers

.NET/C#/PowerShell: building .NET DiscUtils library for virtual disk images

Posted by jpluimers on 2014/03/11

Mono Mac

.NET DiscUtils is an interesting open source .NET library for accessing and manipulating virtual disk images. Since it is entirely written in C# (without the need for P/Invoke), you should even be able to run this on non-Windows machines using mono. Later on, you will see the 0.11.0 build fails this, but it gives good hope it eventually will.

Virtual disk formats supported are DMG, ISO, RAW (IMG/IMA/VFD/FLP/BIF), VDI, VHD, VHDX, VMDK, and XVA, regular disks like Physical, iSCSI and NFS.

There are two ways of getting the .NET DiscUtils tools to run:

  1. download pre-build binaries (at the time of writing: version 0.10) from via .NET DiscUtils – Home, or
  2. from the latest source page, click the download button, then build the binaries from the source package. At the time of writing, that version is 0.11.

This post describes the second way, and requires PowerShell to be installed on your system (which probably is, as Windows 7 and Windows Server 2008 R2 include it).

Download, build and run .NET DiscUtil tools

These are the steps to perform:

  1. Download the latest source page,
  2. Unzip the downloaded ZIP file,
  3. Start a command prompt,
  4. Change the directory to the one where “build.ps1” resides,
  5. Copy the “compile.ps1” script below to a new file “compile.ps1” in that directory,
  6. Run the “compile.ps1” in PowerShell using this command:
    PowerShell -File compile.ps1

    If you get an error like this:
    C:\temp.jwp\discutils>PowerShell -File compile.ps1 File C:\temp.jwp\discutils\build.ps1 cannot be loaded because running scripts is disabled on this system. For more information, see about_Execution_Policies at http://go.microsoft.com/fwlink/?LinkID=135170.
    + CategoryInfo : SecurityError: (:) [], ParentContainsErrorRecordException
    + FullyQualifiedErrorId : UnauthorizedAccess C:\temp.jwp\discutils>

    Then you have to enable PowerShell script execution for the current user by executing this command from Enabling powershell to run unsigned scripts for the current user only:

    PowerShell Set-ExecutionPolicy -Scope CurrentUser -ExecutionPolicy RemoteSigned

  7. Copy the “copyAllUtils.ps1” script below to a new file “copyAllUtils.ps1” in the same directory as “build.ps1” resides,
  8. Run the “copyAllUtils.ps1” in PowerShell using this command:
    PowerShell -File copyAllUtils.ps1It lists the files it has copied, on my systemC:\temp.jwp\discutils\Utils\BCDDump\bin\Debug\BCDDump.exe
    C:\temp.jwp\discutils\Utils\BCDDump\bin\Debug\DiscUtils.Common.dll
    C:\temp.jwp\discutils\Utils\BCDDump\bin\Debug\DiscUtils.dll
    C:\temp.jwp\discutils\Utils\DiscUtils.PowerShell\bin\Debug\DiscUtils.PowerShell.dll
    C:\temp.jwp\discutils\Utils\DiskClone\bin\Debug\DiskClone.exe
    C:\temp.jwp\discutils\Utils\DiskDump\bin\Debug\DiskDump.exe
    C:\temp.jwp\discutils\Utils\ExternalFileSystem\bin\Debug\ExternalFileSystem.exe
    C:\temp.jwp\discutils\Utils\FileExtract\bin\Debug\FileExtract.exe
    C:\temp.jwp\discutils\Utils\FileRecover\bin\Debug\FileRecover.exe
    C:\temp.jwp\discutils\Utils\iSCSIBrowse\bin\Debug\iSCSIBrowse.exe
    C:\temp.jwp\discutils\Utils\ISOCreate\bin\Debug\ISOCreate.exe
    C:\temp.jwp\discutils\Utils\MSBuildTask\bin\Debug\DiscUtils.MSBuild.dll
    C:\temp.jwp\discutils\Utils\NTFSDump\bin\Debug\NTFSDump.exe
    C:\temp.jwp\discutils\Utils\OSClone\bin\Debug\OSClone.exe
    C:\temp.jwp\discutils\Utils\VHDCreate\bin\Debug\VHDCreate.exe
    C:\temp.jwp\discutils\Utils\VHDDump\bin\Debug\VHDDump.exe
    C:\temp.jwp\discutils\Utils\VirtualDiskConvert\bin\Debug\VirtualDiskConvert.exe
    C:\temp.jwp\discutils\Utils\VolInfo\bin\Debug\VolInfo.exe

running the VirtualDiskConvert on Windows

C:\Users\developer\Desktop\Disk-Images>\bin\DiscUtils\VirtualDiskConvert.exe -outputFormat VMDK-fixed CSBOOT.IMG CSBOOT.vmdk
VirtualDiskConvert v0.11.0, available from http://discutils.codeplex.com
Copyright (c) Kenneth Bell, 2008-2011
Free software issued under the MIT License, see LICENSE.TXT for details.

Progress (100%)  |===============================================| 00:00:00.0
C:\Users\developer\Desktop\Disk-Images>dir CSBOOT*
...
04/10/2013  09:57 PM        67,125,248 CSBOOT-flat.vmdk
04/21/2011  06:41 PM        67,125,248 CSBOOT.IMG
04/10/2013  09:57 PM               471 CSBOOT.vmdk

running the VirtualDiskConvert with Mono on a Mac

Jeroens-MacBook-Pro:VM jeroenp$ mono ../Dropbox/bin/DiscUtils/VirtualDiskConvert.exe -of VMDK-fixed ./CSBOOT.IMG CSBOOT.vmdk
VirtualDiskConvert v0.11.0, available from http://discutils.codeplex.com
Copyright (c) Kenneth Bell, 2008-2011
Free software issued under the MIT License, see LICENSE.TXT for details.

Unhandled Exception: System.IO.DirectoryNotFoundException: Could not find a part of the path "/Users/jeroenp/VM\/CSBOOT.IMG".

What probably happens here is that DiscUtils uses the wrong System.IO.Path.DirectorySeparatorChar Field.

compile.ps1 script

Put the compile.ps1 script in the same directory as the DiscUtils.sln solution, then run the script so the solution gets built by the most current version of msbuild.

Based on This is just a little trick I use to find InstallUtil and MsBuild, and make sure I’m using the latest version of them by Joel Bennet.

### http://poshcode.org/896

## Because of Split-Path, I get the "Framework" folder path (one level above the versioned folders)
$rtr = Split-Path $([System.Runtime.InteropServices.RuntimeEnvironment]::GetRuntimeDirectory())

## Then I loop through them in ascending (numerical, but really ascii) order
## each time I find installutil or mdbuild, I update the alias to point at the newer version
foreach($rtd in get-childitem $rtr -filt v* | sort Name) {
   if( Test-Path (join-path $rtd.FullName installutil.exe) ) {
      set-alias installutil (resolve-path (join-path $rtd.FullName installutil.exe))
   }
   if( Test-Path (join-path $rtd.FullName msbuild.exe) ) {
      set-alias msbuild (resolve-path (join-path $rtd.FullName msbuild.exe))
   }
}

msbuild DiscUtils.sln | out-null
if(-not $?)
{
  Write-Host "msbuild of DiscUtils.sln Failed"
  Exit
}

copyAllUtils.ps1 script

This script uses quite a few tricks, so here are some links for further reading:

PowerShell relies heavily on piping the output of one cmdlet to another. The reason is that the pipeline system does not only support text, but any object or collection of objects.

That the pipelining and filtering system is one of the reasons PowerShell is becoming more and more popular.

Three very important aspects of the pipeline:

  • Where-Object (aliased by ? and where) filters the pipeline.
  • Inside a pipeline, $_ holds the current active object.
  • ForEach-Object receives a collection and unwraps it for you.

Actually, there are both the foreach keyword for use without a pipeline, and the ForEach-Object cmdlet inside a pipeline. They work in a similar way, but there are a few strange things with the foreach keyword. To name a few:

ForEach-Object and foreach are so important that dozens of articles and book fragments have been devoted to it. Read a few of these to get a feel:

Whereas the pipeline does implicitly loop, ForEach-Object loops explicitly. Oh, and there is the difference between the foreach keyword and ForEach-Object cmdlet.

New-Item can create both a new file or a new directory.

When the file or directory exists, it will bail out. There are two alternatives:

  • use the -force parameter with New-Item
    New-Item $AllUtils -type directory
  • use the .NET framework
    [System.IO.Directory]::CreateDirectory($AllUtils)

Actually, there are 2 more ways of creating directories as described by Learn Four Ways to Use PowerShell to Create Folders – Hey, Scripting Guy! Blog.

Remove-Item can remove a directory or files inside the directory (there even are options -recurse and -exclude, and even -include – which is default). For safety, you can use the -whatif option so it only shows what it would do without the -whatif option.

Get-ChildItem gets you items from a directory. Like Remove-Item, there is are options for -recurse, -exclude and -include.

Since we want to copy both EXE and DLL files we need to pass an array. For this, PowerShell as the @() array subexpression. Some other expressions are grouping and subexpressions. Read more about those at Powershell Concatenation by Akim and at Effective PowerShell Item 10: Understanding PowerShell Parsing Modes | Keith Hill’s Blog.

Though PowerShell is line oriented, you hardly need to use the line continuation character `

Copy-Item copies files to a target. Like New-Item, it has the -force option so you can overwrite existing files. And it supports wildcards too.

By default PowerShell sends the result of an expression to StdOut.

If you want to write your own output, you can for instance use Write-Host.

These posts show various other ways of emitting output:

One of my ideas was to keep a System.Collections.Generic.Dictionary<TKey, TValue> mapping between files to be copied and their target, then only copy the non-duplicate targets.

There are two reasons I didn’t do this:

For the current solution, I needed to start with an empty array. That is unusual in PowerShell (99% of the times you start with an array that already has values). But it turned out to be remarkably simple as shown by Coretech Blog » Blog Archive » PowerShell: How to create an empty array!.

Arrays in PowerShell are strange anyway, as they

  • can contain a plethora of object types
  • can be added to (i.e. they behave more like a list of objects than an array of objects)
  • are mutable (so you can change values)

Read more about that on the “Hey, Scripting Guy!” blog: Add, Modify, Verify, and Sort Your PowerShell Array – Hey, Scripting Guy! Blog.

PowerShell has many comparison operators, some support regular expressions for matching.

I used the -contains operator for checking if a file was already processed.

## Copy the compiled binaries to the AllUtils directory

$AllUtils = ".\AllUtils"

$AllUtilsDirectory = [System.IO.Directory]::CreateDirectory($AllUtils)

Remove-Item ($AllUtils + "\*.*")

$CopiedFiles = @()

Get-ChildItem ".\Utils\" -recurse -include @( "*.exe", "*.dll" ) |
  Where-Object { $_.Directory.BaseName -eq "Debug"  } |
  Where-Object { $_.Directory.Parent } |
  Where-Object { $_.Directory.Parent.name -eq "bin" } |
  ForEach-Object {
    $fileName = $_.Name.ToUpperInvariant()
    if ($CopiedFiles -contains $fileName) {
##      Write-Host $_.Name " already copied"
    }
    else {
      Copy-Item $_.FullName $AllUtils -Force
      $CopiedFiles += , $fileName
      Write-Host $_.FullName
    }
  }

Exit

Note that you can replace Where-Object with ? and ForEach-Object with %. Some people find that easier to read. I don’t.

## Copy the compiled binaries to the AllUtils directory

$AllUtils = ".\AllUtils"

$AllUtilsDirectory = [System.IO.Directory]::CreateDirectory($AllUtils)

Remove-Item ($AllUtils + "\*.*")

$CopiedFiles = @()

Get-ChildItem ".\Utils\" -recurse -include @( "*.exe", "*.dll" ) |
  ? { $_.Directory.BaseName -eq "Debug"  } |
  ? { $_.Directory.Parent } |
  ? { $_.Directory.Parent.name -eq "bin" } |
  % {
    $fileName = $_.Name.ToUpperInvariant()
    if ($CopiedFiles -contains $fileName) {
##      Write-Host $_.Name " already copied"
    }
    else {
      Copy-Item $_.FullName $AllUtils -Force
      $CopiedFiles += , $fileName
      Write-Host $_.FullName
    }
  }

Exit

–jeroen

via: .NET DiscUtils – Wikipedia, the free encyclopedia.

One Response to “.NET/C#/PowerShell: building .NET DiscUtils library for virtual disk images”

  1. […] I should emphasize the importance of the object pipeline even more than I already did. […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

 
%d bloggers like this: