Sunday 7 March 2021

Microsoft Dynamics 365 Finance - OneBox Virtual Machine - How to download VHD files fast with AzCopy.exe

 

Background

It's possible to download a complete working Microsoft Dynamics 365 Finance virtual machine (VM) called OneBox from the LifeCycle services (LCS) asset library. By the way, this VM will only run with Hyper-V and will not run with other virtualisation software.

This VM can run locally on your companys infrastructure or even on your laptop provided you have a solid state disk (SSD) with enough space and your laptop has enough memory. My Dynamics 365 Finance VM takes up about 120GB of disk space on my external SSD drive.

I enabled Hyper-V on my laptop in Windows features and I am able to run the Dynamics 365 Finance VM with 24GB of RAM allocated to the VM and 8GB RAM for my Windows 10 host. The VM runs okay but it's definitely not suitable for full time development.

When I downloaded the 12 virtual hardisk (VHD) part files for the first time using with my browser, I discovered that it took a long time to download the files and many of the downloads failed. I had to retry them and it was just painful. Each file is about 3 GB. The separate part files, or volumes, will become one large VHD file when executing the first part file which is an executable.

In the LCS in asset library, on each downloadable file, I saw a button with the text "Generate SAS link". When I clicked on it, I got a message that the "SAS link for the asset has been copied to my clipboard and that I can use AzCopy to download the the asset".

I researched AzCopy a bit and downloaded a copy from the Microsoft site to my laptop and extracted it to a folder.

AzCopy can download files much faster than a browser from BLOB storage would because it uses multi threading to download different parts of the file at the same time. A browser uses a single thread per file being downloaded.

You can compare it to a race to empty two identical swimming pools, containing exactly the same amount of water manually just using buckets. With the first swimming pool, there is just one person emptying the pool but with the other, there are ten persons. Not really fair is it? But the ten persons will empty the pool a lot faster than one right?

I selected the first VHD file and clicked on the "Generate SAS link" button. I pasted the link to a text editor to create a command for AzCopy. I  opened a command prompt and downloaded the first file using AxCopy.

To make things easier, I got the idea to add the separate AzCopy command lines in a PowerShell script, one command for each file part . The PowerShell script can download all the part files one after each the other.


Create PowerShell script

  1. Download azcopy and extract it in a folder somewhere like c:\azcopy.
  2. Create a new text file called "downloadvhd.ps1" in the same folder as where azopy.exe is located.
  3. Log in to LCS and go to the Asset Library of your project.
  4. Select the "Downloadable VHD" asset type on the left. If you don't have any part files listed on the right, import them from the Shared Library.
  5. Select the first Part01 VHD file and click on the "General SAS link" button for that file. This will copy the link to  the clipboard.
  6. Open the file created in step 2. with a text editor. On the first line, type: .\azcopy.exe copy '
  7. After the aposthophe, paste the SAS link from the clipboard by pressing Control + V or Paste from the text editor menu. Make sure there is no space between the aposthrophe and the SAS link.
  8. The first part file is an executable file, the rest of the part files are RAR archive volume files with the .rar extension. After the SAS link, type a closing aposthrophe a space and the destination file name e.g.: ' c:\temp\FinandOps10.0.13.part01.exe
  9. For each part file in LCS, create a line in the script file, their file names will be FinandPos10.0.13.part02.rar, FinandPos10.0.13.part03.rar and so on. Important: Click on the "Generate SAS link" button for each file and paste it in the file for the corresponding part file.
  10. When all part files command are in the script file, save it and execute it.

Example PowerShell file

Replace SAS-link-for-Partxx with your own SAS links and change the file names.

.\azcopy.exe copy 'SAS-link-for-Part01' C:\temp\FinandOps10.0.13.part01.exe
.\azcopy.exe copy 'SAS-link-for-Part02' C:\temp\FinandOps10.0.13.part02.rar
.\azcopy.exe copy 'SAS-link-for-Part03' C:\temp\FinandOps10.0.13.part03.rar
.\azcopy.exe copy 'SAS-link-for-Part04' C:\temp\FinandOps10.0.13.part04.rar
.\azcopy.exe copy 'SAS-link-for-Part05' C:\temp\FinandOps10.0.13.part05.rar
.\azcopy.exe copy 'SAS-link-for-Part06' C:\temp\FinandOps10.0.13.part06.rar
.\azcopy.exe copy 'SAS-link-for-Part07' C:\temp\FinandOps10.0.13.part07.rar
.\azcopy.exe copy 'SAS-link-for-Part08' C:\temp\FinandOps10.0.13.part08.rar
.\azcopy.exe copy 'SAS-link-for-Part09' C:\temp\FinandOps10.0.13.part09.rar
.\azcopy.exe copy 'SAS-link-for-Part10' C:\temp\FinandOps10.0.13.part10.rar
.\azcopy.exe copy 'SAS-link-for-Part11' C:\temp\FinandOps10.0.13.part11.rar

Syntax

azcopy.exe copy [source] [destination]

Example:
azcopy.exe copy 'SAS-link-for-Part01' C:\temp\FinandOps10.0.13.part01.exe

Notes

The first part file is an executable file which will extract and combine all the part files into a single VHD file.

You have to check the file names in LCS and adjust the file names  in the script accordingly to match the version in LCS. Don't use my example version.

The SAS link is enclosed in single apostrophies. If the file names contain spaces, then the file name part must also be enclosed with single apostrophies.

Download in progress

Download VHD files from LCS with AzCopy