# Tales from the trenches: resizing a Windows Azure virtual disk the smooth way

We’ve all been there. Running a virtual machine on Windows Azure and all of a sudden you notice that a virtual disk is running full. Having no access to the hypervisor nor to its storage (directly), there’s no easy way out…

Big disclaimer: use the provided code on your own risk! I’m not responsible if something breaks! The provided code is as-is without warranty! I have tested this on a couple of data disks without any problems. I've tested this on OS disks and this sometimes works, sometimes fails. Be warned.

When searching for a solution to this issue,the typical solution you’ll find is the following:

• Delete the VM
• Delete the original .vhd from blob storage
• Recreate the VM
• Use diskpart to resize the partition

That’s a lot of work. Deleting and re-creating the VM isn’t that bad, it can be done pretty quickly. But doing a download of a 30GB disk, resizing the disk and re-uploading it is a serious PITA! Even if you do this on a temporary VM that sits in the same datacenter as your storage account.

Last saturday, I was in this situation… A decision would have to be made: spend an estimated 3 hours in doing the entire download/resize/upload process or reading up on the VHD file format and finding an easier way. With the possibility of having to fall back to doing the entire process…

Being a bit geeked out, I decided to read up on the VHD file format and download the specs.

Before we dive in: why would I even read up on the VHD file format? Well, since Windows Azure storage is used as the underlying store for Windows Azure Virtual Machine VHD’s and Windows Azure storage supports byte operations without having to download an entire file, it occurred to me that combining both would result in a less-than-one-second VHD resize. Or would it?

Note that if you’re just interested in the bits to “get it done”, check the last section of this post.

## Researching the VHD file format specs

The specs for the VHD file format are publicly available. Which means it shouldn't be to hard to learn how VHD files, the underlying format for virtual disks on Windows Azure Virtual Machines, are structured. Having fear of extremely complex file structures, I started reading and found that a VHD isn’t actually that complicated.

Apparently, VHD files created with Virtual PC 2004 are a bit different from newer VHD files. But hey, Microsoft will probably not use that old beast in their datacenters, right? Using that assumption and the assumption that VHD files for Windows Azure Virtual Machines are always fixed in size, I learnt the following over-generalized lesson:

A fixed-size VHD for Windows Azure Virtual Machines is a bunch of bytes representing the actual disk contents, followed by a 512-byte file footer that holds some metadata.
Maarten Balliauw – last Saturday

A-ha! So in short, if the size of the VHD file is known, the offset to the footer can be calculated and the entire footer can be read. And this footer is just a simple byte array. From the specs:

Let’s see what’s needed to do some dynamic VHD resizing…

## Resizing a VHD file - take 1

My first approach to “fixing” this issue was simple:

• Write null values over it and resize the disk to (desired size + 512 bytes)
• Write the footer in those last 512 bytes

Guess what? I tried mounting an updated VHD file in Windows, without any successful result. Time for some more reading… resulting in the big Eureka! scream: the “current size” field in the footer must be updated!

So I did that… and got failure again. But Eureka! again: the checksum must be updated so that the VHD driver can verify the footer is valid!

So I did that… and found more failure.

## Resizing a VHD file - take 2

Being a persistent developer, I decided to do some more searching. For most problems, at least a partial solution is available out there! And there was: CodePlex holds a library called .NET DiscUtils which supports reading from and writing to a giant load of file container formats such as ISO, VHD, various file systems, Udf, Vdi and much more!

Going through the sources and doing some research, I found the one missing piece from my first attempt: “geometry”. An old class on basic computer principles came to mind where the professor taught us that disks have geometry: cylinder-head-sector or CHS information for the disk driver which can use this info for determining physical data blocks on the disk.

Being lazy, I decided to copy-and-adapt the Footer class from this library. Why reinvent the wheel? Why risk  going sub-zero on the WIfe Acceptance Factor since this was saturday?

So I decided to generate a fresh VHD file in Windows and try to resize that one using this Footer class. Let’s start simple: specify the file to open, the desired new size and open a read/write stream to it.

1 string file = @"c:\temp\path\to\some.vhd";
2 long newSize = 20971520; // resize to 20 MB
3
4 using (Stream stream = new FileStream(file, FileMode.OpenOrCreate, FileAccess.ReadWrite))
5 {
6     // code goes here
7 }

Since we know the size of the file we’ve just opened, the footer is at length – 512, the Footer class takes these bytes and creates a .NET object for it:

1 stream.Seek(-512, SeekOrigin.End);
2 var currentFooterPosition = stream.Position;
3
5 var footer = new byte[512];
7
8 var footerInstance = Footer.FromBytes(footer, 0);

Of course, we want to make sure we’re working on a fixed-size disk and that it’s smaller than the requested new size.

1 if (footerInstance.DiskType != FileType.Fixed
2         || footerInstance.CurrentSize >= newSize)
3 {
4     throw new Exception("You are one serious nutcase!");
5 }

If all is well, we can start resizing the disk. Simply writing a series of zeroes in the least optimal way will do:

1 // Write 0 values
2 stream.Seek(currentFooterPosition, SeekOrigin.Begin);
3 while (stream.Length < newSize)
4 {
5     stream.WriteByte(0);
6 }

Now that we have a VHD file that holds the desired new size capacity, there’s one thing left: updating the VHD file footer. Again, the Footer class can help us here by updating the current size, original size, geometry and checksum fields:

1 // Change footer size values
2 footerInstance.CurrentSize = newSize;
3 footerInstance.OriginalSize = newSize;
4 footerInstance.Geometry = Geometry.FromCapacity(newSize);
5
6 footerInstance.UpdateChecksum();

One thing left: writing the footer to our VHD file:

1 footer = new byte[512];
2 footerInstance.ToBytes(footer, 0);
3
4 // Write new footer
5 stream.Write(footer, 0, footer.Length);

That’s it. And my big surprise after running this? Great success! A VHD that doubled in size.

So we can now resize VHD files in under a second. That’s much faster than any VHD resizer tool you find out here! But still: what about the download/upload?

## Resizing a VHD file stored in blob storage

Now that we have the code for resizing a local VHD, porting this to using blob storage and more specifically, the features provided for manipulating page blobs, is pretty straightforward. The Windows Azure Storage SDK gives us access to every single page of 512 bytes of a page blob, meaning we can work with files that span gigabytes of data while only downloading and uploading a couple of bytes…

Let’s give it a try. First of all, our file is now a URL to a blob:

1 var blob = new CloudPageBlob(
2     "http://account.blob.core.windows.net/vhds/some.vhd",
3     new StorageCredentials("accountname", "accountkey));

Next, we can fetch the last page of this blob to read our VHD’s footer:

 1 blob.FetchAttributes();
2 var originalLength = blob.Properties.Length;
3
4 var footer = new byte[512];
5 using (Stream stream = new MemoryStream())
6 {
8     stream.Position = 0;
10     stream.Close();
11 }
12
13 var footerInstance = Footer.FromBytes(footer, 0);

After doing the check on disk type again (fixed and smaller than the desired new size), we can resize the VHD. This time not by writing zeroes to it, but by calling one simple method on the storage SDK.

1 blob.Resize(newSize + 512);

In theory, it’s not required to overwrite the current footer with zeroes, but let’s play it clean:

1 blob.ClearPages(originalLength - 512, 512);

Next, we can change our footer values again:

1 footerInstance.CurrentSize = newSize;
2 footerInstance.OriginalSize = newSize;
3 footerInstance.Geometry = Geometry.FromCapacity(newSize);
4
5 footerInstance.UpdateChecksum();
6
7 footer = new byte[512];
8 footerInstance.ToBytes(footer, 0);

And write them to the last page of our page blob:

1 using (Stream stream = new MemoryStream(footer))
2 {
3     blob.WritePages(stream, newSize);
4 }

And that’s all, folks! Using this code you’ll be able to resize a VHD file stored on blob storage in less than a second without having to download and upload several gigabytes of data.

## Meet WindowsAzureDiskResizer

Since resizing Windows Azure VHD files is a well-known missing feature, I decided to wrap all my code in a console application and share it on GitHub. Feel free to fork, contribute and so on. WindowsAzureDiskResizer takes at least two parameters: the desired new size (in bytes) and a blob URL to the VHD. This can be a URL containing a Shared Access SIgnature.

Now let’s resize a disk. Here are the steps to take:

• Shutdown the VM
• Delete the VM -or- detach the disk if it’s not the OS disk
• In the Windows Azure portal, delete the disk (retain the data!) do that the lease Windows Azure has on it is removed
• Run WindowsAzureDiskResizer
• In the Windows Azure portal, recreate the disk based on the existing blob
• Recreate the VM  -or- reattach the disk if it’s not the OS disk
• Start the VM
• Use diskpart / disk management to resize the partition

Here’s how fast the resizing happens:

Woah! Enjoy!

We’re good for now, at least until Microsoft decides to switch to the newer VHDX file format…

This is an imported post. It was imported from my old blog using an automated tool and may contain formatting errors and/or broken images.

Tags:

Updated:

#### 48 responses

1. January 8th, 2013

What an awesome piece of research and utility! Nice one Maarten.

2. January 11th, 2013

Nice utility. I would have expected the size to be in GB rather than bytes. Does anyone need that level of granularity when sizing VHDs?

3. January 11th, 2013

Good point. Guess I started with bytes since all internal stuff works in bytes as well. Feel free to create a pull request (should be an easy change)

4. January 14th, 2013

I see you already took care of this over the weekend - Thx

5. February 1st, 2013

Great utility - please update your screenshots - took a while before i realised that I was dealing in GB not bytes!
Really helped me when I tried to convert a blob to a disk
Add-AzureDisk : HTTP Status Code: BadRequest - HTTP Error Message: The VHD http://xxx.blob.core.win
dows.net/vhds/fixeddisk1.vhd has an unsupported virtual size of 53686402560 bytes. The size must be a whole number
(in MBs).

6. February 5th, 2013

The program crashes for me. I think its has something todo with my blob credentials. Is it the same email and password i use to login to portal for the blob creds?

7. February 15th, 2013

You are so seriously awesome for posting this, it really saved me. I can&#39t imagine why MS would think it reasonable to default the C drive to 30 gigs when Windows takes up 20+ gigs out of the box... so stupid. I wonder how many fire drills this has caused... Well anyways I owe you a beer if you&#39re ever in South OC!

Cheers!

8. March 7th, 2013

I keep geting "The specified storage credentials are invalid". Did anyone come accross that and how did you fix it? My blob url does not include SAS. So I am typing the storage account name, the Storage access key as well as the full blob url to my OS disk.

9. March 7th, 2013

Steve, Have you gotten any reply on why the crash?

10. March 7th, 2013

That&#39s right Matt! But unfortunately it keeps rejecting my credentials. Which ones are you using?

11. March 8th, 2013

Just checking: are you calling WindowsAzureDiskResizer.exe <size> <blob> < account> <key> ? E.g. WindowsAzureDiskResizer.exe 60 "http://blob.core....." "foo" "HDHDHDHDHDGD=" ?

12. March 8th, 2013

Hi Maarten, Thank you!
I don&#39t know exactly why it happened but I finally got everything working following the syntax. So sweet! I resized my VHD from 30 to 50.

[b]Can&#39t comprehend how Microsoft managers sometimes make their decisions. [/b]

13. March 29th, 2013

You&#39re a hero!

14. June 22nd, 2013

You sir have just saved me a TON of time. Thank you!!!

15. July 8th, 2013

Awesome! Great job figuring this out!

16. July 16th, 2013

This is amazing, you're a gentleman and a scholar. I think the max OS disk size now in GA is 127GB; I attempted to expand one beyond this and was presented with a warning. I'm not sure if this is your code or the Azure API.

17. July 17th, 2013

Yeah now you can use the tool to shrink the disk if needed :-)

18. July 19th, 2013

Your my hero, so much time saved!

19. July 29th, 2013

Could not get the resizing done. Tried to resize a 30 Gbyte VM disk with Windows 2008 operating system. Have created URL with SAS, always get the message "The given disk size exceeds 127 GB. Azure will not be able to start the virtual machine stored on this disk if you continue. Aborted".

I have tried various disk sizes from 64424509440 (60 GByte) down to 2147482648 Bytes (2 GByte) - always thesame result, even when I manipulate the SAS. Also tried with account name and account key directly (do I needtoput them in "")?

20. July 29th, 2013

Can you try specifying in GB? E.g. just "60" for 60GB instead of the full bytes?

21. July 29th, 2013

OK, I have tried "60" instead of 64......... - then the SAS was not accepted ( I have generated the SAS with UTC time). After generating the SAS with local time the resize toolwent trough. Great & thank you for your rapid help. NowI try to mount the disk again and see what's happening...
This is the trial - ifit succeeds, then I will try the "hot" system.
Many thanks - savedalot of my time!

22. July 29th, 2013

23. July 29th, 2013

OK, I have tried "60" instead of 64......... - then the SAS was not accepted ( I have generated the SAS with UTC time). After generating the SAS with local time the resize toolwent trough. Great & thank you for your rapid help. NowI try to mount the disk again and see what's happening...
This is the trial - ifit succeeds, then I will try the "hot" system.
Many thanks - savedalot of my time!

24. July 30th, 2013

Maarten, it also worked on the "hot" system perfectly.Many thanks - saved me a lot of time.
Do you think it is also possible to write a tool that resets e.g. the RDP port (3389) ? Sometimes it happens that you exclude yourself blocking this port and then you cannot access the VM again. The standard procedure is the same as for "resizing OS disk" - shut down and delete VM, delete disk, download disk, re-configure vhd using a vm tool, etc. ...

25. July 30th, 2013

You can set the port through the management portal, no? Or can you give a detailed example of this?

26. July 30th, 2013

Hope this is more clear now.

27. July 30th, 2013

That would be really hard to do I'm affraid. The only option is to also enable PowerShell remoting in the VM and change the RDP port using PS remoting.

28. August 14th, 2013

"The specified VHD blob is larger than the specified new size. WindowsAzureDiskResizer can only expand VHD files." awwww needed to downsize a VHD from 127GB to something smaller. :-(

29. August 14th, 2013

Try compiling the latest sources, they can shrink as well.

30. August 22nd, 2013

Yea i tried the latest version and it was shrinkable, thanks. However it is definitely a unsafe operation as the new VM setup based on the shrunk VHD was not able to properly boot up the OS; cannot RDP into it, let alone the custom app services it is supposed to run.

31. August 22nd, 2013

Did you first shrink the partitions of the disk as outlined on https://github.com/maartenb... ?

32. August 22nd, 2013

Ah i did not notice that part; just downloaded, compiled, and executed it. Anyway we don't have much more time to conduct additional experiments on this aspect (have deleted the test set), so we'll just leave the VHDs at stock 127GB size since Windows Azure won't charge for unused blob pages. Downloading them will be a lengthy chore though. Thanks.

33. August 22nd, 2013

Shrinking partitions beforehand is the key to success, just shrinking the disk basically kills the file system as it potentially misses partition tables.

34. October 15th, 2013

Maarten, you have saved my day a second time. Last time I could increase the VM disk sucessfully, now I had to shrink one. Microsoft has increased disk size in their virtual machines (VMs) to 127 GByte! On the one hand this is plenty of space for almost all applications. However VHDs with this size are tedious to move.
I compiled the new version of WindowsAzureDiskResizer and could resize my VHD from 127 GByte down to 38 GByte. It took two trials - I got error messages while running the tool. I think I forgot to generate an URL with SAS with Write Premissions. There was an exception that has not been caught - maybe you can add catching the exception in the case that the writing permission is missing in the SAS string. I also had to put the URL including the SAS between quotations ("") to get it interpreted correctly. Without I got errors.
Many thanks - now I can move the VHD at reasonable times and efforts.

35. November 26th, 2013

Worked like a charm. Thanks for the saved time!

36. December 28th, 2013

Worked perfectly for me, thanks a lot!

37. November 18th, 2014

How do you get the URL containing the Shared Access Signature? I cannot see this anywhere in the portal. Thanks.

38. November 18th, 2014

In the end, I generated the Shared Access Signature using this tool: https://app.cloudportam.com...

BEWARE: I also found the instructions unclear regarding the deletion of disks. If you do this on the wrong screen, you are not asked if you want to retain the data and you lose all your data. There are about 3 different places where you can delete a disk and it must be done from 'beneath' the VirtualMachine where you see the option to "Retain the .VHD"

39. February 23rd, 2015

Unhandled Exception: System.FormatException: The input is not a valid Base-64 string as it contains a non-base 64 charac
ter, more than two padding characters, or an illegal character among the padding characters.
at System.Convert.FromBase64_Decode(Char* startInputPtr, Int32 inputLength, Byte* startDestPtr, Int32 destLength)
at System.Convert.FromBase64CharPtr(Char* inputPtr, Int32 inputLength)
at System.Convert.FromBase64String(String s)
at Microsoft.WindowsAzure.Storage.Auth.StorageCredentials.UpdateKey(String keyValue, String keyName)
at WindowsAzureDiskResizer.Program.ResizeVhdBlob(Int64 newSize, Uri blobUri, String accountName, String accountKey) i
n d:\Projects\Git\WindowsAzureDiskResizer\src\WindowsAzureDiskResizer\WindowsAzureDiskResizer\Program.cs:line 82
at WindowsAzureDiskResizer.Program.Main(String[] args) in d:\Projects\Git\WindowsAzureDiskResizer\src\WindowsAzureDis
kResizer\WindowsAzureDiskResizer\Program.cs:line 73

I was taking a 87 gig VHD to 90 (Hyper-V Inspect Disk on the original VHD states 87.72gb Maximum Disk Size 232.89gb).

40. February 23rd, 2015

Think that there's something wrong with your storage account key then, as that's where (according to the stack trace) the error occurs.

41. July 12th, 2015

Great utility! Worked exactly as described. Saved me a lot of time. Keep improving!

42. September 10th, 2015

Great utility - works exactly as described - saved me 20+ hours of extending the vhd and uploading again! Thank You!

43. November 10th, 2015

Really simple and easy to follow! However, the last step: "Use diskpart / disk management to resize the partition", how do you do that?

44. November 22nd, 2015

Thank you! Thank you! Thank you Save me 2 days of uploading

45. December 19th, 2015

Many thanks for this utility. It saved me lots of time and troubles.

46. May 14th, 2016

I tried to resize 118.71 GB VHD to 119GB or 125GB and each time I'm getting following error after running WindowsAzureDiskResizer tool:

Unhandled Exception: Microsoft.WindowsAzure.Storage.StorageException: The remote

server returned an error: (400) Bad Request. ---> System.Net.WebException: The

remote server returned an error: (400) Bad Request.

at System.Net.HttpWebRequest.GetResponse()

at Microsoft.WindowsAzure.Storage.Core.Executor.Executor.ExecuteSync[T](Stora

geCommandBase1 cmd, IRetryPolicy policy, OperationContext operationContext)

--- End of inner exception stack trace ---

at Microsoft.WindowsAzure.Storage.Core.Executor.Executor.ExecuteSync[T](Stora

geCommandBase1 cmd, IRetryPolicy policy, OperationContext operationContext)
at Microsoft.WindowsAzure.Storage.Blob.CloudPageBlob.Resize(Int64 size, Acces
sCondition accessCondition, BlobRequestOptions options, OperationContext operati
onContext)

at WindowsAzureDiskResizer.Program.ResizeVhdBlob(Int64 newSize, Uri blobUri,

String accountName, String accountKey) in d:\Projects\Git\WindowsAzureDiskResize

r\src\WindowsAzureDiskResizer\WindowsAzureDiskResizer\Program.cs:line 131

at WindowsAzureDiskResizer.Program.Main(String[] args) in d:\Projects\Git\Win

dowsAzureDiskResizer\src\WindowsAzureDiskResizer\WindowsAzureDiskResizer\Program

.cs:line 73

47. March 28th, 2017

You know when something works, and it happens exactly as you expect, and it's so fast, that you're like holy shit... yeah, this was a holy shit moment. Nice work, I just compiled from source with VS 2017 and used it to shrink a vhd so I can convert it to one of the new managed disks, and now I can pay less, because it fits in a smaller tier. :)

48. October 12th, 2018

Thank you Maarten, is still working like a charm. And also thank you for the info about VHD specs.
I will share my steps to shrink a 128gb disk (P10) to 64gb (P6 Free tier)
-Resize the disk in VM before everything using PowerShell (PS>Resize-Partition -DiskNumber 0 -PartitionNumber 1 -Size 64GB )

-Deattach VM and copy snapshot to storage account

-Update footer of the snapshot with diskresizer

-Make a new disk from snapshot

-Atacch new disk to VM and delete both old disk (original and snapshot)

Enjoy the free tier for a year... BTW without this is not posible to get REAL ACCESS to free tier, because default disk of VM are 128gb not covered by free tier, but if you resize to 64gb yo can do it for free.