Sunday, September 07, 2008

Virtualbox: Expanding A Disk Drive

I love Virtualbox, it's definitely my virtualization platform of choice for daily tasks* on a Linux-host, and it's quickly replacing VMware on Windows hosts as well.

However, recently I had to expand a virtual partition of a Windows guest. There are already lots of options and methods available to do this:

None of these actually offered a really good or quick solution, until I found this blog post. Tom's statement of hostility to copyright allows me to provide a summary... However, I've added some extra steps and tips for both Windows and Linux guests.

You’ll need:
  • A Gparted LiveCD.
  • Two virtual disk images: the old one (the one you're currently using, the small one), and a new one, which you can easily create in Virtualbox, make sure it's "raw" (unpartitioned).

1. Mount or burn the Gparted disk in Virtualbox so that it'll boot from it (make sure the "virtual bios" boots from it first).

2. Using VirtualBox, create a new VDI with a larger size that you want to expand the virtual machine to (see: "You'll need").

3. Leave your existing VDI as the primary IDE master. Set the new VDI to be the Primary IDE slave for the machine.

4. In Gparted: run fdisk -l or use Gparted GUI to view your partitions. You should get /dev/hda and /dev/hdb. /dev/hdb shouldn’t be partitioned at this point. /dev/hda should be your primary master (the original VDI that is too small for your needs), and /dev/hdb should be the new image (virtual primary IDE slave).

5. Do not mount those (yet). Instead, use dd to copy the old image to the new one:
dd if=/dev/hda of=/dev/hdb
Warning: don't despair, this may take some time, and the hard drive status icon of Virtualbox should indicate that something's happening.

Steps for Linux guests:

1. Launch GParted, and resize the partition on /dev/hdb (the new image) to take up the entire disk. This may take awhile. If you cannot move your partition because the swap partition is in the way, first remove the swap, then add a new swap partition add the end of the hard drive (linux-swap), and then extend the non-swap partition. The following screenshot shows how it should look after recreating the swap partition, right before extending the data-partition:

3. Extra step (update - thanks to comments): modern Linux distributions use disk IDs to identify disks instead of sda, sdb,... This might give you the following message in a few steps: "Waiting for device /dev/disk/by-id/scsi-SATA_VBOX_HARDDISK_VB... to appear", like the user in the comments of this post. So before you continue with the following steps, open a Terminal window.

I'm going to create two mount points for the old and new partition and mount them. You can use GParted to find out the exact device location (the new partition should probably be /dev/hdb1):
mkdir /mnt/old_disk
mkdir /mnt/new_disk
mount /dev/hda1 /mnt/old_disk
mount /dev/hdb1 /mnt/new_disk

Screenshot:

I'm now going to take a peek at the fstab-file on my old partition:
cat /mnt/old_disk/etc/fstab

With an Ubuntu partition, you get something like this:

Ubuntu uses UUIDs to identify partitions. We'll have to edit our new partition's fstab file with the new UUIDs. Your distribution could use:
  • Device locations, like /dev/sda1 (yes, you see this in my fstab file as well, but everything after a pound ('#') character is a comment, so it's not really using that). You can find out your new locations using GParted. Make sure you only change the numbers if you have to! So don't change sda to sdb or hdb! GParted might be showing you /dev/hdb, but when we'll make our new partition the master VDI, your Linux installation will see this as /dev/sda!
  • IDs, not common, you can find the new IDs by using ls /dev/disk/by-id -lah.
  • Paths, not common, you can find them by using ls /dev/disk/by-path -lah.
  • UUIDs (like me), more and more common. You can find your new UUIDs by using ls /dev/disk/by-uuid -lah. The new UUIDs will be listed next to ../../hdb-lines (remember: hda is our old hard disk).

Now let's edit our new fstab. Open a new Terminal window (so you can still see the UUIDs) and enter:
nano /mnt/new_disk/etc/fstab

Nano is a Terminal text editor. Just use it like Notepad, use CTRL+O to save your changes, and CTRL+X to exit. You can use it to edit your UUIDs.
Note: make sure you leave the proc-line intact! Also, note that the cdrom-line doesn't use an UUID but /dev/scd0.
Extra note: the data-partition will probably have the same UUID as before (we have only extended it). The swap partition has a new UUID (because I deleted and recreated it).

Following screenshot shows my edited fstab, I have removed the comments.

4. All done? Good. Poweroff with the GParted LiveCD.

5. Finally, set the new image (resized) as the primary master for the virtual machine. Uncheck the primary slave. Boot into the machine to verify that it worked. If everything works you may delete the small disk VDI.

Screwed? Then make the small image master and the large one slave again, and use GParted with nano to double check. Don't be afraid making a mistake, if you only edit the large VDI, you can always rollback to your old small one and start over.

Steps for Windows guests:

1. Do not use gparted or ntfsresize to resize the partition. I know you can, but I found it a tad risky. Instead, set the new image as the primary master for the virtual machine, uncheck the primary slave. Boot back into Windows.

2. You should be back into Windows, but still on a small partition. If you use Disk Management in Administrative Tools you should see that there is a lot of unallocated space now on the virtual hard disk, how can we expand our drive?

3. Open a cmd. Type diskpart.exe, this is a tool that comes built in with most Windows distributions.

Type extend and press enter. This will extend your partition. The full command is extend [size=n] [disk=n] [noerr], but we can leave all options to their defaults.

4. Once this is done, use chkdsk to make sure everything is okay.

*For server-oriented platforms, Xen is the undisputable champion.

28 comments :

  1. hi
    hello
    how was your day?
    i liked your blog
    you are fantastic!!!

    really nice blog
    fabulous fantastic
    bye
    take care
    see you

    ReplyDelete
  2. Hey there! Is seems that GParted can't recognize both partiotions. If i boot from the primary ide gparted only recognizes hda, else only hdb (the new unpartitioned one). Any ideas?

    ReplyDelete
  3. extending volumes doesnt work on system volumes or volumes containing page files...

    http://support.microsoft.com/kb/325590

    ReplyDelete
  4. @Anonymous: are you sure you are setting up both master and slave VDI's? (As in step 3?) And not using just one master?

    @Spud: true, the KB says: "System or boot volumes may be blocked from being extended, and you may receive the following error". It did work in my case, but it may not work in yours. You can still try to use the *NIX ntfsresize tool, then, but save your old (small) VDI for when things don't work out!

    ReplyDelete
  5. Thank you for posting these instructions. I believe I followed them as described, but I have a problem when I try to boot off the new VDI. After I remove the original VDI and make the new VDI the Primary Master, the boot sequence starts, but it fails when
    "Waiting for device /dev/disk/by-id/scsi-SATA_VBOX_HARDDISK_VB... to appear". It asks me if I want to fall back to the same disk ID. Answering Y or N results in the same failure, leaving me at a shell in the RAM disk. Is there anything else I need to do to make the new VDI bootable?

    ReplyDelete
  6. @Anonymous: thanks for letting me know, I have updated the post with instructions to remedy this. I hope you read this, since I cannot contact you via mail. Recent Linux distributions use disk UUIDs in their /etc/fstab-configuration, which have changed in your new VDI. You have to edit /etc/fstab (in GParted for example, or the emergency shell) to recognize them.

    ReplyDelete
  7. for anyone who tried to boot their newly created partition, made it a primary master, and you get a black screen w/ cursor at virtualbox bootup, make sure your new partition has the 'boot' flag (can do this through the gparted gui)

    ReplyDelete
  8. @Spud:

    you are correct. There exists freeware you can download that will do it for you though. I suggest

    http://www.download.com/Easeus-Partition-Manager/3000-2248_4-10863346.html?tag=mncol

    which did it for me in about 3 mins time including reboots.

    ReplyDelete
  9. Thanks for the tutorial. Couple of notes on the diskpart.exe:

    - as others mentioned it cannot be used on the system partition. Simple solution: boot to your old VDI with the new disk image as secondary slave. Run extend on the new partition as a slave.

    - Before typing "extend", I had to "select" the disk and partition. Disks are numbered starting at 0, and partitions starting at 1, so with the drive mounted as primary slave, the commands were:
    select disk 1
    select partition 1

    It worked great!

    BTW, while gparted is a great disc to have around, just about any linux liveCD image will work. They've all got dd on them.

    ReplyDelete
  10. None of the described uses of diskpart worked for me but Easeus-Partition-Manager did. Using dd to clone harddrive partitions is and old (but good) trick. I guess any disk imaging system should work provided that you do enough fiddling with the partitions later. Watch out for Windoze tricks for calling your slave partition c: and your master e:!

    ReplyDelete
  11. Indeed, Easeus is another good alternative, and I would certainly recommend it to anyone using Windows. (It's one of the best partition managers around, forget Partition Magic.)

    It's a pity they don't offer a free version for 64bit platforms though.

    ReplyDelete
  12. I used system rescue (latest version). That is the way to go, but I would still recommend using GParted even for windows. You may have to run check disk from windows go back to a system rescue and run GParted to expand the partition. It worked great, where the winblooze utility diskparted was unable to expand the partition (kept erroring out). Needless to say I bow to the power of the Penguin!!!

    ReplyDelete
  13. I too found that the diskpart utility didn't work. I am using Windows XP Pro 32-bit I ran diskpart, entered "select disk 0", then "select partition 1" to pick my only disk/partition (Use "list disk" and "list partition" to see your numberings). When I tried to use "extend", it told me that it could not do it, and to make sure my partition was valid for extension. Looks like I'll be using gparted again... I've used it to resize a number of NTFS partitions successfully in the past, I just thought it would be a good idea to go the MS way. Guess I was wrong. Windows should run checkdisk automatically on boot after messing with it in gparted, but it should be ok after that.

    ReplyDelete
  14. works great - thank you. been wanting to do this for quite a while.

    ReplyDelete
  15. Thank you for this! I had one problem though... Tried to run the DD command (from /root of the RescueCD ISO) and got this error:

    dd: opening '/dev/hda': No such file or directory

    GPARTED shows the partitions as /dev/sda and /dev/sdb, so I changed the command to use that and it worked OK.

    The next problem was the actual extending of the disk... DISKPART did not work, even when I made the new (larger) partition a slave. GPARTED would not extend it either. I ended up using VHD Resizer to create a larger VHD, and then Rescue CD's GPARTED to extend it. Now I am in business.

    ReplyDelete
  16. Easy solution:

    Create a new virtual disk, connect it as slave. Install programs to this drive :P

    ReplyDelete
  17. Your post was really helpful. I had a Xubuntu guest that ran out of space that really caused me some pain. It only took me an hour or so to get things right thanks to your post.

    ReplyDelete
  18. Hi all,

    If your root partition is lvm type and you are using LVM try this out:
    --------------------------------------------------------------------------------------------------------------------------------------------
    Use LiveCD with dd and fdisk utilities:
    ----------------------------------------
    1) dd if=/dev/hda of=/dev/hdb # /dev/hda - old disk with smaller size, /dev/hdb - new disk with bigger size
    2) fdisk -u /dev/hdb
    3) Steps from 2) to 6) (incl.) described here - http://www.linuxquestions.org/questions/fedora-35/lvm-partition-resizing-666683/#post3281556
    4) Shutdown the Virtual machine and remove the old disk and the LiveCD from the virtual CDROM

    After the shutdown, boot the system normally (only with the new disk and without the LiveCD):
    ---------------------------------------------------------------------------------------------
    5) pvdisplay -v # Checks the PV name (e.g. /dev/hda2)
    6) pvresize --verbose /dev/hda2
    7) vgdisplay -v # Checks the Free PE (e.g. 384)
    8) lvextend -l+384 /dev/VolGroup00/LogVol00
    9) resize2fs /dev/VolGroup00/LogVol00
    10) Наздраве!
    --------------------------------------------------------------------------------------------------------------------------------------------

    Tested with VM:
    CentOS release 5.4 (Final)
    kernel-2.6.18-164.11.1.el5
    lvm2-2.02.46-8.el5_4.2

    Don't do that on production systems but better create a new partition with fdisk and a new PV, extend the existing VG and LV after that (see http://www.linuxquestions.org/questions/fedora-35/lvm-partition-resizing-666683/#post3281556).

    Be sure that you have a backup!!!

    Best Regards,

    Boyan Boychev
    Bulgaria

    ReplyDelete
  19. Very useful, thank you very much!

    ReplyDelete
  20. Your blog is very good,very novel,let I learned a lot from,thank you very much.if you have time also attention our products.

    ReplyDelete
  21. A great blog can give one person new feeling, let person get happiness, let person broaden people’s horizons. I think so your blog that is. This topic is very interesting. I find that this site to be very informative.

    ReplyDelete
  22. Hi, I like your site, you have useful information even for me.

    ReplyDelete
  23. This is a good article. I agree, if there is a method of saving costs then I am all for it

    ReplyDelete
  24. For those who diskpart is not working, the same Gparted tool used at the beginning has the utility to grow NTFS

    ReplyDelete
  25. example command:
    VBoxManage modifyhd --resize 14336 win7.vdi

    ReplyDelete
  26. R4 cardthat are perfect to your ds liteM3i Zero

    ReplyDelete
  27. It's been a very long time since this post, and I wanted to point out that VirtualBox now has a command-line method to more easily resize a hard disk: VBoxManage modifyhd < path to the VDI file > --resize < size in megabytes >

    ReplyDelete