Exporting Digital Ocean images/droplets to Xen, KVM, VMware, or $other
Digital Ocean is great. Its really fast, simple & cheap. Backups are simple and everything just works. However you can’t export backups or snapshots to a non digital ocean host.
This is a pain. Sometimes you want to export a small machine to some other service, or more likely you want to keep a backup image on another service other than Digital Ocean.
Here is a very dirty way of exporting your droplets.
Warnings:
- This is a dirty block rip. You *MUST* make sure that you have a way of verifying the end product.
- All services barring the network *must* be turned off
- Don’t try this on something critical
First steps:
Having turned off all unnecessary services (bonus points for bringing up your droplet in single user mode and enabling networking with ifup) Log in as root. Our method of block ripping is DD.
if=/dev/vda | gzip -1 | ssh your.server.com dd of=/mnt/storage/imagename.img.gz
This is where the magic happens, DD is reading the block device directly, bit by bit. Normally there is a lot of empty zeros in a machine image. To save upload time, we use gzip on its lowest setting to remove contiguous lumps of zeros.
Its then forced over ssh where dd then outputs the stream into a file. There is nothing stopping you using pigz to get faster compression with multicore machines. You can optionally decompress on the receiving end as well.
Recovering the Image
You now have an image, but it’ll not boot. turds. This is because your droplet has an old version od grub installed. I suspect this is to allow Digital Ocean to do something awesome.
For this step you’ll need:
- A working modernish linux
- x86 chipset
- internets
For the next steps I’m going to assume that you’ve presented the droplet imaged to a machine on /dev/sdb (use fdisk -l to find out where it is for you. if you can’t find it, stop.)
Chroot, its awesome.
What we need to do is install grub2, and a new MBR onto your droplet image. First we need to mount the image:
mount /dev/sdb1 /disk
Once again, make sure thats the right disk. If you get a message about it already being mounted, think very carefully before proceeding.
mount --bind /dev /disk/dev
Make it look like /dev/ also appears inside the mounted droplet
chroot /disk
Now, this is magical. You’ve now effectively started a new bash as the droplet. (yes, this is basically docker.) This means that you can apt-get update, even start a webserver. However you don’t want to do that. We’re now going to install grub2
apt-get update apt-get install grub2
Now you have grub, It’ll possibly ask you about chain-loading stuff. Don’t worry defaults are fine.
grub-install --force /dev/sdb
This will install the grub2 manager onto the MBR. You’ll get a warning about how its not got a proper partition layout. But ignore that, this is the CLOUD MAN.
Now feel free to attach the image into it’s own VM. You should be able to run it as normal. (if your not using KVM you might have a few issues with fstab and such, but nothing you can sort out in the chroot)
Bonus Networking issues
If your Droplet had a private networking interface, you might have a few issues with Eth0 attaching to the wrong nic. This is because udev is not correctly setup. You’ll need to manually assign adaptors to mac addresses in:
/etc/udev/70-persistent-net-rules.conf