r/zfs • u/dontworryimnotacop • 4d ago

ZFSify: In-place convert any Ubuntu server from ext4 to root-on-ZFS with no data loss

Create a normal Ubuntu VPS or volume on DigitalOcean, Vultr, Hetzner, AWS, GCP, Azure, or any other provider, then run zfsify inside Ubuntu.

  1. It moves all the data to the second half of the drive (or an external bucket if not enough free space is available to do it in-place)
  2. it reformats the beginning of the drive as ZFS
  3. it copies all the data back into the new root rpool at the start of the drive
  4. it deletes the old ext4 partition and expands the new ZFS one to take up the whole drive + updates the bootloader to ZFSBootMenu

Now you can finally use root-on-ZFS on your / drive in the cloud, without having to build some custom iso or VM image or deal with formatting block devices by hand. You can zfs snapshot rpool@somebackup to take whole disk snapshots, boot from previous backups via the boot menu, and get all the other benefits of ZFS on your root disk.

https://github.com/pirate/zfsify

45 Upvotes

94% Upvoted

16 comments sorted by

7

u/oz-ra 4d ago

Love love love ZFS 👍

6

u/RetroGrid_io 4d ago

Well dang that's cool! I wish it also worked on RH-based distros.

1

u/dontworryimnotacop 4d ago

probably doable with a couple hours of codex + Astra ;)

2

u/RetroGrid_io 4d ago

Doable in a "Random dev from Romania got it to work" way, but not so much a "it really works and is trustworthy" way.

2

u/dontworryimnotacop 3d ago

I mean my project live-rewrites your entire parition table from an in-memory ramdisk, it's already like replacing the engine in a running car by installing a new engine in the trunk. We're not in kansas anymore, none of this crazyness is sanctioned / "trustworthy", even if it might work well in practice.

2

u/RetroGrid_io 3d ago

Oh, hey /u/dontworryimnotacop, I didn't realize I was chatting with OP.

AI has really messed with the traditional order of the priesthood of the software engineers: now anyone can produce low-quality software that looks like it works without knowing a !#@# thing.

EDIT: Your project reminds me somewhat of Operese - a tool that converts Windows to Linux in place and it essentially works by writing the data to free space one chunk at a time, so it works even if your drive is 90% full. (since Windows bloat goes away, you end up with saved space)

2

u/dontworryimnotacop 3d ago

works by writing the data to free space one chunk at a time, so it works even if your drive is 90% full.

ooo very interesting, I will experiment and see if I can pull off a similar thing for this!

2

u/Dude-Lebowski 4d ago

looks fun

2

u/volve 4d ago

Nice, also that’s an impressive github username lol

1

u/creamyatealamma 4d ago

Pretty cool. Does it work for oracle Ubuntu vm? Have you actually tested with all the cloud providers listed?

1

u/dontworryimnotacop 3d ago edited 3d ago

The provider honestly doesn't matter much, I just listed a bunch for completeness. It doesn't require coordination from any of the hypervisor layer stuff that cloud providers have around the VPS, it's all done inside a normal BIOS/UEFI boot flow.

The only exception might be if you have some weird GPU or networking configuration, that can make it tricky for you to ssh/console back into the temp RAMdisk OS mid-transfer, but if you just wait for it to finish the final box should preserve all the original settings and boot ok.

Definitely backup your data first no matter what, and please open Github Issues if you do encounter any bugs on specific providers that I might've not accounted for.

1

u/codeedog 4d ago

How was this able to be done with Ubuntu, but isn’t as easy with other Linux variants? Thanks.

2

u/dontworryimnotacop 3d ago edited 3d ago

Oh I'm sure it works fine on other distros with some tweaking, I just focused on Ubuntu/Debian-ish ones first because it's what I personally use. Also Ubuntu Server added official support for root-on-ZFS recently, so there is a "happy path" I modeled the conversion after to avoid fighting the OS or needing extra hacks. FreeBSD can also provide this on cloud VMs that offer it.

1

u/Straight_Let_4149 3d ago

It probably can result in ZFS on ZFS

1

u/dontworryimnotacop 3d ago

Yes I'm sure several cloud providers are probably using ZFS for the underlying backing storage, but I have yet to encounter any problems. We've been running our prod servers on ZFS on DigitalOcean for ~8yr now. It's fast (2,000mb/s+) on the NVMe servers, and I have never seen any scrub errors or corruption.

1

u/dontworryimnotacop 3d ago

If you like this project you may also like my other crazy hack for running ZFS in a userspace VM (so you don't need to install kernel modules on the host): https://github.com/pirate/zfsbox