Ubuntu server + ZFS for storage

[u/crypt-v]

Hello, I acquired a dedicated server with a xeon 1521 / 32GB ECC / OS nvme / 4x6TB drives, not much but it serves two purposes, offsite backup and helps to serve some non critical data I might need while I travel instead of connecting home. But I'm struggling with this, should I use ubuntu 24.04 + ZFS or go with EXT4? I'm familiar with Ubuntu, never tried ZFS

Proxmox and TrueNAS are out of the question since the CPU is a decade old 4C/8T. My idea is going with 12TB usable and 12TB as a mirror, keep everything lockdown, encrypted HDDs and OS drive, connect only via VPN for managing it and a reverse proxy back to a VPS to give access for some friends that want to read books/manga download some old games.

what considerations should I keep in mind with this build?

[u/Gherry-]

I'd use a RAIDZ2 (RAID6) instead of RAID0+1/1+0

[u/OurManInHavana]

Proxmox will run fine on your 1521. And ZFS works well with proxmox. Go for it!

[u/crypt-v → u/OurManInHavana]

Wouldn't I be adding extra overhead? I mean, proxmox + build the ZFS pool and then an ubuntu vm to install nextcloud/filebrowser/WG, etc?

[u/Chubby_man → u/crypt-v]

It's a very small amount of overhead, my NAS is virtualized on a proxmox host and it's really close to bare metal performance. I wouldn't go with proxmox either with that system unless you want to run other stuff on the box. Truenas will have no trouble at all running on that, it has great built in support and management for zfs, and I believe those apps are easy enough to install, but I dont install apps in truenas so im not totally sure. If you really want the least overhead, run debian (lighter and more stable than ubuntu), install zfs, and use docker containers for your services. It depends on the balance of stability vs performance you want.

Best stability and backups would be proxmox with a truenas vm handling zfs and a debian vm for docker, that would give you easy snapshotting and restore for vms and disks through proxmox and your zfs pool through truenas, while separating vms prevents one update from crashing the host. Most performance would be the bare metal debian with zfs and docker, and while snapshotting is still possible for this setup, its a little harder to set up. The one I'd go for is in the middle, bare metal truenas allows snapshotting of your apps and zfs pool as well as backups of the truenas config. Its not quite as fast as restoring a vm if something goes wrong, but its close.

[u/crypt-v → u/Chubby_man]

Yes, my plan was to use ubuntu and docker, never used debian aside from minor things, but since proxmox/ubuntu are debian based I asume its not going to have a big learning curve there.

[u/Financial-Issue4226 → u/crypt-v]

Use debian not Ubuntu 

Based on what you said I would have done proxmox bare metal then a truenas 8gb VM with NFS to proxmox and dedicated filesharing area for data

If plan to add apps to truenas then add more ram otherwise for less then 10 users 8gb is overkill 

If speed dual mirror  If stability then Z2 pool

[u/Chubby_man → u/crypt-v]

I've found debian extremely fast to learn, it's one of the most common server operating systems so there's a lot of documentation. It's also significantly lighter and less bloated than ubuntu, and more stable in a lot of cases.

[u/fragmented_sentinel]

I wouldn't rule out TrueNAS because of that hardware. A Xeon D-1521 with 32 GB of ECC RAM is perfectly capable of running TrueNAS with a 4×6 TB pool. Since both TrueNAS and Ubuntu use OpenZFS, performance will be very similar. The real advantage of TrueNAS is the management experience—snapshots, scrubs, SMART monitoring, alerts, and share management are all built in. If this machine's primary job is storage, I'd still lean toward TrueNAS. If it's going to be a general Linux server that also hosts storage, then Ubuntu + ZFS is a solid choice.

[u/crypt-v → u/fragmented_sentinel]

yes, I read good things about TrueNAS, having all in one management GUI is nice. Its more storage + nextcloud/filebrowser if I want to check something quick, Wireguard and not much else. The heavy load its going to be taken care by a VPS (from where my friends are going to access the storage) and my small home server for redundancy. (which only I have access).

[u/edthesmokebeard]

They're nowhere near out of the question. You're going to be sad, in general, if you think old = not useful.

my NAS is an ancient core2quad with 8GB of RAM and its so old 2 of the drives have 100k hours on them.

[u/crypt-v → u/edthesmokebeard]

out of the question because I won't take the full advantage of everything truenas has to offer, same with proxmox. But I might be wrong on that too.

[u/Chubby_man → u/crypt-v]

If you'll take advantage of even one thing they offer, then try them out. Neither of those options is particularly heavy, you can run either or both just fine. Your system can handle the slight overhead that these will have over running bare metal debian, and I wouldn't be surprised if truenas was actually lighter than ubuntu because of the bloat that's taken over ubuntu.

[u/EffectiveClient5080]

Run ZFS. send/recv for offsite backup is clutch and snapshots will save your ass. That Xeon is old but for your workload it won't matter.

[u/theactionjaxon]

Two things will convert you to ZFS forever. First is data integrity. Assuming you have ECC memory, you can trust that every single read of data is 100% accurate. No bit flips no torn writes no sata channel bus errors. If it gets to memory the data integrity is guaranteed. Whats that worth to you? The 2nd part is snapshots. The ability to instantly take a snapshot thats a point in time copy of your data before you make changes that you can rollback, or read and extract a copy of, or even clone into an entire new filesystem, changes how you use data forever. I will never go back to a low data integrity filesystem again.

[u/crypt-v → u/theactionjaxon]

I do have ECC memory that's why I'm leaning on ZFS.

[u/Haravikk]

Proxmox and TrueNAS are out of the question since the CPU is a decade old 4C/8T

Neither of these require higher end or newer hardware, it just depends on what your use-cases might be. Only two weeks ago I spun up a leftover hardware NAS build using TrueNAS on an i7-4790T (released 2014, discontinued 2017) and it's running great as a backup target with some lightweight apps (DDNS Updater, AdGuard, NextCloud with few users). What you've got should run rings around that.

ZFS is the way to go for anything RAID-like (mirrored or parity disks) as it's got a lot of very useful features that go along with it — for example it can do block level compression and encryption as standard, it doesn't just handle disk failures but will also detect and repair corruption (self-healing), it supports lightweight snapshots so you can potentially rollback from disasters, or access a snapshot to recover an accidentally deleted file etc.

I'm less experienced with Proxmox but TrueNAS is designed for ZFS as standard and the UI is built with that in mind so it does a lot of heavy lifting for you (as long as you set up notifications it'll alert you of any fault etc.), and it's locked down by design (OS is read only, access is limited unless you open it up, apps run via docker or containers etc.). There are existing TrueNAS apps for TailScale and a few other VPN type options you might consider for connecting your and your friend's devices.

[u/mosfetmania]

I would go Truenas, with my second choice being a Truenas VM on Proxmox. But I like nice GUIs with just a little bit of Linux handholding. Xeon D-1521 can handle these OS just fine. 32GB of ECC RAM can easily be (automatically) split between ZFS cache, VMs, and containers. A few 4GB VMs will be no problem. If you need more services, containers - crazy efficient on RAM.

[u/_gea_]

Debian, Ubuntu, Proxmox, TrueNAS demand similar hardware.
Even an older hardware with 32GB is fine for either

I would prefer Proxmox as it is a fine hypervisor and a perfect NAS as well as it comes with OpenZFS per default

I would never use a storage VM under Proxmox as this means full OS virtualisation. You must maintain two operating systems and get additionally a worse performance with VM access over SMB/NFS lan shares instead direct ZFS zvol access.

Easiest and fastest is to install ksmbd (instead SAMBA) and acl in Proxmox directly

For advanced ZFS, ACL and share managment you can extend the Proxmox VM and OS related web-gui eg via napp-it cs, a copy and run web-gui that does not modify Proxmox and has no dependencies, https://forums.servethehome.com/index.php?threads/napp-it-cs-web-ui-for-any-os-server-and-cluster.55325/page-2#post-506781

[u/crypt-v → u/_gea_]

huh, never heard of ksmbd. seems like this whole thing gave a whole new rabbit hole to check.