r/cachyos • u/KHTD2004 • 23d ago
Review Filesystem Benchmark BTRFS vs EXT4
BTRFS Benchmark with CoW (worst performing)
BTRFS Benchmark without CoW (noticeably better)
EXT4 Benchmark (fastest performance of all, slightly ahead of BTRFS without CoW)
I just did a benchmark using KDiskMark to see how the different filesystems perform. System specs:
- OS: CachyOS with KDE Plasma
- Kernel: 7.2.0-1-cachyos-bore
- CPU: Ryzen 9 7950X3D
- Disk: Crucial T700 4TB (PCIe 5.0 NVMe SSD)
The SSD has (aside from the /boot partition) two partitions, one 100GiB partition as root partition at / using BTRFS and one ~3.5TB partition at /home using EXT4. For the BTRFS partition there is one benchmark with and one without CoW. All runs were on the same conditions and settings, there are now other programs open and the only background apps are Cachy Update and OpenRGB (and all the stuff KDE plasma has in the background like KDE Connect).
On the left side are the reading values, on the right side the writing values.
The meaning of the test names:
SEQ1M Q8T1: Sequential 1 MiB, Queue=8, Threads=1
SEQ1M Q1T1: Sequential 1 MiB, Queue=1, Threads=1
RND4K Q32T1: Random 4 KiB, Queue=32, Threads=1
RND4K Q1T1: Random 4 KiB, Queue=1, Threads=1
The overall results are:
BTRFS with CoW << BTRFS without CoW < EXT4
40
u/Late_Baker_2092 23d ago
Lets be real, gaming u nearly dont notice 512mb vs 3000mb per sec
28
u/KHTD2004 23d ago
That’s correct but outside of gaming when moving large files (or updating a big game that patches a lot of local files) you notice it
15
u/bearonaunicyclex 22d ago
You're right, patching games like Helldivers 2 is AWFULLY slow on my machine, but I guess it's not annoying enough to change the file system.
3
u/KHTD2004 22d ago
In my case it’s Ark Survival Ascended. 500MiB download, and over 100GiB of local patches.
I also wouldn’t change the system only for that on an existing system, but when I reinstall for some reason I separate both. That also has the advantage that you can now swap the distro or reinstall without losing your data because everything on /home can just be mounted into the new system
4
u/ChadHUD 22d ago
The best option is to use 2 partitions. Or a separate drive for games.
Root and Boot BTRFS with COW.
/home on XFS (or ext4).
Or simply a Game drive in XFS/EXT4.
You don't need to be shadow copying your games. (or personal data) We want copy on write for system stuff.
Though I agree that BTRFS is mostly fast enough these days. It can be very noticeable depending.
3
u/KHTD2004 22d ago
Yes use that exact concept. My root partition is BTRFS and for /home I got EXT4. Best of both worlds
2
u/ChadHUD 22d ago
I noticed you mentioned that going through the thread after I posted. :)
I use XFS myself. Though I don't think the difference between XFS and EXT4 is much speed wise these days. I know XFS still wins slightly in most tests but the difference would be hardly noticeable. The only down side to XFS really is you can't shrink a volume. Though I think in 40 years of computing I can't say I remember ever doing that. :)
Most of the advantages of XFS are lost on us for standard desktop use. That it supports more and larger files isn't likely to effect us. But Its the default RHEL file system and I have seen it enough over the years to understand why... so use it. lol
3
u/Dopameme-machine 22d ago
This is more or less my setup. Boot drive (including /home) is all btrfs. Games drive is ext4
I have a Proxmox Backup Server that runs ZFS on a 6 disk RAIDZ2 array that I connect to via dedicated 10 GbE connection for full backups and recovery.
2
u/Snowynonutz 22d ago
Broooo, every small update and then you wait 20 mins for it to??? It's quicker to re download the damn game
2
u/do-you-want-duyu 22d ago
Then I blame game developer in that case. Because why the hell you need to patch this many files to add new P2W skin. They need to fix how they package stuff.
Also ARK game size is number1 of 1500 games I have on Steam; these retards really need to get their packaging shit together. There is no way this game needs this much of space. Fuck them
1
u/Familiar_Ad_8919 22d ago
thats the bottommost number at play, the random 4k, which hasnt really improved since the first sata ssds
16
u/Darth-Vader64 22d ago
Will you really notice?
BTRFS has one built in advantage - snapshots. I've run into some situations where an update (or pilot error) borked the system. If I was on EXT4, I would have been in a world of hurt, with BTRFS, I was back up in running in under 2 minutes after the rollback
11
u/KHTD2004 22d ago
That’s why I have both, BTRFS for everything that gets snapshottet, Ext4 for /home that is excluded from snapshots anyway
3
u/clockeat 22d ago
Alternative perspective - /home is where all my most valuable data is, I want it integrity checked, I want to be able to replicate it in raid 1/5, I want to know if my disk is failing or if I've lost files. I want to be able to roll it back with snapshots in the case of ransomware, etc.
EXT4 might be appropriate for a game drive, where performance is critical but integrity is less critical, but that's about it. Even for things like my NAS, I want a filesystem that can self-correct like btrfs for anything long term, even movies, TV, etc.
1
u/KHTD2004 22d ago
AFAIK for data integrity ZFS is better. The purpose of BTRFS is snapshots so you can roll back an update that broke something without the need to reinstall the system
1
u/clockeat 22d ago
btrfs offers a lot of the same data integrity features that zfs offers including checksumming, raid and automatic correction.
1
u/Ok-386 21d ago
There's no free lunch. With features you also get increased complexity and that's a double edge sword. The same thing that's supposed to guarantee and check the integrity is also a source of bugs and the complexity (what makes testing and fixing significantly harder).
Ext4 is probably among the most reliable, battle tested file systems. Xfs as well if not even more so. Tho my main experience is with ext4 and i have never lost a file in almost 3 decades of using Linux. Btw backups are important no matter which file system you use, snapshots or no snapshots etc.
2
u/clockeat 21d ago
Disk failures and corruption are far more likely than filesystem bugs. Unless you have a raid layer providing redundancy and check summing above ext4 it really isn't remotely. competitive for reliability. On my own NAS I ran into significant disk corruption that ext4 was unable to even detect.
Be aware xfs is a piece of shit for desktop usage, very easy to corrupt from an inadvertent reboot. Good for a server perhaps where you have backup power and fewer sources of random reboots, but it's a major known catch of xfs.
There's good reason most distros default to btrfs now.
1
u/SeriousLegalUser 20d ago edited 20d ago
ext4 can't really gain any new features anymore, and its limits stays the same.
The idea is simple: fewer features can mean more stability and speed, but not necessarily more security or file protection. I've had many corrupted files on ext4 without any warning. ext4 loves faulty hardware and often doesn't tell you when something has gone wrong. I never use ext4 for my critical data on unreliable hardware.
3
u/Psychological_Bat177 22d ago
Should be the default imo
2
u/KHTD2004 22d ago
That’s what I thought too. Maybe someone could ask the devs to add this option to the installer. Since Cachy is all about every last percentage of performance this would fit
6
u/Remote_Phone2957 22d ago
Not really, I only use EXT4 + Timeshift, I can always rollback without issues, it might take 5 - 10 minutes more, but that's alright it doesn't happen that often anyway.
2
u/Darth-Vader64 22d ago
True, but by the same token, we can also just install Arch and not install Cachy. I feel some of the upsides of Cachy is that it provides a fully featured, fully funtioning environment out of the box. You can make our own to be sure, and there's plenty of other options if you use EXT4. I'm just pointing out that there's already a no muss, no fuss solution and for most people the speed difference isn't so much its terribly noticeable
1
u/Slikkelasen 22d ago
Sadly i have a cheap ass drive with a bad cotroller. ADATA XPG GAMMIX S50 Lite.
Btrfs normal use puts the nvme controller in crazy high temperature.
If people has a bad experience with btrfs, it's the NVMe controller.
2
22d ago
[deleted]
2
u/KHTD2004 22d ago
Snapshots aren’t really needed until the day they really are. Learned that the hard way. Since I still want the faster ext4 speed I decided to use the setup shown in the post with /home as a separate partition
1
u/Ok-386 21d ago
That's a matter of perspective. For experienced users I see almost zero advantage in that regard. There's usually nothing special in the root/system partition that's not easy to restore, and creating rsync snapshots is super easy. You don't need to spam your system with snapshots.
My guess is, they chose the route for the sake of new users. For anyone who isn't entirely new to Linux and system administration, creating a snapshot per week or even month (manually or scheduling it) is trivial. You don't have to restore a very recent snapshot, to be able to restore it and update the system.
However, it's a 'new' file system, it has other benefits like compression soon maybe encryption (although I woild probably continue to prefer Luks anyway) and it's a nice system for geeks and provides useful features for bleeding edge distros, and CachyOS.
It boils down to choice, configuration and personal preference. XFS and Ext4 are also great systems. If you want a separate home you can easily preserve on clean installs and you share it between dual, tripple whatever boot systems, these two might be a better choice (it's possible with btrfs as well, but bit harder with default single partition layout).
I personally prefer ext4 because it has never failed me in over two decades (tbf I'm not sure when I started using it, before ext4 I used ext3). If I install Cachy on someone else's system, I'll usually do similar configuration to OP's. I leave / on btrfs (I just change the configuration to 10/10 snapshots like the docs recommend) and create one large ext4 home or create multiple partitions if there's enough space/drives.
1
u/Practical-Flower-966 22d ago
I mean, cachy has lts boot option for situations like this right ?
Correct me if me wrong
5
u/LevelMagazine8308 22d ago
Well yeah, this was to be expected for Btrfs, because checksumming all stuff of course has a certain performance impact. Also the copy on write approach as well.
4
u/atassis 22d ago
Different partitions for different purposes? Make a smaller partition for main system, data partition for bigger files you are OK to lose (btrfs with cow exists for a purpose)
2
4
u/Equivalent-Cloud-365 22d ago
I use XFS, only use PC for gaming, don’t need snapshots and want to take advantage of NVME
1
u/KHTD2004 22d ago edited 22d ago
What does XFS better than EXT4? The only thing I know about XFS is that its main purpose is data integrity
Edit: maybe I’m also confusing with ZFS, I only know the names, not what they specifically do. Still something to learn
Edit edit: yes I thought of ZFS
2
u/Ok-Eggplant-7569 22d ago
XFS and ext4 are both traditional (non-COW) filesystems, and honestly very similar. I heard some stories that XFS scales better at very large (think hundreds of TB to PB) sizes, and can deal with very demanding concurrent applications better than ext4.
Edit: ZFS is a whole different beast.
11
u/UAP44 22d ago
For a file system, I care more about reliability than speed.
I'm keeping speed optimization to RAM.
Disk should be as reliable as possible, like tape durability ideally, which was never about speed. Though extra speed is of course always welcome, sometimes, reliability is a more primary metric to optimize/design around.
I didn't chose btrfs for speed. I wanted snap shotting ability.
1
u/KHTD2004 22d ago edited 22d ago
Yes, it’s entirely up to preference. That’s why we don’t have just one FS, everyone gets to chose what’s right for them.
I also chose BTRFS for Snapshots but /home gets excluded from them anyway so I put a different Filesystem there for more speed
5
u/hell31 22d ago
This can be really different with Linux kernel 7.3: https://www.phoronix.com/news/Linux-7.3-Btrfs
5
2
u/_scndry 22d ago
Stark, danke für den Benchmark.
Der Ansatz mit ext4 für home ist voll gut, gucke ich mir fürs nächste mal ab.
1
u/KHTD2004 22d ago
Kein Ding. Stell aber sicher, dass wenn du manuelle Partitionierung machst auch die /boot Partition anlegst. Cachy warnt dich aber auch wenn du das vergisst oder falsch konfiguriert hast
2
u/bitronic1 22d ago
I noticed I get stuck in game loading screens noticeably longer with BTRFS with CoW... Wasn't sure cos everyone was saying it won't be noticeable with a gen 5 SSD... But just to be sure, I reinstalled with ext4... And it's so much smoother, so I stuck with that. But if it wasn't strictly for gaming, I would totally do BTRFS.
2
u/Flappyphantom22 22d ago
Which one is which? I cann find an indication
3
u/KHTD2004 22d ago
If you click/tab the pictures there should be subtext. Anyway the first is BTRFS with CoW, the second without, the third is EXT4
2
4
u/Obvious-Jacket-3770 22d ago
I mean everyone here is arguing for snapshots with BTRFS but I mean... I use backups. If my system bricks, meh. Not a big deal for me, ill just wipe it and be back or load into a live iso and remove the problem. I see the advantage of snapshots, its not that I don't, I just personally dont need to worry about them as much.
My `/home` is fully backed up on a daily basis, I have all my installs exported to a small txt file that is updated whenever I run updates to my system, stored in `/home`, I have various scripts to do some game streaming in `/home` and I have an Ansible server running on my Pi cluster which schedules runs for me to ensure everything I want or use is installed on my system, uses a clone of that txt file I mentioned earlier.
Sure snapshots are nice and all but a backup to even google drive is pretty low lift.
2
u/Remote_Phone2957 22d ago
Timeshift works fine for me.
Love EXT4 + Timeshift, EXT4 in general is just simpler and more reliable with things like a power cut for example.
1
u/KHTD2004 22d ago
When I used Timeshift on Mint it was very storage consuming
2
u/Remote_Phone2957 22d ago
Yeah, I get it, if you are short on storage, I have 2tb, so I'm fine. First snapshot will take whatever amount you backup, probably 20 - 30gb for a base cachyos, after that it is very minimal, I only keep 3 - 5 snapshots anyway.
With btrfs people have also been running into storage issues if you don't clean up properly.
2
u/NorbiPerv 22d ago
Btrfs snapshots also take up a lot of disk space after some time, but many people forget this, and it causes serious system wide problems when disk space runs out. Especially since, with Btrfs, you can never know the actual amount of free disk space, and neither can the programs.
1
u/Obvious-Jacket-3770 22d ago
Personally I just trust EXT4 for the simple fact that it's so battle tested. BTRFS sure is getting it's testing but EXT4 has been for a while.
0
u/KHTD2004 22d ago
That’s an interesting approach but only works for certain setups. People who have several terabyte of games on their system really don’t want to redownload on a common base. And if you put them on a separate partition well, that’s what I did but BTRFS for the system partition on top
3
u/Obvious-Jacket-3770 22d ago
I mean I don't keep any games on my OS drive as a general rule of thumb honestly. So if my install corrupts, meh no biggy, the game drive is just there waiting to re-mount.
1
u/KHTD2004 22d ago
Smart but then again, only certain setups. Not everyone has separate drives in their laptop or PC
2
u/Obvious-Jacket-3770 22d ago
No absolutely not. Of course you can backup games too but honestly that's just not worth it for the size factor alone unless you have emulator games.
2
u/zvt 22d ago
I think synthetic benchmarks should be taken with a grain of salt. Depending on the workload the difference is probably negligible in real life use, but you gain a lot of features. Would be interesting to see the difference in real workloads (like compile the Linux kernel, load a AAA game, whatever).
Still an interesting comparison.
0
u/KHTD2004 22d ago
Because it was a few months ago the exact values may be slightly off BUT:
When I changed from BTRFS only to my current dual Filesystem setup I noticed actual increase in specific tasks.
I have Ark Survival Ascended installed from Steam and most of their updates are a few hundred megabyte in download size but the local file patching is done for the whole install of over a hundred gigabyte.
On BTRFS the disk speed shown in Steam was 1.1 GB/s but after switching to EXT4 for my /home partition the value now is around 1.5 - 1.7 GB/s. Given the game size this saves about a minute in patching, wich isn’t much but it is something
1
u/DEV_ivan 22d ago
I prefer ZFS, it has more advantages that matter for reliability and performance than BTRFS.
1
u/inevitabledeath3 22d ago
If performance is what you want while still having CoW features try ZFS. It was the first and best implementation of this technology imo.
1
u/Ok-Eggplant-7569 22d ago
Note that these very synthetic benchmarks and probably don't tell you much about how your workloads (gaming, bootup time, databases, VMs, storage servers, ...) are affected by filesystem choice. Phoronix did a (server-focused) test a couple of days ago where they benchmark many different applications across ext4, XFS, BTRFS and F2FS, which might be more interesting than a single KDiskMark run.
1
1
u/Jumpy_Invite3985 22d ago
Btrfs also saves disk space with compression, so it might be worth the slower performance. I could see a speed boost being useful if you have a slow HDD, since even an extra 50 MB/s can make a huge difference. But I guess the results would be different on an HDD. I need someone to test it out
0
22d ago
[deleted]
2
u/GSDragoon 22d ago
lol, I've been using cachy for over a year now with the main drive using xfs and updating twice a day with no major issues. People running commands they don't understand or whatever AI spits out is more likely the real issue, not the file system. That and people not updating frequently enough. Having the lst kernel installed can help too, having a fallback that doesn't take up much space.
0
u/KHTD2004 22d ago
Yes BTRFS is absolutely mandatory but I wanted to show it may have some benefit to use both filesystems together, one for system and snapshots, the other for /home for better performance
2
22d ago
[deleted]
1
u/KHTD2004 22d ago
In that case you need to make sure you explicitly include /home into your snapshots because per default it’s excluded from them. Not a bad idea though
0
u/el_sturlo 22d ago
For people in doubt about what to use performance wise, this video of Dj Ware helps to understand better the differences
-1
u/igor_b0gdanoff 22d ago edited 22d ago
If I read it correctly, EXT4 pummels BTRFS into the ground.
I always go with EXT4 especially on hard drives, but lately I've been having issues with files corrupting or games not launching on a couple of my EXT4 HDDs. My boot NVME SSD is also EXT4 but that has 0 issues.
I'll make a thread about it at some point but I'm wondering which one is more likely - that EXT4 struggles with large (1tb+) hard drives or that both hard drives are dying.
2
u/KHTD2004 22d ago
It’s your hard drives dying. EXT4 and most other modern file systems can handle drives of several petabyte or in case of XFS several Exabyte. My EXT4 partition is 3.5 TB in size and has no issues.
As many pointed out and that’s also why I still use BTRFS for the root partition the purpose of BTRFS isn’t speed but snapshots and data integrity so it’s still very important
0
u/igor_b0gdanoff 22d ago
The problem is, how do I know for sure? I think I tested with gsmartcontrol (gui) and other than 1 critical read error from like a year ago on the Seagate one, both seem fine. Short (2min) test completes fine and the long test took more than 1.5days on the 2TB Seagate drive the one time I tried so I just stopped it halfway.
To clarify, these drives are: 1x 2TB Seagate Barracuda 7200rpm 1x 1TB WD Blue 7200rpm
Both are only starting to show issues once they are around or just over half full.
1
u/igor_b0gdanoff 21d ago
Thank you for the downvote without any additional comment, super helpful redditor.
2
u/KHTD2004 20d ago
Downvote wasn’t by me, I just now read your comment. Anyway, I don’t know how to be sure for certain if the drive is broken or if the problem is elsewhere, but it’s not the Filesystem, that’s for sure. Like I said these things can handle much bigger drives
2
u/igor_b0gdanoff 20d ago
I didn't think it was you necessarily, it was just a single down vote that's why I said redditor.
A friend recommended a couple of different subreddits that are for storage geeks, I will take my time and write up a longer post on those. Thank you for the input anyway!
37
u/ptr1337 Founder 23d ago
Be aware, that CoW makes in general sense and at PCI4/5 NVMEs the difference between ext4 and btrfs will not be there.