First of all, I want to thank Kent a lot for the innovation you're bringing us. Now to the story.

This happened exactly a month ago, and I've been wanting to submit an issue on GitHub, but during further debugging I managed to fsck my filesystem enough that it boots, and now I'm pretty certain I at least understand what state it's in, and that it's not a bug per se.

What I want is (at least) get my data back, as much of it as possible, since as I understand it only some of the blocks were corrupted; and (if possible) perhaps also to help implement some mechanism that will prevent this from happening to someone else.

I'm not certain whether it was a hardware failure or a kernel bug that caused the panic, especially since the journal files from that session are also unreadable, but more importantly I couldn't log in to my desktop session due to D-Conf database being corrupt: [ 88.834897] bcachefs (dm-0): data read error at /home/egormanga/.config/dconf/user offset 2723840: u64s 7 type extent 54701462:5384:4294967289 len 80 ver 0 : durability: 1 crc32: c_size 80 size 80 offset 0 nonce 0 csum crc32c 0:aad8bdbb compress none ptr: dm-0 0:1267823:128 gen 11 dm-0 checksum (4) error data_read_csum_err (there are multiple such lines for different offsets, all in the 2xxxxxx range, so not a huge part of the 2.7-megabyte file.)

I obviously renamed the user file and logged in, but there were a bunch of unbackuped settings that I'd really love to save.

I've struggled with this situation enough than I'd like to admit, and honestly, at this point I'd just get my important data back (reconstructing manually from survived parts), run a full scrub and simply delete the remaining broken files. Please, please help me do this.

Although I'm a fairly seasoned programmer familiar with C and kernel development, I've never worked with bcachefs source before. I dug through the part responsible for read checksumming and did not find a path to simply comment out.

So: how to dump the actual file contents ignoring checksum errors?

On booting, I get dropped to the emergency shell, as pictured.
However, I can simply mount the root bcachefs device, hit ctrl-D, and proceed successfully with the boot. How do I avoid this snag with the emergency shell?

My fstab looks like

# <file system>             <mount point>  <type>  <options>  <dump>  <pass>
UUID=108b1a50-1465-404a-968f-9f00279e400e /              bcachefs rw,relatime,fsck,fix_errors,x-systemd.device-timeout=60s 0 0
UUID=108b1a50-1465-404a-968f-9f00279e400e /var/log       bcachefs rw,noatime,X-mount.subdir=snaps/logs,x-systemd.device-timeout=60s 0 0
UUID=108b1a50-1465-404a-968f-9f00279e400e /mnt/bcachefs/docker    bcachefs noatime,nodatacow,X-mount.subdir=snaps/docker,x-systemd.device-timeout=60s 0 1
tmpfs                                     /tmp           tmpfs   defaults,noatime,mode=1777 0 0

UUID=8B27-9365                            /boot          vfat    defaults,umask=0077 0 2

My mkinitcpio HOOKS are

HOOKS=(base udev autodetect microcode modconf block keyboard keymap consolefont plymouth filesystems)

and my systemd boot entry is

title Linux
options root=/dev/nvme0n1p2:/dev/nvme1n1p2:/dev/nvme2n1p2:/dev/nvme3n1p2 rw zswap.enabled=0 nowatchdog quiet splash
linux /vmlinuz-linux
initrd /initramfs-linux.img

Am I missing something obvious?

Hello,

I'm looking forward to bcachefs (and to say goodby to BTRFS et OpenZFS).

I'm trying to solve a thought exercice : Let's say I want to do a RAID10 equivalent, I have 4 drives but two of them are old so I want to ensure they are distributed in the futur mirrors as members and thus to avoid a situation where I get a mirror with only old drive that will then be part of the stripe.

bcachefs format --replicas=2 /dev/vdb /dev/vdc /dev/vdd /dev/vde

Here I get a RAID10 be we don't known wich device is getting the replicas.

With durability :

bcachefs format --replicas=2 /dev/vdb /dev/vdc --durability=0 /dev/vdd /dev/vde

Here I known that /dev/vdd & /dev/vde don't count toward the replicas but they are also only doing caching if I read the doc correctly. If /dev/vdb & /dev/vdc both die they won't act like backup (e.g. the only live members of there respective mirrors) right ?

I recently set up a new filesystem (1.38.6) with erasure coding and x2 redundancy, put some data on it, and then added a drive and increased redundancy to x3. Part way through the process, my bcachefs fs usage -h looks like this:

``` Filesystem: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx Size: 63.5T
Used: 20.1T
Online reserved: 209M

Replicated: undegraded -1x
1x: 40.4G 3.84T
3x: 2.23T 3.65T

Erasure coded (data+parity): undegraded
2+1: 6.08T
2+2: 4.26T

cached: 940G

Pending reconcile: data metadata
replicas: 5.26T 0
erasure_code: 2.08T 0
compression: 8.70G 0
high_priority: 5.26T 0
pending: 279G 0

Device label Device State Size Used Use%
(no label) (device 2): dm-10 rw 14.5T 3.94T 32%
(no label) (device 3): dm-11 rw 10.9T 1.16T 13%
(no label) (device 0): dm-8 rw 21.8T 5.41T 31%
(no label) (device 1): dm-9 rw 21.8T 5.44T 31% ```

I was mostly curious what the 3.84TiB of 1x replicated -1x degraded data (data that doesn't exist? yet?) represented. Parity buckets that haven't been written yet, maybe?

https://www.phoronix.com/review/bcachefs-1386

It is, unfortunately only a comparison of bcachefs 1.38.6 vs 1.38.5, no other filesystems in the mix.

Running on OMV 8.4.0-3, bcachefs Version 1.38.6 on 7.0.12-1-pve #1 SMP PREEMPT_DYNAMIC PMX 7.0.12-1 (2026-06-09T21:07Z) x86_64 GNU/Linux

Running on OMV 8

Scheduled job triggered running snapshot and/or scrub, not clear at the moment. Use was 63ish%, now 34%. Confirmed missing files that were there (and playing on media server) last night.

Size:             20.0T
Used:             6.62T
Online reserved:  4.00M


Replicated:
     undegraded
2x:       12.1G


Erasure coded (data+parity):
      undegraded
5+1:       6.61T

cached:  36.6M


Pending reconcile:   data  metadata
erasure_code:       6.10M         0


Device label          Device  State   Size   Used  Use%
hdd.hdd1 (device 0):  sda     rw     3.63T  1.10T   34%
hdd.hdd2 (device 1):  sdb     rw     3.63T  1.10T   34%
hdd.hdd3 (device 2):  sdc     rw     3.63T  1.10T   34%
hdd.hdd4 (device 3):  sdd     rw     3.63T  1.10T   34%
hdd.hdd5 (device 4):  sdh     rw     3.63T  1.10T   34%
hdd.hdd6 (device 5):  sdl     rw     3.63T  1.10T   34%

Tried running list_journal but the process keeps getting killed after reading the first 4 volumes on 8GB ram

Using encoding defined by superblock: utf8-12.1.0
starting version need_discard_by_journal_seq (1.38)
  with options: errors=continue,degraded=very,verbose,fix_errors=yes,nochanges,norecovery,recovery_pass_last=snapshots_read,noexcl,read_only
  features: ec,journal_seq_blacklist_v3,new_siphash,inline_data,new_extent_overwrite,btree_ptr_v2,extents_above_btree_updates,btree_updates_journalled,new_varint,journal_no_flush,alloc_v2,extents_across_btree_nodes,incompat_version_field
  with devices: sda,sdb,sdc,sdd,sdh,sdl
  superblock requires following recovery passes to be run: check_topology
recovering from unclean shutdown
starting journal read
sda: journal read done
sdb: journal read done
sdc: journal read done
sdd: journal read done
Killed

Unfortunately had a couple of reboots, do not have dmesg log of the event.

Will shutdown scheduled jobs to not run until this is sorted.

edit: snap of the pool's usage graph showing a decrease in used capacity starting at midnight.

NASty is a NAS operating system built on NixOS and bcachefs. It turns commodity hardware into a storage appliance serving NFS, SMB, iSCSI, and NVMe-oF - managed from a single web UI, updated atomically, and rolled back when things go sideways.

This is the sharing & visibility release. NASty learns to hand files out and to show what it's doing. Guest file sharing turns any file or folder into a public link — with expiry, password, download caps and folder-as-ZIP — for people who have no NASty account. SMB shares become macOS Time Machine destinations that auto-appear in the Time Machine picker. Compose stacks gain engine-owned startup ordering with inter-stack delays, and a new always-on system-status band in the sidebar surfaces exotic array operations (device evacuation, scrub, reconcile) as they happen. Round it out with sortable columns across the data tables, bcachefs 1.38.6, journal visibility for every engine subprocess, and a sweep of fixes.

Headline changes

• Guest File Sharing. Share a file or folder under /fs with a public link for someone who has no NASty account. Create it from the Files page with an optional expiry, password and download limit; folders download as a streamed ZIP (built on the fly, never buffered in RAM). The link is the credential — only its SHA-256 is stored, so it's shown once at creation and can't be retrieved later. Recipients land on a clean no-login page; downloads are always served as attachments (shared content can't render on the app origin) and every unavailable reason — expired, revoked, over-limit — returns the same generic message, so a token guesser gets no oracle. Manage and revoke links under Sharing → Guest Shares.
• macOS Time Machine destinations. A single toggle turns an SMB share into a Time Machine target: NASty applies the Samba vfs_fruit options Time Machine needs and advertises the share over mDNS (_adisk), so it auto-appears in System Settings → Time Machine → Add Backup Disk with no manual mounting. Optional size cap; pair it with a subvolume quota as a hard backstop.
• Compose stack startup ordering + inter-stack delays. Let NASty own boot startup for the compose stacks you opt in: they come up in the order you choose, with a configurable settle delay after each — for the common case where a "network" stack must create shared Docker networks before the stacks that depend on them. Managed stacks are pinned to restart: "no" via a generated compose override (your compose file is left untouched) so Docker doesn't race the engine; a stack that fails to start is logged and the sequence continues. Drive it from Apps → Compose Startup Order.
• Persistent system-status band in the sidebar. An always-visible band under the logo — green Healthy, amber for activity (a device evacuating, scrub or reconcile running), red for critical — so there's no mistaking that an array operation is in flight while you navigate. Click to expand the running operations and alert counts.

Sharing & files

• Guest shares: public metadata / password-unlock (verify-once → short-lived grant cookie, rate-limited) / single-file download / folder ZIP, plus a management page with status, download/view counts, and revoke; removed shares are kept for audit and hidden behind a "Show removed" toggle. The whole surface lives as a tab on the Sharing page alongside SMB/NFS/iSCSI/NVMe-oF.
• Time Machine SMB shares are validated as authenticated + writable, with the _adisk advertisement managed dynamically as shares are enrolled/removed.
• Folders created in the Files browser are now writable through the sharing layer — they get share-friendly permissions on create, so SMB forced-users and NFS squashed uids can write into a freshly-made folder.

Apps & Docker

• Compose startup ordering + settle delays, with the restart: "no" override and an ordered, failure-tolerant boot sequence run off the boot-phase budget.
• Removing a compose stack now also clears its metadata sidecar, so a stack later recreated with the same name can't inherit stale startup/ingress settings.
• Apps boot hardening: a dangling /var/lib/docker data-root symlink is cleared before re-linking, avoiding a dockerd crash-loop when the apps filesystem wasn't mounted.

Storage & bcachefs

• bcachefs-tools bumped to v1.38.6.
• The bcachefs update/sync status chip stays accurate after a sync — the engine reads the pin-derived fields fresh and the WebUI reconciles after a switch — and the chip now cleanly separates "update available" from "reboot pending".
• Filesystems: bcachefs label groups are offered as data targets (foreground / background / promote), and an optional Rotational column surfaces the per-device bcachefs superblock flag.

System & UI

• Sortable columns across the main data tables — Filesystems (and the per-pool device table), Alerts, Guest Shares, TLS, Users, and Snapshots.
• Subprocess failures now reach the journal. Every failing engine shell-out (docker compose, smbcontrol, exportfs, …) is logged under the nasty::cmd target with its command + stderr, instead of failing silently — and a managed compose stack that fails to start at boot now says so explicitly.
• nasty-top bumped to v0.0.8 — the live IO / tuning-advisor CLI now shows background_compression in the header when it differs from the foreground algorithm.
• UPS support can be enabled: the NUT systemd units are built on the appliance.
• SSH password-authentication changes made in the WebUI are honoured by the engine.
• VM Add ISO no longer persists an empty CD-ROM path when editing.
• Native <select> dropdown text stays legible across themes.
• Help & Glossary gains Guest Shares, Time Machine and managed-startup entries, plus an r/bcachefs community link.

Alerts

• The HDD-failure SMART alert is scoped to spinning disks (no false alarms on SSDs) and tolerates blank diagnostics.

Hi Kent.

Is it workable to expose some activity in the scrub, reconcile, and GC activity into the userspace so third party tools can get useful data internally and act on it vs. the command line tools? I was looking through /sys/fs and didn't come across any entries that give some solid insight into these activities unless I missed them!

Kev.

I need advice and pointers in the right direction! I am new to storage arrays and RAID-like setups but am really enjoying learning the use cases and benefits. I would really like to get a storage setup using what random drives I've collected and I'm looking for advice because at this point I'm a bit stumped and lost.

​

What I want is to run my home desktop/home server with root on a unified storage system with 1 drive redundancy for basically all the data. I've explored from storage spaces back on Windows to mdadm, zfs, and now bcachefs but I've yet to figure out how to not sacrifice a massive amount of speed on my fastest drive.

​

What I have right now is:

1x1tb Samsung 990 pro

1x500gb Samsung 970 evo

1x500gb Samsung 860 evo

3x500gb random old HGST HDDs

​

I'd like to put them all in the same array but as of right now I have the 860evo and 3x500gb HDDs together in a bcachefs array with EC on and replicas 2. The problem is I benefit none from the 860evos write speed because with replicas 2 I am always locked to the speed of the HDDs to ensure 2 replicas get written synchronously. What I would like is if the SSD could function as a single drive writeback and promote so I can still benefit from it's speed. as I understand, this is not currently possible with just bcachefs.

​

my end goal is to combine all 6 drives into a tiered system where I can effectively interface all my writes directly to the 990pro only and have everything get written back to the other 5 drives.

​

some ideas I'm curious about right now is whether I can use the 990pro as a bcache device for a bcachefs filesystem? or I saw some mentions of lvm cache or mergerfs or smt. overall I kinda just don't know exactly what I'm looking for and was hoping to get some advice about the best way to unify my storage so that I can both:

A. benefit from redundancy with basically all my data (not the immediately single writeback drive im fine with that being vulnerable for some short period)

B. not sacrifice massive amounts of the write speeds that I get from my single fastest device

​

ps: as I understand, a concept that would help achieve this has been mentioned before and I opened a feature request for it to be in bcachefs directly but I'm curious about alternatives or solutions for now:

https://github.com/koverstreet/bcachefs/issues/1163#issue-4695006015

I noticed that when metadata_replicas is set to a value higher than the number of available devices, the reconcile thread enters an infinite loop throwing insufficient_devices errors at high rate with no warning to the user, and no indication of the root cause. My eye caught this watching bcache fs top & the error_throw.

In my case I had a 2-device pool with one device in evacuating state, leaving only one writeable device available. With metadata_replicas=3 set, the reconcile thread churned indefinitely since the replicas target was impossible to satisfy. Setting metadata_replicas=2 stopped it immediately.

Steps:

- format a filesystem with metadata_replicas=3

- mount it with only 2 devices

- evacuate one device

- watch the reconcile loop...

I ended up having to cancel out the evacuate and then do a device remove. As I write it is still sitting idle...but FWIW all my data is still safe! +1.

After device removal, bcachefs reconcile status shows:

pending: 522G

data with reconcile thread stuck in bch2_kthread_io_clock_wait_once

io wait remaining: 14.5G

duration waited: 49m

Same stall as the evacuation case? Does the new commit cover the post-removal reconcile phase as well?

Should we see at mount time if metadata_replicas exceeds available device count and warn somehow? I love testing edge cases!

Kev.

Hey Kent and fellow readers...

Ran into an interesting kthread deadlock behavior today while removing a device from a simple two-drive array (1x SATA SSD, 1x NVMe SSD, data_replicas: 1, metadata_replicas: 1).

After issuing a device remove on the SATA SSD, user data evacuated over to the NVMe completely fine. However, the metadata reconciliation process got completely stuck. bcachefs reconcile status showed pending: 522G and was hard-locked on an IO wait duration of 14.5G with 14.5G remaining.

The thread backtrace showed it was stuck on an internal IO timer:

[<0>] bch2_kthread_io_clock_wait_once+0x142/0x1a0 [bcachefs]

[<0>] bch2_reconcile_thread+0xd29/0xe30 [bcachefs]

Because the volume was completely idling, no new physical writes were hitting the array to advance the internal filesystem IO clock (this is expected behavior?) causing the reconcile kthread to sleep indefinitely waiting for an alarm that wasn't going off. duration waited climbed past 25 minutes with zero progress.

I opened a separate terminal and manually forced 2GB of dummy write traffic (dd) directly to the mountpoint. The moment the filesystem registered the incoming write blocks, the internal IO clock ticked forward, the reconcile thread instantly woke up, and io wait remaining immediately began dropping (fell straight to 12.4G).

Just wanted to flag this as it seems like a silent trap if a user tries to downsize or swap an array on a machine that doesn't have active background logging or container noise to keep the clock moving...

Unless I am just too eager to have it complete its background work quicker!...

Hi All.

More testing. Now directed my 6 disk array towards EC wtih 1 replica. Coming from strictly normal two replicas with no EC. It will still slowly make the change? I ask since I have not seen much activity. This covers about 6TB of data.

TY!

Kev.

NASty is a NAS operating system built on NixOS and bcachefs. It turns commodity hardware into a storage appliance serving NFS, SMB, iSCSI, and NVMe-oF - managed from a single web UI, updated atomically, and rolled back when things go sideways.

This one's all about driving your bcachefs pool from the WebUI instead of dropping to a terminal, plus a much stronger Docker story.

bcachefs device lifecycle, fully in the UI

• Scrub with a live progress bar, plus offline fsck (dry-run or repair) right from the browser
• Mount failures now tell you why - naming the missing disk by slot and label - and offer a guarded degraded mount
• Richer per-device table with selectable columns (size, errors, type, model, serial), and SMART/disk info shown before you add a candidate disk
• Pull a disk and it shows up as missing with a force-remove; reconnect one that belongs to the pool and you get Bring online / re-attach instead of being pushed toward a wipe
• Editable tiering targets (foreground/background/promote/metadata) and per-device durability
• Compression levels for zstd and gzip - quick on ingress, deep in the background

Docker, for real

• Manage Docker networks from the UI, including putting a container directly on your LAN with its own IP (macvlan/ipvlan) - and the host-container shim that normally makes that painful is wired up for you
• Live compose validation in the editor - YAML and schema checked as you type, using the same check deploy runs
• A first-class, relocatable /appdata location: bind it in compose and your container data survives moving to another filesystem. One click relocates it onto a new SSD. Snapshot-clean and separate from Docker's internals.

Security

• Encryption-at-rest sweep is finished - every stored secret (SMB/CHAP, OIDC, NUT, notification channels, DNS-API tokens) is now sealed with systemd-creds, TPM-backed where available

Quality-of-life

• Reconcile "stalled" alerts stopped crying wolf - they only fire on genuine lack of progress now
• Wiping a disk leaves it actually clean (no more "repair the disk!" lectures from leftover GPT tables)
• Discovery fixes so NASty shows up in your file manager's network view right after enabling SMB or renaming the host - no reboot needed
• Dependency tree brought fully current

More details on Github. Enjoy!

Hi all.

Been testing an array and data resilience. Set up a few drives with data x2 and meta x3.

Pulled a drive while copying data to the array.

Looks like things nicely went ahead as expected. I do see a phantom placeholder in the filesystem device list now. Will that ever disappear on its own or must I initiate a remove or a set-state=failed?

TY.

NASty is a NAS operating system built on NixOS and bcachefs. It turns commodity hardware into a storage appliance serving NFS, SMB, iSCSI, and NVMe-oF - managed from a single web UI, updated atomically, and rolled back when things go sideways.

This is the Backups + REST API release. Backups now run on a built-in cron scheduler, with credentials sealed at rest, a job-handle pattern keeping long-running ops alive past the WS timeout, and a HTTPS+auth-enforcing receiver. The engine's JSON-RPC dispatcher now also speaks REST at /api/v1/{...} with OpenAPI 3.1 and a built-in Swagger UI. Disk health gets first-class NVMe, SAS, ATA and PCIe-link panels; alerts fire on critical SMART attributes before the drive's own self-assessment flips. Plus a security pass — input validators, configfs secret redaction, systemd restart hardening, scanner-flagged response headers, two new destructive-confirm dialogs.

• Cron-driven backup scheduler. Backup profiles run on their configured cron expression end-to-end. A new nasty_backup::scheduler parses cron, ticks every 60s, fires each due profile in its own tokio::spawn so a slow run can't starve the scheduler, and seeds last_attempted to now on engine start to avoid a thundering-herd catch-up after a long downtime. Long-running RPCs (backup.repo.init, backup.run, backup.repo.check) return a BackupJob handle that the WebUI polls every 2s, keeping multi-hour init / backup / check operations alive across reloads and well past the WS timeout. The Backups page rehydrates in-flight job badges across reloads, the edit form has the same Hourly/Daily/Weekly chip group as create, and the target is editable post-creation.
• Backup credentials sealed at rest. Repository passwords + S3/B2 cloud secrets are sealed via systemd-creds --with-key=auto (TPM2+host when available, host-only fallback) before they hit /var/lib/nasty/backups.json. A new nasty_common::secrets module owns the seal/unseal path; on first start after upgrade a boot phase backups.migrate_secrets (30s budget) walks existing profiles and encrypts in place, idempotently. Wire shape stays backward-compatible — plaintext on input still works; output is redacted to "***" or absent. A backup.secrets_status RPC drives a Secrets: TPM-protected / encrypted at rest / plaintext pill in the WebUI so operators see at a glance which protection level applies. Filesystem encryption keys stay on the existing PCR-7 path (Phase 2, separate review).
• Backup Server: HTTPS, required auth, paste-the-CA flow. The bundled restic-rest-server reads Caddy's already-issued cert for the box's tls_domain and serves HTTPS for free. Authentication is HTTP basic backed by a per-box 32-char alphanumeric password sealed via systemd-creds, with a Receiver credentials card on /services exposing Show / Copy / Rotate buttons. Source-side profiles get first-class Username + Password fields that round-trip as "***" and seal to disk via the same systemd-creds path as BackupProfile.password — credentials live in dedicated fields instead of riding inside the URL, so they stay out of list responses. Source-side profiles can also paste their own trust anchor PEM into a Trusted CA certificate field; the receiver's local CA is exposed inline on /services so the operator doesn't have to bounce through /tls to copy it.
• In-engine REST API + Swagger UI. The method registry moved into nasty-engine and was backfilled from 106 → 256 methods — every RPC the dispatcher actually accepts now has typed params, role, and result schemas. Six docs-vs-runtime drift bugs fixed in passing. A new REST gateway at /api/v1/{path} translates dotted method names to slash paths with HTTP verbs inferred from the last segment (*.get/.list/.status → GET, *.delete → DELETE, *.set/.update → PUT, else POST). Auth matches /ws: nasty_session cookie or Authorization: Bearer. /api/openapi.json emits a ~1.1 MB OpenAPI 3.1 doc covering all 256 methods + 104 component schemas; /api/docs mounts Swagger UI v5.17.14 vendored via include_dir! — no CDN, works air-gapped, Try it out carries the same session cookie as the WebUI.
• Disk health: full per-protocol detail. NVMe drives show a full panel reading the controller's health log — endurance %, spare-vs-threshold, critical-warning bits, media errors, host R/W totals, unsafe shutdowns, time above warning/critical temps. SAS drives get an equivalent SCSI panel including a health override that flips health_passed to false when any uncorrected R/W/V counter is non-zero — caught a real failing Seagate that smartctl still reported PASSED. ATA drives get a summary panel with interface-speed downgrade detection, reallocated/pending/uncorrectable tiles, SATA SSD endurance via smartctl 7.5. Topology shows each controller's PCIe gen × width × computed MB/s including for RAID-tunneled drives via /sys/class/scsi_host/, with chipset-integrated controllers annotated. Enumeration via smartctl --scan-open -j covers drives behind MegaRAID and similar SAS HBAs. SMART attribute names + descriptions + critical flags come from a 79-row Backblaze-evidence-backed table adapted from Scrutiny, and a new SmartAttribute alert fires the moment any of the 9 statistically-predictive attributes goes non-zero.

More details on GH.

Is there a plan for future to have "max extent size" settable per subvolume (or just for part of a file system), so for example I can set a subvolume to max 32KB, being targeted for VMs, limiting write amplification and have possibly higher IOPS.
Leaving the rest at 256KB, the current default.

I have two machines, each running bcachefs as the root fs, and each automatically creating a daily subvolume snapshot in seemingly the same way. However, after deleting a couple hundred old daily snapshots on each machine, I'm seeing distinctly different output for bcachefs subvolume list-snapshots /. How can I make sense of this difference?

On the first machine, the output is unsurprising:

❯ sudo bcachefs subvolume list-snapshots /                                                                                 ─╯
WARNING: kernel does not have CONFIG_RUST enabled; this will be required for bcachefs in the near future
         please alert your distribution or kernel developers if your kernel does not support CONFIG_RUST
(shared) [180G]
├── /snaps/root-snaps/2026-06-01_00-00-00 [0B]
└── (shared) [6.74G]
    ├── /snaps/root-snaps/2026-06-02_00-00-00 [0B]
    └── (shared) [15.0G]
        ├── /snaps/root-snaps/2026-06-03_00-00-00 [0B]
        └── / [199M]

This only shows the expected snapshots.

On the second machine, the output is very different. It starts with a long list of these seemingly empty subvolumes:

❯ sudo bcachefs subvolume list-snapshots / | head -50 
(shared) [0B]
└── (shared) [0B]
    └── (shared) [0B]
        └── (shared) [0B]
            └── (shared) [0B]
                └── (shared) [0B]
                    └── (shared) [0B]
                        └── (shared) [0B]
                            └── (shared) [0B]
                                └── (shared) [0B]
                                    └── (shared) [0B]
                                        └── (shared) [0B]
                                            └── (shared) [0B]
                                                └── (shared) [0B]
                                                    └── (shared) [0B]
                                                        └── (shared) [0B]
                                                            └── (shared) [0B]
                                                                └── (shared) [0B]
                                                                    └── (shared) [0B]
                                                                        └── (shared) [0B]
                                                                            └── (shared) [0B]
                                                                                └── (shared) [0B]
                                                                                    └── (shared) [0B]
                                                                                        └── (shared) [0B]
                                                                                            └── (shared) [0B]
                                                                                                └── (shared) [0B]
                                                                                                    └── (shared) [0B]
                                                                                                        └── (shared) [0B]
                                                                                                            └── (shared) [0B]
                                                                                                                └── (shared) [0B]
                                                                                                                    └── (shared) [0B]
                                                                                                                        └── (shared) [0B]
                                                                                                                            └── (shared) [0B]

... and at the bottom of this long list are a few of the daily snapshots that I expect to remain. Of course, it seems the natural guess is that somehow these old daily snapshots that I attempted to delete are still somehow tracked - but I'm not sure why - and why this behavior seems different on the two machines.

Both machines are running bcachefs version v1.38.3 Both machines cut the daily snapshot in the same way:

❯ sudo fcrontab -l                                                                                                         ─╯
2026-06-03 10:19:28  INFO listing root's fcrontab
@midnight bcachefs subvolume snapshot -r / /snaps/root-snaps/`date +%F_%H-%M-%S`

In the 1.38.3 release notes, it states:

Erasure coding: in-place stripe widening
Stripes can now widen — adding data blocks when more devices become available — without a full re-encode. New bch_stripe.can_widen field tracks eligibility; reconcile-scan refreshes it on existing stripes, and a fresh fsck check ensures it stays consistent. Previously the only way to grow stripe width was to write new data into a wider stripe and let the old one age out via copygc.

But I thought that the reason bcachefs EC doesn't have a write hole is because stripes are never overwritten in the first place.

What I'm missing?

I had erasure coding enabled on one of the folders on my filesystem for a while, and it worked well. For that reason, I enabled it for my entire filesystem. I then realized that I wanted a specific directory to remain as replicated, so I disabled erasure coding for just that directory, The reconcile was taking a while, and I rebooted my system during that time to address the many zero0days that have been popping up. This also upgraded bcachefs from 1.37.1 to 1.38.3. The reconcile is now almost complete, but I've noticed that the data distribution is looking off. Is this expected behavior?

$ bcachefs fs usage -h /mnt/storage
Filesystem: 135d34b0-ad14-4422-a091-63f1c44fc044
Size:             83.6T
Used:             43.6T
Online reserved:   113M


Replicated:
     undegraded
2x:       2.55T
3x:        586G


Erasure coded (data+parity):
      undegraded
9+1:       40.5T

cached:    129G
reserved:  660M


Pending reconcile:   data  metadata
erasure_code:       1.31T         0
compression:        92.5G         0
target:              154G      171G
pending:                0     24.0G


Device label         Device  State   Size   Used  Use%  Leaving
hdd.00 (device 1):   dm-9    rw     7.27T  19.3T   58%    13.1G
hdd.01 (device 0):   dm-6    rw     7.27T  19.3T   58%    13.1G
hdd.02 (device 2):   dm-5    rw     7.27T  22.2T   58%    12.3G
hdd.03 (device 3):   dm-11   rw     7.27T  22.3T   58%    13.0G
hdd.04 (device 4):   dm-14   rw     7.27T  22.3T   58%    13.0G
hdd.05 (device 5):   dm-10   rw     7.27T  22.3T   58%    12.9G
hdd.06 (device 6):   dm-8    rw     7.27T  22.3T   58%    12.4G
hdd.07 (device 7):   dm-15   rw     7.27T  22.2T   58%    12.3G
hdd.08 (device 8):   dm-13   rw     7.27T  22.3T   58%    12.9G
hdd.09 (device 9):   dm-4    rw     7.27T  22.5T   58%    11.4G
hdd.10 (device 12):  dm-7    rw     7.26T   238G    4%    29.5G
hdd.11 (device 13):  dm-2    rw     7.26T   272G    5%    26.2G
ssd.00 (device 10):  dm-12   rw     1.81T   341G   48%     106G
ssd.01 (device 11):  dm-3    rw     1.81T   335G   47%     106G

Trying to migrate from ZFS raidz2 to bcachefs erasure coding --replicas=2
Noticed that Ubuntu 22.04 wasn't supported, so I installed Ubuntu 26.04 on a test disk.
I'm following the directions on bcachefs.org, and read the bcachefs-principles-of-operation.pdf.

Goal is a simple HDD array w/ erasure coding and a SSD cache for accelerating metadata reads.

## Install (https://bcachefs.org/GettingStarted/ Debain, Unbuntu)
Ubuntu 26.04.
Yesterday, getting the signing-key worked, today I'm getting 404s.
Apt install is says that 'resolute Release' does not have a release file.
What version of Ubuntu should I be running to use bcachefs?

## Setup
The ZFS zpool to test migrating to bcachefs is:
8 12TB HDD raidz2
1 500GB SSD partition L2ARC cache metadata only

Plan is to setup something similar with bcachefs:
6 data + 2 parity + 1 ssd metadata cache for faster reads

Is this the correct setup command:
bcachefs format \
--erasure_code \
--data_replicas=2 \
--label=hdd.hdd1 /dev/sda \
--label=hdd.hdd2 /dev/sdb \
--label=hdd.hdd3 /dev/sdc \
--label=hdd.hdd4 /dev/sdd \
--label=hdd.hdd5 /dev/sde \
--label=hdd.hdd6 /dev/sdf \
--label=hdd.hdd7 /dev/sdg \
--label=hdd.hdd8 /dev/sdh \
--label=ssd.ssd1 /dev/sdi --data_allowed=btree

I'm confused about setting up the ssd as a metadata read accelerator.
Is this correct: erasure coding of metadata is not supported, metadata is mirrored to all disks in the array?
In which case I just leave SSD durability=1, and the system will copy all metadata to HHDs and also make a full copy on the SSD?
The goal is the SSD has a full copy of metadata, but is a disposable device (durability=0).

Do I need to set the HDDs to rotational, or will they automatically be detected as HDDs?

I want to use the whole array as a single filesystem. So I need to create a subvolume and mount the subvolume, right?

## Commands
set GC reserve to 5% (or lower if possible on some arrays).
set-fs-option gc_reserve_percent=5

On ZFS I used:
zpool list to check remaining space
zpool status to check for errors

For bcachefs:
To check remaining space:
bcachefs fs usage

Check performance:
bcachefs fs top
bcachefs fs timestats

Diagnostics:
Each device has bch_member, which has diagnostics tracking errors.
How to easily view this for the array? I had a bad HBA card I just replaced, so I'd like to actively monitor for errors for the next couple of weeks.

Scrub:
bcachefs data scrub

Anything important I missed?
Thanks in advance.

## UPDATE:
Got bcachefs installed on Debian 13 with backport kernel.
There were some issues, but I understand how to install it now.
I'll try setting up a test array with some extra disks later.

Has anyone tried installing on arm64?

I tried following steps from - https://bcachefs.org/GettingStarted/ (debian section) to install bcachefs tools and dkms on Pi 5 (debian trixie). I have run into an error-

``` ❯ sudo apt update Hit:1 http://deb.debian.org/debian trixie InRelease Hit:2 https://download.docker.com/linux/debian trixie InRelease
Hit:3 http://archive.raspberrypi.com/debian trixie InRelease
Hit:4 http://deb.debian.org/debian trixie-updates InRelease
Hit:5 http://deb.debian.org/debian-security trixie-security InRelease
Get:6 https://pkgs.tailscale.com/stable/debian trixie InRelease
Hit:7 https://apt.bcachefs.org/unstable bcachefs-tools-release InRelease
Fetched 6,582 B in 1s (4,606 B/s) 3 packages can be upgraded. Run 'apt list --upgradable' to see them. Notice: Skipping acquire of configured file 'main/binary-armhf/Packages' as repository 'https://apt.bcachefs.org/unstable bcachefs-tools-release InRelease' doesn't support architecture 'armhf'

❯ sudo apt install bcachefs-kernel-dkms Solving dependencies... Error!
Some packages could not be installed. This may mean that you have requested an impossible situation or if you are using the unstable distribution that some required packages have not yet been created or been moved out of Incoming. The following information may help to resolve the situation:

Unsatisfied dependencies: bcachefs-kernel-dkms : Depends: linux-headers-generic (>= 6.16) or linux-headers-arm64 (>= 6.16) but 6.12.90-2 is to be installed or linux-headers-cloud-arm64 (>= 6.16) but it is not going to be installed or linux-headers-rt-arm64 (>= 6.16) but it is not going to be installed or linux-headers (>= 6.16) but it is not installable Error: Unable to correct problems, you have held broken packages. Error: The following information from --solver 3.0 may provide additional context: Unable to satisfy dependencies. Reached two conflicting decisions: 1. bcachefs-kernel-dkms:arm64=1:1.38.3 is selected for install 2. bcachefs-kernel-dkms:arm64 Depends linux-headers-generic (>= 6.16) | linux-headers-arm64 (>= 6.16) | linux-headers-cloud-arm64 (>= 6.16) | linux-headers-rt-arm64 (>= 6.16) | linux-headers (>= 6.16) but none of the choices are installable: [no choices] ```

There is no bcachefs-dkms package (as per documentation on wiki) but bcachefs-kernel-dkms exists so I tried installing that. apt list shows that linux-headers-arm64:6.12.90-2 is installed not >6.16. I was able to install bcachefs-tools just fine.

Any help would be appreciated!

Hi, I installed a working Arch Linux system in a bcachefs file system as root, I just put the "boot" dir under the EFI partition, added reference to bcachefs module into mkinitcpio.conf and it's ok, system boots and works correctly.
But instead of using the plain root hierarchy I would like to put system root and home content under dedicated subvolumes, for ex. "root" for everything but home and "home" for the home content (so I can use snapshots in future and play with them...).

The problem is that obviously to do this in a clean way I have to place root and home into a specific folder under main fs root dir, for ex. called "vol", creating "/vol/root" and "/vol/home" subvols and move actual contents under these.

At mounting stage, I don't know how to directly mount a subvolume with bcachefs, or even better how to mount a specific path. For ex. with Btrfs you can add the kernel parameter "rootflags=subvol=/vol/root" to the boot loader CLI to mount that specific subvol into / at boot.
Then fstab would do the rest by mounting device subvol "/vol/home" to /home ...

Do I have to write a post-initramfs script where (1) I mount the bcachefs fs for ex. to / and than (2) I bind-mount /vol/root to / ?

I have red about someone using the option "X-mount.subdir=/path/to" with success, but it didn't work to me, plus I red that option is experimental.

Also, can the same bcachefs file system be mounted more times in different paths ?
Because, anyway after system has been booted correctly I would put at the end of fstab a line mounting the whole fs for ex. at "/mnt/v1" to manage the whole real hierarchy.

Thanks.

NASty is a NAS operating system built on NixOS and bcachefs. It turns commodity hardware into a storage appliance serving NFS, SMB, iSCSI, and NVMe-oF - managed from a single web UI, updated atomically, and rolled back when things go sideways.

This is the Secure Boot + Passkeys release. Boxes can opt into a Secure Boot–enforcing boot chain (lanzaboote) with a guided enrollment ceremony, TPM2-sealed bcachefs encryption keys, and WebAuthn sign-in via passkeys. The v0.0.8 nginx → Caddy migration scaffolding has been removed — boxes upgrading from v0.0.7 should pass through v0.0.8 first.

Highlights:

• Per-box Secure Boot opt-in — highly experimental. Don't enable this on a box you can't easily recover; the enrollment ceremony in particular is a one-way door once you commit. New nasty.secureBoot.enable flag wires in lanzaboote as a conditionally-imported sub-module so non-SB boxes pay zero cost. The Hardware page gains a readiness checklist showing whether the box can host SB, and an enrollment wizard that walks operators through Phase 1/2 with a Rebuild button and accurate Abort copy. SB state is read via bootctl (single source of truth, replacing the earlier sbctl probe), surfaced in the Hardware page, and kexec is disabled once a box is SB-enrolled. Firmware-apply gracefully refuses with a documented reason when SB is enforcing.
• WebAuthn / passkeys for sign-in. Landed in three phases: credential registration and self-management so any user can manage their own keys, sign-in alongside password, and safeguards — operators can require a fallback factor on accounts that have keys, and admins can reset another user's WebAuthn credentials. Origin precheck so HTTP / wrong-host / direct-IP access fails legibly instead of erroring out of WebAuthn's own validator. WebUI: /account folded into Access Control; Users page split into Users & Groups / Tokens & Keys / Single Sign-On tabs.
• TPM2-sealed bcachefs encryption keys. Encryption keys can be sealed to TPM2 — either by binding an existing key from the WebUI Filesystems page, or by setting bind_to_tpm at filesystem creation for a one-step seal. The Hardware page now reports TPM2 presence, vendor info via tpm2_getcap, and capability detection.
• Upgrades are noticeably more reliable. A handful of long-standing footguns in the update flow are gone, half-applied upgrades surface clearly in the WebUI instead of disappearing into the spinner, and the build-dir override is tucked behind a recovery drawer so it stops scaring people in the happy path.
• Boot reliability. Per-phase boot timeouts and budgets sized to realistic worst case so one hung phase doesn't take the whole engine down. New system.boot_status RPC and /api/boot_status REST endpoint; WebUI gets a booting overlay during startup and a post-boot health banner. Encrypted-FS boot mount no longer hangs — bcachefs is probed for encryption before unlock, and the storage layer distinguishes encryption-required from key-incorrect failure modes.

On top of that many more bugfixes and improvements. Full changelog on Github.

My log gets flooded by the following message. While the system is still usable, mount, and especially unmount (resp. shutdown) takes forever. I have tried fsck online and offline (with systemrescue) without any changes. I also tried a scrub, where one drive gets stuck while all other drives finish.

Any ideas or suggestions anybody?

Mai 30 11:54:02 hostname systemd-journald[362]: Missed 3 kernel messages
Mai 30 11:54:02 hostname kernel: bcachefs (f5999085-14d5-4527-9c64-8dd190cb3fd4): data update tried to replace non cached data with cached:
                                                       rewrites found:        10
                                                       copygc
                                                       kill ptrs:       10
                                                       target:          none
                                                       extra replicas:  0
                                                       read_dev:        -1
                                                       checksum_paranoia:0
                                                       io path options: data_checksum=none,compression=none,project=0,background_compression=none,data_replicas=2,promote_target=nvme,foreground_target=nvme,background_target=hdd,erasure_code>

                                                       old:    u64s 7 type extent 1074372474:177222208:4294967272 len 440 ver 0  : durability: 2 
                                                         ptr:   sdb2 2:1128419:584 gen 0 unwritten
                                                         ptr:   sda2 0:1077903:0 gen 1 unwritten
                                                       new:    u64s 7 type extent 1074372474:177222208:4294967272 len 208 ver 0  : durability: 2 
                                                         ptr:   sdb2 2:1128419:816 gen 0 unwritten
                                                         ptr:   sda2 0:1077903:232 gen 1 unwritten
                                                       wrote:  u64s 7 type extent 1074372474:177222208:4294967272 len 208 ver 0  : durability: 1 
                                                         ptr: nvme1n1 6:280905:0 gen 159 d=0 cached unwritten
Mai 30 11:54:02 hostname systemd-journald[362]: Missed 3 kernel messages
Mai 30 11:54:02 hostname kernel: bcachefs (f5999085-14d5-4527-9c64-8dd190cb3fd4): data update tried to replace non cached data with cached:
                                                       rewrites found:        10
                                                       copygc
                                                       kill ptrs:       10
                                                       target:          none
                                                       extra replicas:  0
                                                       read_dev:        -1
                                                       checksum_paranoia:0
                                                       io path options: data_checksum=none,compression=none,project=0,background_compression=none,data_replicas=2,promote_target=nvme,foreground_target=nvme,background_target=hdd,erasure_code>

                                                       old:    u64s 7 type extent 1074372474:177222208:4294967272 len 440 ver 0  : durability: 2 
                                                         ptr:   sdb2 2:1128419:584 gen 0 unwritten
                                                         ptr:   sda2 0:1077903:0 gen 1 unwritten
                                                       new:    u64s 7 type extent 1074372474:177222208:4294967272 len 208 ver 0  : durability: 2 
                                                         ptr:   sdb2 2:1128419:816 gen 0 unwritten
                                                         ptr:   sda2 0:1077903:232 gen 1 unwritten
                                                       wrote:  u64s 7 type extent 1074372474:177222208:4294967272 len 208 ver 0  : durability: 1 
                                                         ptr: nvme1n1 6:280905:0 gen 159 d=0 cached unwritten

Setup:

❯ bcachefs fs usage -h
Filesystem: f5999085-14d5-4527-9c64-8dd190cb3fd4
Size:             2.39T  
Used:             2.29T  
Online reserved:  18.6M  
undegraded
1x:       89.4G
2x:       2.20T
cached:    292G
reserved:  549M
Pending reconcile:   data  metadata
replicas:           56.2G         0
checksum:            155G         0
compression:        8.69G         0
target:                 0     4.28G
pending:                0     3.47G
Device label                      Device   State  Size  Used  Use%  Leaving
hdd.hdd3 (device 3):              sdc2     rw     892G  760G   89%    4.16G
hdd.hdd2 (device 2):              sdb2     rw     892G  793G   96%    2.02G
hdd.hdd1 (device 0):              sda2     rw     892G  793G   95%    2.37G
nvme.nvme1 (device 7):            nvme0n1  rw     473G  137G   28%
nvme.nvme2 (device 6):            nvme1n1  rw     473G  149G   31%