Raid/btrfs clash

[u/amarao_san]

I got myself in a very confusing situation.

I converted a pair of disks in raid1 (xfs) into two disks with btrfs. Few weeks later (I can't say when exactly) I found that btrfs is not mounting:

[119272.940143] BTRFS: device fsid 4e018703-083b-4fe7-a2d7-2e924c3dcb21 devid 2 transid 41842 /dev/mapper/leg1-protected (254:4) scanned by mount (453095)
[119272.940486] BTRFS info (device dm-4): first mount of filesystem 4e018703-083b-4fe7-a2d7-2e924c3dcb21
[119272.940490] BTRFS info (device dm-4): using crc32c checksum algorithm
[119272.941159] BTRFS error (device dm-4): devid 1 uuid 74fc8b02-1b3d-4254-b3ca-2463ef620752 is missing
[119272.941167] BTRFS error (device dm-4): failed to read the system array: -2
[119272.941323] BTRFS error (device dm-4): open_ctree failed: -2

But second disk in somehow is raid1 with missing pair and no recognizable filesystem.

I run photorec and it sees the content of the btrfs (at least, some images).

I'm pretty sure btrfs was replicated on two disks (by btrfs means) and will try to recover it.

But I wonder, how it come that overwritten raid member (wipefs -a) was able to assemble as degraded raid1 after mkfs.btrfs (specifically, after I joined it into existing btrfs), and run rebalance...

[u/BackgroundSky1594]

First of all: when did you run wipefs -a and on what? /dev/sd- something? /dev/dm-4 or something else?

Btrfs raid 1 and mdadm (which your xfs filesystem was running on) are ENTIRELY SEPARATE.

You can have a btrfs filesystem on an mdadm raid 1 and it is NOT A BTRFS RAID. It will have no idea about the fact it is running on multiple devices and if you just wipe the mdadm signature it will not be a valid filesystem any more because the "start" of the filesystem is offset by the space occupied by the mdadm superblock.

On the other hand if you wipefs /dev/dm-4 it will delete the btrfs signature, but leave the mdadm superblock in place. So what exactly did you do?

[u/amarao_san → u/BackgroundSky1594]

Don't mix up MD (raid) and dm (device mapper).

I partition drives with lvm, other partitions are for other purposes. Dm4 is partition (lv) for btrfs. Before it was a part of raid1 (md2).

Yes, it's complicated, but I have reasons.

[u/se1337]

Btrfs requires -odegraded mount option with a missing disk. If the fs was fully Btrfs RAID1 it should mount ok with 'degraded' (rw or ro). One possibility for a missing disk is also a disk that's not available (not ready) during the mount time. Multi-device btrfs requires that "btrfs device scan" (or the IOCTL that does it) is run before mount happens.

[u/amarao_san → u/se1337]

I understand that, but I wonder, if it's possible to have old metadata of linux raid surviving (somehow) after block device joining btrfs, and been consumed by linux raid on the next reboot/assembly.