Lost promote

[wisniew]

I have two disks in the background, two in the foreground, and two in promote with metadata replication set to 1.

A few days ago, the controllers in the promote disks failed simultaneously, and I am unable to read or write anything to these disks.
Is there any way to recover data purely from the background and foreground? Or does this mean that all data is lost?

[munkien]

Just trying to chip in

What did you try in the last few weeks in an attempt to fix the problem?

What was the reason for choosing Metadata replication 1?

Did you try to unplug the drives and use just one of them at a time?

🤞 That you found a way

[himikof]

Generally, bcachefs can try to recover from many things as best as it can, although in some cases it means accepting partial data loss. Having --metadata_replication=1 certainly does not help, unfortunately.

The standard way to try it would be:

• First try to mount the FS with -o degraded,nochanges
  It will very likely fail and tell you in dmesg that it cannot mount without some devices, but if it succeeds then you are very lucky and have no data loss at all.
• Then try to mount with -o degraded=very,nochanges (after verifying that all member devices that can be online are)
  There will be a repair process, possibly quite long. Most likely it will succeed, and whatever could be recovered will be available in the FS (or maybe inside lost+found directory there). In case the repair stops with repair_not_implemented error, please report the full repair log in this issue (so that we can implement it).

Please disregard the advice about trying to mount one device at a time, it is hard to do (by design), not helpful for recovery and can only do further harm if you are not careful.

If you need additional help, there is an bcache IRC channel on OFTC where people can advice you further.