unbootable system "Impossible to mount filesystem"

[phloggu]

Installing unifi-pfsense leads to an unbootable system if pfSense was installed with MBR (BIOS) boot mode. The error is:
[...]
Filesystems are clean, continuing...
Mounting filesystems...
ERROR: Impossible to mount filesystem, use interactive shell to attempt to recover it

I've tested on the same machine once installing pfSense as MBR (BIOS) and once as GPT (BIOS); it works without that issue in the latter but not in the first case.

pfSense 2.4.4-p3
PC Engines APU2 (APU4d4)
unifi-pfsense 5.12.35 (7024e82 on 23 Nov 2019)

[Itay1787]

Install pfsense on UFS NOT zfs
I personally haven't found a way to make it work with ZFS

[phloggu]

Do you have any clue why this does not work with ZFS? I believe the FS should be unrelated to software running on it, if the software doesn't fuss with the FS.

It seems to run on ZFS too, if its on GPT and running in an UEFI system. Did you try this?

[cypherstream]

I run zfs without issue. Only /var/run is listed as ufs.

[phloggu]

@cypherstream: In GPT/UEFI or MBR/BIOS mode?

[jmbwell]

If we can determine what about this script is affected by the boot partition or filesystem, let's open a new issue with more details.

[phloggu]

There is enough detail:

• it runs on ZFS
• it runs under GPT/UEFI
• it doesn't run under BIOS/MBR

It's a grave functionality issue which destroys running pfSense Systems, I detailed the steps to reproduce. Instead of closing this bug you should reproduce it:

• take a system able to boot in legacy and uefi mode
• install pfsense and the script in either mode and report back

[jmbwell]

I don't discount the gravity of the issue; certainly the destruction of a running pfsense system is grave. But without knowing what I'm looking for, I am not able to make this happen.

All this script does is copy files and install packages. It's unclear how the disk header would even be affected by anything this script is doing. But anything is possible.

You've concluded that this script is responsible for the issue, presumably for a reason. If you know of something this script is doing that is likely to affect the disk header in a way that would cause an MBR disk to be unbootable but not a GPT disk, we should start there.

Otherwise, I think the only place to start is the steps to reproduce. What steps do I take, what settings do I choose on what screens in what order, to make my system not bootable? Explain it like I'm five.

I'll leave the issue open. Help me help you.

[phloggu]

How to reproduce:
(Tested on Zotac ZBOX pro ci329 nano, SSD, 16GB RAM
and on Shuttle DS77U, mSATA SSD, 8GB RAM)

• Set your BIOS to boot in legacy mode
• Install pfsense 2.4.5 on your box:
  • Boot into the installer, use defaults
  • From the Partitioning Menu chose: "Auto (ZFS)"
    Then set:
    • Pool Type: ada0 to stripe
    • Swap Size: 0g
    • Partition Scheme: MBR (BIOS)
  • continue installtion and reboot
• Boot into pfsense for the first time:
  • set up LAN and WAN ifaces
  • finish setup
• Open a shell from the console menu (chose 8)
• Installl unifi-pfsense using:
  fetch -o - https://git.io/j7Jy | sh -s
• reboot

-> you will run into the reported error

Set your BIOS to boot in UEFI mode and repeat the installation of pfsense and uifi-pfsense following the above steps with one exception:
Use the Partition Scheme: GPT (UEFI) instead of MBR (BIOS) as before

-> you will not run into an error.

Please confirm you can reproduce the error with the described procedure.

I have no explanation of why this happens. The scripts alters /etc/fstab with the following code:
67: echo -e "fdesc\t\t\t/dev/fd\t\tfdescfs\trw\t\t0\t0" >> /etc/fstab
This actions seems most relevant to me what concerns failures for booting the system.

[jmbwell]

Thank you. I am able to reproduce this, following your instructions. No insight yet on what is causing this.