Unable to create rollback file after OS reinstall (Regenerate TOTP/HOTP)

[marmarek]

Please identify some basic details to help process the report

A. Provide Hardware Details

1. What board are you using (see list of boards here)?

2. Does your computer have a dGPU or is it iGPU-only?

• dGPU
• iGPU-only

3. Who installed Heads on this computer?

• Insurgo
• Nitrokey
• Purism
• Other provider
• Self-installed

4. What PGP key is being used?

• Librem Key
• Nitrokey Pro 2
• Nitrokey Storage
• Yubikey
• Other

5. Are you using the PGP key to provide HOTP verification?

• Yes
• No
• I don't know

B. Identify how the board was flashed

1. Is this problem related to updating heads or flashing it for the first time?

• First-time flash
• Updating heads

2. If the problem is related to an update, how did you attempt to apply the update?

• Using the Heads GUI
• Flashrom via the Recovery Shell
• External flashing

3. How was Heads initially flashed

• External flashing
• Internal-only / 1vyrain
• Don't know

4. Was the board flashed with a maximized or non-maximized/legacy rom?

• Maximized
• Non-maximized / legacy
• I don't know

5. If Heads was externally flashed, was IFD unlocked?

• Yes
• No
• Don't know

C. Identify the rom related to this bug report

1. Did you download or build the rom at issue in this bug report?

• I downloaded it
• I built it

2. If you downloaded your rom, where did you get it from?

• Heads CircleCi
• Purism
• Nitrokey
• Somewhere else (please identify)

Please provide the release number or otherwise identify the rom downloaded

https://circleci.com/gh/linuxboot/heads/14178 ( x230-hotp-maximized_usb-kb of 4a57c61)

Please describe the problem

Describe the bug

Creating rollback file fails after OS reinstall (including wiping /boot).

To Reproduce
Steps to reproduce the behavior:

1. Install Qubes OS 4.2.0
2. On reboot choose to re-generate HOTP secret and then sign boot files
3. When prompted creating TPM counter, provide TPM owner password as prompted
4. See error:

sha256sum: can't open '/tmp/counter-': No such file or directory
sha256sum: can't open '65683996': No such file or directory
!! ERROR: /boot: Unable to create rollback file !!!

Expected behavior

Rollback file successfully created.

Screenshots

https://openqa.qubes-os.org/tests/88760/video?filename=video.ogv&t=92.9

The link above includes full flow leading to the failure, I recommend watching with 25% speed otherwise it's hard to follow.

Additional context

The problem didn't happened when I preserved heads-related files in /boot across reinstall (then it only required re-signing boot configs, which works fine).

[tlaurion]

@marmarek you need to reset TPM instead of resealing totp from TPM menu

Normally, flow after installing OS is to run oem factory reset / re-ownership.

Doing OEM re-ownership resets TPM as well.

[marmarek]

Has it changed at some point? I think the current flow coded in that openQA test worked before (but not sure when, definitely not recently)...

[tlaurion]

Has it changed at some point? I think the current flow coded in that openQA test worked before (but not sure when, definitely not recently)...

I can check deeper in the next week but that code hasn't changed for 6 years. But string concatenation might be flaky here, while counter clearly doesn't exist here in shared output.

heads/initrd/bin/kexec-sign-config

Line 68 in 25d7b06

|| die "$paramsdir: Unable to create rollback file"

[tlaurion]

@marmarek I am getting lost in the refactoring under #1935.

Has it changed at some point? I think the current flow coded in that openQA test worked before (but not sure when, definitely not recently)...

Let's clarify here. You previously were able to reinstall an OS without resetting TPM? Can you detail what flow worked before and not anymore on OpenQA?

[marmarek]

Unfortunately I don't have logs old enough to answer definitely. But I do have git log that says I needed to added resetting TPM on OS reinstall after some heads update (I don't have exact data of the update anymore, unfortunately, it might be some weeks or even months before that commit date). Before that, it needed only to regenerate HOTP.

[tlaurion]

Has it changed at some point? I think the current flow coded in that openQA test worked before (but not sure when, definitely not recently)...

The kexec_rollback.txt is created upon creating a TPM LUKS DUK key in default boot, and makes sure and makes sure that what is signed under /boot cannot be rollback (replaced by a signed backup of /boot) when reading TPM counter to tmpdir, and then verifying it's existence through hash verification.

Looking at your qa change, qa checks are reactive to console output. video is not available anymore.

Qa check for default hotp pin, which changed recently when adapting code to nk3, which doesn't use gpg Admin PIN anymore but secret app distinct PIN, so prior check to default PIN might have changed if you switched to nk3.

Then qa check for TPM duk also depending on console output. Was TPM DUK set? A default boot entry set?

The error in OP shows lack of proper string concatenation and validation upon tpm counter read and tpm counter increment.

I will revisit this codepath to Adress it with a simple fix, seperating big PR into sub PR, but addressing this issue distinctively.

Tldr:

• kexec_rollback.txt can be backuped and restored alongside hotp counter in your hacky OS reinstallation QA test, yes. But I'm not sure this is a proper test for OS reinstallation, and a corner use case. If OS is reinstalled, I think Heads should actually do a little more it does (checking tpm unseal op possible, check hotp unseal but if failing check for other /boot kexec* files).
• As if prior test upon OS reinstall use case (not yours), totp unseal would succeed because same firmware parts in pcr2, same pubkey and trusted in pcr6, but then fail because hotp rollback (unsigned). Resealing hotp would fail later because of tpm counter could be different alongside everything else signed under /boot. If a previous default boot config was there, hotp reseal triggered a /boot resign, which read the counter, wrote it, hashed it and signed it alongside everything else /boot.

The simple solution still stands. Resetting tpm after tpm unsealing successful confirms through Tpmtotp that firmware in expected state. Resetting tpm creates new tpm counter and triggers a /boot hash and sign.

What do you think is missing following those explanation? What qa was reacting on and what did your changes meant from your observations? Why a tpm reset upon os reinstall is problematic in your use case simulation?