Boot stuck when using ih8sn and applying OTA update on LOS19

[Th0masL]

Hi,

I'm using LineageOS 19.1 on a Samsung S10e (SM-970F).

I upgraded from LineageOS v18.1 to v19.1 around 2 weeks ago, when the most up-to-date version was 2022-06-09.

I followed the precise instructions on LineageOS website, and everything was working normally.

This week I received the notification that an OTA update was available for the version 2022-06-16, so I did it.

My phone ended up in bootloop, and the only thing I was able to access was Download mode or Recovery.

No matter what I tried, I just couldn't get the system to start normally, so I had to Format Data, and re-flash with the ROM 2022-06-09.

I continued doing some testing, to try to understand what could be the reason of this failure of OTA update, and basically the only moment I was able to OTA updates is if I delete the ih8sn files created by the push.sh script.

When ih8sn is installed, my phone is stuck in bootloop after the first OTA update.

Without ih8sn installed, I was able to do the 2 OTA updates that were pending : from 2022-06-09 -> 2022-06-16 -> 2022-06-23.

I'm happy to try to do more testing, so feel free to let me know if you want me to provide some logs or things like that.

Thanks !

[Th0masL]

I think that the problem might be coming from the script 60-ih8sn.sh, which is responsible for backing up and then restoring the ih8sn files during an upgrade.

I'll try to deploy all the ih8sn files except this upgrade script, and then I'll try to do the OTA updates to see if it works.

Of course, I'll then have to manually re-install ih8sn, but that's fine.

Update: Actually the upgrade script 60-ih8sn.sh is the thing that should help to avoid breaking the OTA updates, so skipping this script is definitely not helping.

[Th0masL]

I'm posting the logs here, in case we can find something:
last_log: https://pastebin.pl/view/7b03570f
last_kmsg: https://pastebin.pl/view/2a9d945e

[Kaalip]

Hey, i assume you either executed these commands or used the scripts included.

adb wait-for-device root
adb wait-for-device remount
adb wait-for-device push 60-ih8sn.sh /system/addon.d/
adb wait-for-device push ih8sn /system/bin/
adb wait-for-device push ih8sn.rc /system/etc/init/
adb wait-for-device push ih8sn.conf /system/etc/

After the files are pushed onto your device before reboot you need to execute
adb enable-verity
Otherwise the phone will bootloop after OTA

Hope this helps.

[Th0masL]

That's correct, I've been using exactly the list of 6x top commands.

I'll give a try with the command adb enable-verity, to see if that helps, and I'll let you know

[Th0masL]

This is the message I get when I try to run adb enable-verity :

> adb enable-verity
enable-verity only works for userdebug builds

I guess I could also delete all the ih8sn files before applying the OTA upgrade, and that would do the job, right ?

Isn't the goal of the script 60-ih8sn.sh, which is supposed to make a backup of the files before the upgrade and restore them after the upgrade ? Maybe this 60-ih8sn.sh script is not working as intended on some phones/models ? 🤔

[Kaalip]

This is the message I get when I try to run adb enable-verity :

> adb enable-verity
enable-verity only works for userdebug builds

Yes it should say that, and after it
Disabling overlayfs
That is what should do the trick
adb remount enables the overlayFS and that is not compatible with OTA
Here is more on the subject.
https://android.googlesource.com/platform/system/core/+/master/fs_mgr/README.overlayfs.md

[Th0masL]

In my case it's only saying enable-verity only works for userdebug builds, and not Disabling overlayfs

[Th0masL]

Maybe the problem with the add-on script 60-ih8sn.sh can be related to this ticket https://gitlab.com/LineageOS/issues/android/-/issues/4469

[Kaalip]

Okay then your overlayFS isn't enabled, can't help you further with that then.

[Th0masL]

As expected, deleting the ih8sn files just prior to doing the OTA updates is preventing the boot loop.

I'll make a small PR on the script to be able to pass a --remove parameter that will automatically delete the few files that got created.

[Th0masL]

This comment seems very relevant to the problem :
https://gitlab.com/LineageOS/issues/android/-/issues/4469#note_860943611

After OTA update addon.d files automatically changes string /tmp/backuptool.functions to postinstall/tmp/backuptool.functions which cause backuptool does not work after the next OTA update. I think that it behavior abnormal.

I think I indeed saw the problem of bootloop after OTA only starting the 2nd OTA update, so that would be spot on.

[Th0masL]

The problem seems to be that on my phone, the script /tmp/backuptool.functions that is used during the OTA update by 60-ih8sn.sh does not exist anywhere on the disk.

1|beyond0lte:/ # find / -type f -name "60-ih8sn.sh" 2>/dev/null                                                                            
/system/addon.d/60-ih8sn.sh

1|beyond0lte:/ # find / -type f -name "ih8sn" 2>/dev/null                                                                                  
/system/bin/ih8sn

1|beyond0lte:/ # find / -type f -name "ih8sn.rc" 2>/dev/null                                                                               
/system/etc/init/ih8sn.rc

1|beyond0lte:/ # find / -type f -name "backuptool.functions" 2>/dev/null                                                                   
< not found>

Is this file supposed to be present at all times ? Or is it supposed to be created dynamically during the OTA update ?

[luk1337]

It's unpacked by the OTA package.

[Th0masL]

Ok so here is what I did:

1. First install:

   • LOS 19 v2022-06-09
   • MindTheGAPPs

2. Do not install ih8sn just yet

3. Here is the content of the addon.d files before the first OTA update :

   beyond0lte:/ # ls /system/addon.d/
   30-gapps.sh  50-lineage.sh
   beyond0lte:/ # cd /system/addon.d/
   beyond0lte:/system/addon.d # grep -rni '.' -e 'backuptool.functions'
   ./30-gapps.sh:7:. /tmp/backuptool.functions
   ./50-lineage.sh:10:. /tmp/backuptool.functions
   

   Prior to the OTA upgrade, we can see that the path for the file backuptool.functions is in the /tmp folder, which is expected.

4. Do OTA upgrade to v2022-06-16

5. Reboot, still not install ih8sn

6. Here is the content of the addon.d files after the fist OTA update:

   beyond0lte:/ # ls /system/addon.d/
   30-gapps.sh  50-lineage.sh
   beyond0lte:/ # cd /system/addon.d/
   beyond0lte:/system/addon.d # grep -rni '.' -e 'backuptool.functions'
   ./30-gapps.sh:7:. /tmp/backuptool.functions
   ./50-lineage.sh:10:. /tmp/backuptool.functions
   

   After to the first OTA upgrade, the path still look good, nothing to worry here.

7. Install ih8sn, reboot

8. The phone is in bootloop, with only access to Download Mode or Recovery

9. Go to Recovery, mount the system partition, delete the ih8sn files

   beyond0lte:/ # find / -type f 2>/dev/null | grep ih8sn
   /mnt/system/system/addon.d/60-ih8sn.sh
   /mnt/system/system/etc/init/ih8sn.rc
   /mnt/system/system/etc/ih8sn.conf
   /mnt/system/system/bin/ih8sn
   
   1|beyond0lte:/ # mount -o rw,remount /mnt/system
   
   beyond0lte:/ # rm /mnt/system/system/addon.d/60-ih8sn.sh
   beyond0lte:/ # rm /mnt/system/system/etc/init/ih8sn.rc
   beyond0lte:/ # rm /mnt/system/system/etc/ih8sn.conf
   beyond0lte:/ # rm /mnt/system/system/bin/ih8sn
   

10. Reboot the phone

11. It starts normally and everything works

12. Which logs do you want me to give you ?