Ubuntu Core fails to boot after eMMC burn

[wills721]

Describe the bug

I purchased 6 CM5's (8GB RAM 32GB eMMC) from PiShop. On all 6 Ubuntu Core 24 (64bit) fails to boot after burning and on first boot.

Tried to update eeprom, no change. Instead of RPI Imager used 'dd' on Ubuntu, no change.

At boot up time I only have a picture of the error (attached).

The very very odd thing is 6 of the CM5's I got from PiShop all fail this way but one CM5 I happened to purchase from Amazon does not.

What could this be?

Steps to reproduce the behaviour

Burn Ubuntu Core 24 64bit on eMMC

Device(s)

Raspberry Pi CM5

Compute Module IO board.

No response

RPIBOOT logs

No response

Kernel logs

No response

Device UART logs

No response

[peterharperuk]

Does Raspberry PI OS boot on them?

[wills721]

Yes -- tried that too. 64bit Raspian boots fine @peterharperuk

[peterharperuk]

It sounds like the issue is with the image you're programming to the devices. I don't understand why you've raised an issue here?

[wills721]

The issue is not the image. As I said, it works on some CM5's but not all. Where should I put this then?

[pelwell]

If RPiOS boots on all of them, the issue is the image.

[wills721]

Ubuntu Core does not boot on all of them

[pelwell]

And that is a problem, but not with RPiOs, and not with usbboot. If Ubuntu Core even starts to boot, as it appears to, usbboot has probably done its job provided the image flashing completes and the device is unmounted correctly.

[wills721]

Well, ok. RIddle me this.

Case 1). CM5 purchased from Amazon burns and has appropriate partitions ubuntu-seed, ubuntu-boot, ubuntu-data and ubuntu-save
Case 2) CM5 purchased from PiShop burns and only has one partition ubuntu-seed.

Same exact process for each.

This is clearly why the PiShop ones do not boot.

What would be the cause of this and I will go there?

@pelwell

[peterharperuk]

I'm guessing it's the Ubuntu installer that's failing. RPi imager verifies the image has been written correctly. So after that's been done usbboot has done its job.

[wills721]

If so the installer indicates no error.

Is it possible differences with the eeprom bootloader can cause this?

[peterharperuk]

Anything is possible. Is there a difference in eeprom version? Your report doesn't mention that. It seems more likely that the Ubuntu core errors you have taken a photo of are caused by an issue in Ubuntu core. Are you seeing these errors on first boot? I would guess it's doing some setup on first boot and that's failing with the errors you are seeing

[wills721]

Yes, first boot fails on one but not the other.

[ 413.242442] I/O error, dev sda, sector 2592 op 0x0:(READ) flags 0x80700 phys_seg 28 prio class 2
[ 413.374277] sd 0:0:0:0: [sda] tag#0 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK cmd_age=0s
[ 413.374294] sd 0:0:0:0: [sda] tag#0 Sense Key : Not Ready [current]
[ 413.374300] sd 0:0:0:0: [sda] tag#0 Add. Sense: Medium not present
[ 413.374306] sd 0:0:0:0: [sda] tag#0 CDB: Read(10) 28 00 00 00 0a 20 00 00 08 00
[ 413.374309] I/O error, dev sda, sector 2592 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 2
[ 413.374321] Buffer I/O error on dev sda1, logical block 68, async page read
[ 413.374884] sda: detected capacity change from 61071360 to 0
[ 413.374905] sd 0:0:0:0: [sda] tag#0 access beyond end of device
[ 413.374914] I/O error, dev sda, sector 2458656 op 0x0:(READ) flags 0x80700 phys_seg 28 prio class 2
[ 413.380774] sd 0:0:0:0: [sda] tag#0 access beyond end of device
[ 413.380803] I/O error, dev sda, sector 2458656 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 2
[ 413.380811] Buffer I/O error on dev sda1, logical block 307076, async page read
[ 413.381131] sd 0:0:0:0: [sda] tag#0 access beyond end of device
[ 413.381140] I/O error, dev sda, sector 2459168 op 0x0:(READ) flags 0x80700 phys_seg 27 prio class 2
[ 413.381173] sd 0:0:0:0: [sda] tag#0 access beyond end of device
[ 413.381180] I/O error, dev sda, sector 2459168 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 2
[ 413.381186] Buffer I/O error on dev sda1, logical block 307140, async page read

After finding above, I think given there is no diff in image and no diff in process and seeing now I/O errors appears all I am facing are bad CM5s with corrupt eMMC. Interestingly, all 6 sent to me by PiShop all are corrupt in the same way. Nuts.

[popcornmix]

After finding above, I think given there is no diff in image and no diff in process and seeing now I/O errors appears all I am facing are bad CM5s with corrupt eMMC. Interestingly, all 6 sent to me by PiShop all are corrupt in the same way. Nuts.

But did you say RPiOS installs and runs on these devices? If so it doesn't sound like bad CM5s.

[timg236]

Yes, first boot fails on one but not the other.

[ 413.242442] I/O error, dev sda, sector 2592 op 0x0:(READ) flags 0x80700 phys_seg 28 prio class 2 [ 413.374277] sd 0:0:0:0: [sda] tag#0 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK cmd_age=0s [ 413.374294] sd 0:0:0:0: [sda] tag#0 Sense Key : Not Ready [current] [ 413.374300] sd 0:0:0:0: [sda] tag#0 Add. Sense: Medium not present [ 413.374306] sd 0:0:0:0: [sda] tag#0 CDB: Read(10) 28 00 00 00 0a 20 00 00 08 00 [ 413.374309] I/O error, dev sda, sector 2592 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 2 [ 413.374321] Buffer I/O error on dev sda1, logical block 68, async page read [ 413.374884] sda: detected capacity change from 61071360 to 0 [ 413.374905] sd 0:0:0:0: [sda] tag#0 access beyond end of device [ 413.374914] I/O error, dev sda, sector 2458656 op 0x0:(READ) flags 0x80700 phys_seg 28 prio class 2 [ 413.380774] sd 0:0:0:0: [sda] tag#0 access beyond end of device [ 413.380803] I/O error, dev sda, sector 2458656 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 2 [ 413.380811] Buffer I/O error on dev sda1, logical block 307076, async page read [ 413.381131] sd 0:0:0:0: [sda] tag#0 access beyond end of device [ 413.381140] I/O error, dev sda, sector 2459168 op 0x0:(READ) flags 0x80700 phys_seg 27 prio class 2 [ 413.381173] sd 0:0:0:0: [sda] tag#0 access beyond end of device [ 413.381180] I/O error, dev sda, sector 2459168 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 2 [ 413.381186] Buffer I/O error on dev sda1, logical block 307140, async page read

After finding above, I think given there is no diff in image and no diff in process and seeing now I/O errors appears all I am facing are bad CM5s with corrupt eMMC. Interestingly, all 6 sent to me by PiShop all are corrupt in the same way. Nuts.

What are those logs from? usboot or Ubuntu ?

As others have said usbboot is just exports the EMMC has a block device. The recommended image flashing tool simply writes the image and verifies the sha256 of the disk.,

If you think there is an issue with that then we would need a reproducible set of steps as follows

• Run rpiboot on a known good host e.g. RPi OS on a Pi3 or newer
• Use rpi-imager command line to write the image - don't use customisation
• Run rpiboot again (don't boot the OS) and verify the sha256 of the partitions on the block device

i.e. Verify whether the EMMC is being flashed correctly. This is not the right place to diagnose Ubuntu Core or other flashing tools but if you can provide a reproducible setup that shows the wrong disk data being flashed then that's of interest.

[wills721]

Yes, I understand. The logs are dmesg -w (kernel logs). This issue is not RPiBoot it is corrupt Cm5's.