RPi5 with Trixie and IMX477 may cause RPI5 failures if image captures are back to back.

[illigtr]

After rebuilding one of my indi-allsky cameras with an RPi5, 8GB, IMX477, NVMe HAT and 500 GB SSD, I have been experiencing RPi failures at night. The web interface to indi-allsky stops responding. I can't SSH in, Samba shares are gone. Only a cold boot restores the system. However, the system will only last 1-3 hours before it fails again.

Strangely this system ran flawlessly during 2 days of dark image calibration and again during 2 days of image capture, only night time captures causes the entire system to fail/freeze.

My day images are 120 seconds apart, night time images are 30 seconds apart. This was my first clue.

Looking at syslog, I noticed these entries just before the system becomes unresponsive... These are occurring about every 30 seconds, in batches, but not all the time.

2026-04-10T21:56:21.769609-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T21:56:51.825607-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T21:57:21.833606-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T21:57:51.837618-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T21:58:21.857606-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T21:58:51.889615-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T21:59:21.933609-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T21:59:51.949606-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:00:21.937609-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:00:32.397607-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:01:02.437616-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:01:32.481628-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:02:02.505630-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:02:32.553604-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:03:02.537619-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:03:32.565605-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:04:02.557611-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:04:32.569608-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:05:02.573606-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:05:32.577606-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:06:02.617607-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:06:32.629613-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:07:02.673601-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:07:32.689608-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:08:02.729608-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:08:32.725625-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:09:02.781611-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:09:32.793606-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:10:02.801607-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:10:32.805612-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:11:02.841601-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:11:32.857607-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:12:02.869611-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:12:32.885607-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:13:02.877638-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:13:32.893610-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:14:02.933609-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:14:32.981605-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:15:02.997605-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:15:33.045614-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:15:43.533610-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:16:13.525620-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:16:43.569607-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:17:01.463053-04:00 indi-allsky CRON[2998]: (root) CMD (cd / && run-parts --report /etc/cron.hourly)
2026-04-10T22:17:13.593635-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:17:43.645623-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:18:13.649605-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:18:43.641613-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:19:13.637604-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:19:43.681612-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:20:13.685605-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:20:43.709610-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:21:13.713614-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:21:43.753609-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:22:13.817609-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:22:43.861608-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:23:13.901605-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:23:43.901609-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:24:13.921606-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:24:43.917605-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:25:13.961608-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:25:43.969606-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:26:14.029604-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:26:44.049631-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:27:14.081607-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:27:44.093612-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:28:14.077602-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:28:44.117605-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:29:14.137614-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:29:44.149613-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:30:14.165608-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:30:44.153604-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:30:54.633628-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:31:24.637611-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:31:54.645607-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:32:24.645605-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:32:54.673606-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:33:24.713600-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:33:54.717606-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:34:24.721602-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:34:54.773619-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:35:24.817611-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:35:54.829605-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:36:24.849612-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:36:54.845608-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:37:24.893606-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:37:54.909605-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:38:24.905604-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:38:54.913604-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:39:24.933603-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:39:54.933618-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:40:24.933610-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:40:54.945616-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:41:24.985610-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:41:54.989607-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:42:24.993616-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:42:55.005609-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:43:25.001603-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:43:55.017602-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:44:25.029603-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:44:55.013608-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:45:25.021606-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:45:55.037611-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:46:05.533619-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:46:35.513608-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:47:05.533606-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:47:35.565611-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:48:05.565607-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:48:35.621605-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:49:05.661605-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:49:35.701608-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:50:05.749603-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:50:35.753601-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:51:05.761605-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:51:35.765607-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:52:05.773608-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:52:35.801609-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:53:05.793611-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:53:35.793623-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:54:05.793608-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:54:35.825606-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:54:56.806916-04:00 indi-allsky wpa_supplicant[722]: wlan0: WPA: Group rekeying completed with e8:9c:25:c9:43:90 [GTK=CCMP]
2026-04-10T22:55:05.853600-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:55:35.853608-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:56:05.889611-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:56:35.909612-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:57:05.917608-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:57:35.953604-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:58:05.977610-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:58:35.969605-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:59:06.021606-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T22:59:36.065605-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:00:06.125605-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:00:36.121608-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:01:06.149610-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:01:16.649620-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:01:46.645607-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:02:16.629606-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:02:46.637607-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:03:16.633611-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:03:46.653614-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:04:16.649606-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:04:46.701605-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:05:16.705619-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:05:46.761604-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:06:16.773602-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:06:46.777601-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:07:16.765609-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:07:46.785608-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:08:16.797639-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:08:46.813645-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:09:16.857605-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:09:46.853631-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:10:16.857661-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:10:46.861648-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:11:16.921608-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:11:46.889623-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:12:16.897603-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:12:46.941609-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:13:16.981617-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:13:46.993605-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:14:17.001616-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:14:47.021608-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:15:17.009604-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:15:47.053605-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:16:17.073607-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:16:27.553614-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:16:57.589605-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:17:01.822366-04:00 indi-allsky CRON[4424]: (root) CMD (cd / && run-parts --report /etc/cron.hourly)
2026-04-10T23:17:27.585612-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:17:57.593607-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:18:27.597606-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:18:57.613604-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:19:27.601614-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:19:57.605603-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:20:27.613604-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:20:57.661606-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:21:27.685606-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:21:57.693607-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:22:27.689608-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:22:57.705607-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:23:27.709612-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:23:57.717603-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:24:27.721603-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:24:57.729605-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:25:27.733609-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:25:57.737607-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:26:27.749604-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:26:57.757611-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:27:27.773611-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:27:57.773602-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:28:27.813611-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:28:57.785614-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:29:27.805606-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:29:57.813606-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:30:27.825611-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:30:57.821606-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:31:27.837605-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:31:38.321616-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:32:08.365602-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:32:38.389608-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:33:08.429605-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:33:38.445605-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:34:08.449606-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:34:38.449613-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:35:08.457605-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:35:38.461600-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:36:08.469602-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:36:38.493608-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:37:08.501608-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:37:38.561605-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:38:08.581610-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:38:38.577606-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:39:08.585609-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:39:38.629607-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:40:08.649607-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:40:38.649612-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:41:08.645604-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:41:38.653611-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:42:08.661607-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:42:38.665604-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:43:08.681612-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:43:38.689603-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:44:08.701606-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:44:38.749607-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:45:08.785605-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:45:38.773612-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:46:08.789604-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:46:38.797609-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:46:49.285622-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:47:19.305604-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:47:49.325602-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:48:19.321600-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:48:49.369619-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
2026-04-10T23:49:19.373609-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using a link rate of 900 Mbps
                                                                                                           

CoPilot suggests these should not be occurring this frequently and that the OS is trying to renegotiate the CSI-2 link far too frequently.

Also found these with regards to the imx477

Line 23046: 2026-04-11T22:31:53.010495-04:00 indi-allsky kernel: rp1-cfe 1f00128000.csi: Using sensor imx477 11-001a for capture
	Line 23363: 2026-04-11T22:32:02.462780-04:00 indi-allsky wireplumber[1297]: [0:00:13.195115212] [1399]  INFO IPAProxy ipa_proxy.cpp:180 Using tuning file /usr/share/libcamera/ipa/rpi/pisp/imx477.json
	Line 23364: 2026-04-11T22:32:02.503753-04:00 indi-allsky wireplumber[1297]: [0:00:13.236108638] [1399]  INFO Camera camera_manager.cpp:223 Adding camera '/base/axi/pcie@1000120000/rp1/i2c@80000/imx477@1a' for pipeline handler rpi/pisp
	Line 23365: 2026-04-11T22:32:02.503849-04:00 indi-allsky wireplumber[1297]: [0:00:13.236134823] [1399]  INFO RPI pisp.cpp:1181 Registered camera /base/axi/pcie@1000120000/rp1/i2c@80000/imx477@1a to CFE device /dev/media1 and ISP device /dev/media2 using PiSP variant BCM2712_C0
	Line 23428: 2026-04-11T22:32:03.572222-04:00 indi-allsky pipewire[1295]: [0:00:14.304543397] [1673]  INFO IPAProxy ipa_proxy.cpp:180 Using tuning file /usr/share/libcamera/ipa/rpi/pisp/imx477.json
	Line 23429: 2026-04-11T22:32:03.604027-04:00 indi-allsky pipewire[1295]: [0:00:14.336390767] [1673]  INFO Camera camera_manager.cpp:223 Adding camera '/base/axi/pcie@1000120000/rp1/i2c@80000/imx477@1a' for pipeline handler rpi/pisp
	Line 23430: 2026-04-11T22:32:03.604096-04:00 indi-allsky pipewire[1295]: [0:00:14.336414915] [1673]  INFO RPI pisp.cpp:1181 Registered camera /base/axi/pcie@1000120000/rp1/i2c@80000/imx477@1a to CFE device /dev/media1 and ISP device /dev/media2 using PiSP variant BCM2712_C0
	Line 23462: 2026-04-11T22:42:38.043912-04:00 indi-allsky python3[1879]: BaseClientQt: Error. Unable to find driver libcamera_imx477

It would appear that night images, back to back exposures close to 30 seconds long are causing issues with the IMX477 on RPi5/Trixie, but not so, during the day when exposures are 120 seconds apart for only less than a millisecond.

Has this been experienced elsewhere?

I know scriptorron (author indi_pylibcamera) has a setting to force reset the IMX477 between frames. Not sure if applicable here, currently I am testing libcamera instead of indi.

I also have this curious entry in the syslog

2026-04-10T10:35:28.543490-04:00 indi-allsky python3[1817]: /home/pi/indi-allsky/virtualenv/indi-allsky/lib/python3.13/site-packages/rawpy/__init__.py:72: RuntimeWarning: rawpy is being used in a process created by multiprocessing with the 'fork' start method. This can cause deadlocks when OpenMP is enabled (which it is in this build). Please use mp.set_start_method('spawn') or mp.set_start_method('forkserver') at the start of your main module. See https://github.com/letmaik/rawpy#im-getting-deadlocks-when-using-multiprocessing-on-linux for more information.

I will be disassembling this unit and checking/replacing the CSI camera cable and ensuring it is properly seated at both ends.

[aaronwmorris]

Are you able to SSH to the system when it stops responding?

[illigtr]

No. Per my initial post, no SSH, no ping, no VNC, no route, no GUI. Logs show that the RPi simply is hung. Watchdog daemon doesn't even reboot the RPi. Historical data (logs) show no new entries from hang to cold boot.

[aaronwmorris]

I do not see anything in the logs you showed that indicates a problem. It all looks like standard background logging. You may have to observe the console when you get the system down to figure out what is happening.

My system using the IMX477 is on a Pi4 which would probably not really help here.

[illigtr]

Observing the console on a remotely installed camera is problematic, for obvious reasons. I will bring the unit it and attempt to recreate the issue, with a connected monitor, while I try to capture back-to-back 30 second exposures. Perhaps I can trap the actual console message.

I let the unit run overnight last night, even though rainy/overcast. Capturing data in Home Assistant is useful for this type of troubleshooting.

You can see the system crashed three times at 4:44, 5:01 and 5:43. I created an automation with a smart plug that if the unit stops responding to a ping after 5 minutes, then power off, wait 1 minute and power up.

Also, of note, is the camera temperature when the fails started to happen.

Here are the logs

indi-allsky_syslog_log_20260413_082939.txt.gz
indi-allsky_support_info_2026413_82932.txt.gz
indi-allsky_kern_log_20260413_082942.txt.gz

[aaronwmorris]

Yeah, I do not see anything out of the ordinary. The system does not appear to be logging anything when it locks up. It makes me think that it might have something to do with the NVMe system. If the drive becomes disconnected/locks/etc, it would not be able to log anything.

[illigtr]

Ok. Still investigating on my end...

[illigtr]

UPDATE:

I have run full stress tests with fio and stress-ng. The RPi5 hangs with no console messages, both under NVMe HAT and SSD or with only a newly imaged SD card. So it appears my RPi5 is defective and I will be requesting replacement under warranty. In the meantime, I will swap out with another RPi5 to continue testing.

[illigtr]

I have another RPi5 running with the same camera/HAT/SSD and I have not seen the issue return. So it appears my first RPi5 PCB was the issue.

I have noticed some strange stacking results occurring. I checked the actual images (FITS save, every image) and the original images are fine, so something with stacking... as you can see some stacked images have been resized.

[aaronwmorris]

The clouds are causing incorrect registration and is improperly scaling one if the images in the stack. There is already a function to bypass images if there is too much rotation. I will have to implement another cutoff if the image are scaled too much.

[aaronwmorris]

Merged #2873 to limit image scale changes.

[illigtr]

I have not seen the incorrect scaling again, but I haven't captured a fully overcast night again. I did capture last night, a reasonably good night. However, you will note by looking at the time lapse that stacking RESTARTS every few frames... look closely at the trees and you can see when there are no duplicated trees that stacking failed, then it restarts. This causes a jumpy video clip. I can't determine any reason for this.

Also note the star counts rising and dropping, again, no reason for this: cloudless, moonless night. Gain and exposure stay steady.

Here is the YT cllip, jumpiness starts after 11pm:
https://www.youtube.com/watch?v=jD8MQTTVnZ8

[illigtr]

Exposure flapping. Perhaps I should open a new discussion or issue, in particular if stacking is used.
I believe the issue is that the exposure is based on the stacked image, which can be 5 exposures behind, please confirm.
I have set the queue to the lowest permissible numbers, so not sure if queue also aggravates the problem.

If I were to add a light sensor pointing at 90d altitude, could the sensor be used to calculate each next image exposure and gain instead?
I think using older than immediate last image, either due to stacking and/or queue is root cause to exposure flapping.

Alternatively, what about taking a short/shorter disposable frame with the only purpose to calculate exposure for the next frame? If exposure/gain is 30s/22g, could one take a 5s/22g disposable image to calculate accurately the correct exposure adjustment?

Just trying to determine some way to avoid flapping and video clip flickering....

[aaronwmorris]

The issue with the stacking restarts are probably due to image registration failures. If a registration solution is not found, the image is discarded from the stack. If the registration takes too long, the whole set is discarded. When there are less frames to stack, less stars will be detected. My guess is registration is failing/timing out periodically.

As for the exposure flapping, there does need to be some sort of hysteresis to prevent exposure flapping. This is a significant challenge. I am not sure using a fixed exposure and gain combination is the right answer (for reasons below).

Exposure is a fairly obvious measurement. It is well defined and usually (not always, some cameras misbehave) has a linear effect. Doubling the exposure generally doubles the brightness. Gain is a different beast. Specifically for libcamera and the IMX477, the reality is: I have no idea what the gain value means. I have not dug deep enough in the libcamera white papers to figure out what it means. With a ZWO camera, almost all of their modern cameras, gain is an objective value with a logarithmic measurement in 0.1 decibels (dB). ZWO gain values usually range from 0-300 (for analog only gain). Given dB calculations, I believe that every 60.2 ZWO gain units (6.02 dB) should have the same effect as doubling the exposure. (I might be wrong on this.)

The libcamera gain values range from 1.0 to 22.26 (for IMX477). Remember, ZWO values start at 0, so there is a different basis with libcamera. I have seen some references that the gain value of libcamera corresponds to ISO, eg gain of 1.0 is ISO 100. This could make sense given libcamera/rpicam-app tend to favor the ISO terminology for people coming from the photography world. This would mean that each doubling of gain (ie ISO) would be the equivalent of 6.02 dB of increase and be the equivalent of doubling the exposure.

I really need to get to the bottom of this.

[illigtr]

Ok, Your explanation of the stacking/star count issue is very helpful. What is the timeout of registration? ( I presume I could toy with this variable a bit to see if I can improve matters)>

Exposure. I understand the need for hysterisis, I'm just wondering if our basis for the next exposure change is based on an image that is not much older, due to either stacking and/or queue depth.

I think you are suggesting this exposure flapping is more prevalent to the IMX477 than other cameras, due to how gain should be interpreted. FWIW CoPilot says that imx477 under libcamera is neither logarithmic nor ISO based. Gain is merely a factor of analog amplification ( Gain 1 = 1 x analog amp, Gain 10 = 10 x analog amp) .

[illigtr]

A Q&D analysis showing 12 "steps" of the Exposure Priority Gain (x-axis) and analog amplification (y-axis) for both camera families. My guess is you are not dividing the gain range the same way if the camera is logarithmic vs linear?