Skip to content

Kernel oops when updating CVS firmware #22

@hughsie

Description

@hughsie

On my unreleased Dell ARL-based laptop I was able to do sudo fwupdtool --plugins intel-cvs install-blob 06CB0701.bin -vv which made the progress bar move encouragingly. When the update had reached 100% of the write:

FuStructIntelCvsStatus:
dev_state: 0x16
fw_upd_retries: 0x5
total_packets: 0x1dd1
num_packets_sent: 0x1dd1
fw_dl_finished: 0x0
fw_dl_status_code: 0x0

... then looking at sysfs, the /sys/devices/pci0000:00/0000:00:14.0/usb3/3-4/3-4:1.0/usbio-i2c.21.auto/i2c-14/i2c-INTC10E0:00 device went away, and then "came back" a few seconds later, where fwupdtool then froze. I saw this in dmesg:

[ 4584.541521] Intel CVS driver i2c-INTC10E0:00: cvs_ctrl_data_pre_show:Device fw version is 132.24.50.50
[ 4584.553813] Intel CVS driver i2c-INTC10E0:00: cvs_ctrl_data_pre_store:Full fw_buffer received. Start fw_download
[ 4584.561392] Intel CVS driver i2c-INTC10E0:00: cvs_dev_fw_dl:Enter
[ 4584.642931] Intel CVS driver i2c-INTC10E0:00: cvs_dev_fw_dl_data:Enter
[ 4697.017840] Intel CVS driver i2c-INTC10E0:00: cvs_dev_fw_dl_data:Exit with status:0x0, fw_st:0x16, cv_st:0x2
[ 4697.680263] usb 3-4: USB disconnect, device number 7
[ 4697.681040] Intel CVS driver i2c-INTC10E0:00: cvs_i2c_remove
[ 4697.681044] Intel CVS driver i2c-INTC10E0:00: cvs_i2c_remove:signal cvs_fw_dl_thread() to stop
[ 4697.681047] Intel CVS driver i2c-INTC10E0:00: cvs_i2c_remove:Wait for cvs_fw_dl_thread() to stop
[ 4697.782082] Intel CVS driver i2c-INTC10E0:00: cvs_dev_fw_dl:Exit with status:0x0
[ 4697.782084] Intel CVS driver i2c-INTC10E0:00: cvs_fw_dl_thread:cvs_dev_fw_dl cancelled
[ 4697.782085] Intel CVS driver i2c-INTC10E0:00: cvs_fw_dl_thread:Received close_fw_dl_task true
[ 4697.782087] Intel CVS driver i2c-INTC10E0:00: cvs_fw_dl_thread:Exiting fw_dl thread
[ 4697.782144] Intel CVS driver i2c-INTC10E0:00: cvs_i2c_remove:cvs_fw_dl_thread() stopped
[ 4698.588970] usbio-bridge 3-4:1.0: USB Bridge disconnected
[ 4698.805860] usb 3-4: new full-speed USB device number 8 using xhci_hcd
[ 4698.930239] usb 3-4: New USB device found, idVendor=06cb, idProduct=0701, bcdDevice= 1.00
[ 4698.930249] usb 3-4: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[ 4698.930252] usb 3-4: Product: SVP7500
[ 4698.930255] usb 3-4: Manufacturer: Synaptics , Inc
[ 4698.930257] usb 3-4: SerialNumber: 01.00.00.00
[ 4698.940840] usbio-bridge 3-4:1.0: bank:0 map:0x000010cf
[ 4698.940844] usbio-bridge 3-4:1.0: bank:1 map:0x00400c00
[ 4698.948224] Intel CVS driver i2c-INTC10E0:00: cvs_i2c_probe with i2c_client:000000008e715515
[ 4699.029268] Intel CVS driver i2c-INTC10E0:00: find_oem_prod_id: ACPI method OPID not found
[ 4699.029283] Intel CVS driver i2c-INTC10E0:00: find_shared_i2c: ACPI method IICS returned:0x0
[ 4699.247227] Intel CVS driver i2c-INTC10E0:00: cvs_i2c_probe:Transfer of ownership success
[ 4699.759779] usbio-bridge 3-4:1.0: acked wait timed out ret:0 timeout:500 ack:0
[ 4699.759791] i2c i2c-14: I2C_READ failed ret:-110
[ 4699.759794] i2c i2c-14: i2c raw read failed ret:-110
[ 4699.759798] Intel CVS driver i2c-INTC10E0:00: cvs_get_device_cap:Device protocol is 1.0
[ 4700.286774] usbio-bridge 3-4:1.0: acked wait timed out ret:0 timeout:500 ack:0
[ 4700.286787] i2c i2c-14: I2C_READ failed ret:-110
[ 4700.286790] i2c i2c-14: i2c raw read failed ret:-110
[ 4700.495837] usbio-bridge 3-4:1.0: bridge write failed ret:-110 total_len:0
[ 4700.495851] i2c i2c-14: i2c start failed ret:-110
[ 4700.702865] usbio-bridge 3-4:1.0: bridge write failed ret:-110 total_len:0
[ 4700.702873] i2c i2c-14: i2c start failed ret:-110
[ 4700.911055] usbio-bridge 3-4:1.0: bridge write failed ret:-110 total_len:0
[ 4700.911067] i2c i2c-14: i2c start failed ret:-110
[ 4701.118968] usbio-bridge 3-4:1.0: bridge write failed ret:-110 total_len:0
[ 4701.118977] i2c i2c-14: i2c start failed ret:-110
[ 4701.327060] usbio-bridge 3-4:1.0: bridge write failed ret:-110 total_len:0
[ 4701.327074] i2c i2c-14: i2c start failed ret:-110
[ 4701.534924] usbio-bridge 3-4:1.0: bridge write failed ret:-110 total_len:0
[ 4701.534934] i2c i2c-14: i2c start failed ret:-110
[ 4701.535084] usbio-i2c usbio-i2c.22.auto: hid INTC10B6 uid VIC0 new uid1
[ 4701.742995] usbio-bridge 3-4:1.0: bridge write failed ret:-110 total_len:0
[ 4701.743008] i2c i2c-15: i2c start failed ret:-110
[ 4701.951660] usbio-bridge 3-4:1.0: bridge write failed ret:-110 total_len:0
[ 4701.951673] i2c i2c-15: i2c start failed ret:-110
[ 4702.158968] usbio-bridge 3-4:1.0: bridge write failed ret:-110 total_len:0
[ 4702.158978] i2c i2c-15: i2c start failed ret:-110
[ 4702.366943] usbio-bridge 3-4:1.0: bridge write failed ret:-110 total_len:0
[ 4702.366952] i2c i2c-15: i2c start failed ret:-110
[ 4702.574983] usbio-bridge 3-4:1.0: bridge write failed ret:-110 total_len:0
[ 4702.574993] i2c i2c-15: i2c start failed ret:-110
[ 4702.782976] usbio-bridge 3-4:1.0: bridge write failed ret:-110 total_len:0
[ 4702.782989] i2c i2c-15: i2c start failed ret:-110
[ 4702.991019] usbio-bridge 3-4:1.0: bridge write failed ret:-110 total_len:0
[ 4702.991035] i2c i2c-15: i2c start failed ret:-110
[ 4703.198940] usbio-bridge 3-4:1.0: bridge write failed ret:-110 total_len:0
[ 4703.198950] i2c i2c-15: i2c start failed ret:-110
[ 4703.199023] usbio-bridge 3-4:1.0: USB Bridge device init success
[ 4703.471266] usbio-bridge 3-4:1.0: bridge write failed ret:-110 total_len:0
[ 4703.471281] i2c i2c-14: i2c start failed ret:-110
[ 4703.471287] Intel CVS driver i2c-INTC10E0:00: cvs_read_i2c:cmd:801 count:-5 (!=2)
[ 4703.974864] usbio-bridge 3-4:1.0: bridge write failed ret:-110 total_len:11
[ 4703.974878] usbio-gpio usbio-gpio.20.auto: usbio_gpio_write failed gpio_id:0 ret -110
[ 4703.974882] usbio-gpio usbio-gpio.20.auto: usbio_gpio_set_value offset:0 val:1 set value failed -5
[ 4704.583198] usbio-bridge 3-4:1.0: bridge write failed ret:-110 total_len:11
[ 4704.583223] ------------[ cut here ]------------
[ 4704.583225] kernel BUG at mm/slub.c:547!
[ 4704.583239] Oops: invalid opcode: 0000 [#1] PREEMPT SMP NOPTI
[ 4704.583246] CPU: 1 UID: 0 PID: 8040 Comm: fwupdtool Tainted: G        W  OE      6.12.9-200.fc41.x86_64 #1
[ 4704.583254] Tainted: [W]=WARN, [O]=OOT_MODULE, [E]=UNSIGNED_MODULE
[ 4704.583256] Hardware name: REDACTED
[ 4704.583260] RIP: 0010:kfree+0x2f0/0x360
[ 4704.583271] Code: 5d 41 5e 41 5f 5d e9 0f d6 ff ff 4d 89 f1 41 b8 01 00 00 00 48 89 d9 48 89 da 4c 89 e6 4c 89 ef e8 a5 f7 ff ff e9 34 fe ff ff <0f> 0b 89 c8 4c 8d 04 03 40 f6 c6 80 0f 84 46 ff ff ff 83 e6 08 0f
[ 4704.583275] RSP: 0018:ffffa65ae85d3820 EFLAGS: 00010246
[ 4704.583280] RAX: ffff9aa5c443a180 RBX: ffff9aa5c443a180 RCX: ffff9aa5c443a188
[ 4704.583283] RDX: 0000000033006001 RSI: ffffffffc0e37334 RDI: ffff9aa5c443a180
[ 4704.583286] RBP: ffffa65ae85d3870 R08: 0000000000000000 R09: 0000000000000000
[ 4704.583288] R10: 000000000000000b R11: 0000000000000000 R12: ffffcdb844110e80
[ 4704.583291] R13: ffff9aa5c004ba00 R14: ffffffffc0e37334 R15: 000000000000000b
[ 4704.583294] FS:  00007f52d43c0b80(0000) GS:ffff9ab50fc80000(0000) knlGS:0000000000000000
[ 4704.583303] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 4704.583307] CR2: 00007f28b19fe000 CR3: 00000001e035c003 CR4: 0000000000f72ef0
[ 4704.583311] PKRU: 55555554
[ 4704.583314] Call Trace:
[ 4704.583317]  <TASK>
[ 4704.583321]  ? __die_body.cold+0x19/0x27
[ 4704.583329]  ? die+0x2e/0x50
[ 4704.583336]  ? do_trap+0xca/0x110
[ 4704.583341]  ? do_error_trap+0x6a/0x90
[ 4704.583346]  ? kfree+0x2f0/0x360
[ 4704.583351]  ? exc_invalid_op+0x50/0x70
[ 4704.583357]  ? kfree+0x2f0/0x360
[ 4704.583361]  ? asm_exc_invalid_op+0x1a/0x20
[ 4704.583367]  ? usbio_control_xfer+0x244/0x350 [usbio]
[ 4704.583375]  ? usbio_control_xfer+0x244/0x350 [usbio]
[ 4704.583382]  ? kfree+0x2f0/0x360
[ 4704.583388]  usbio_control_xfer+0x244/0x350 [usbio]
[ 4704.583394]  usbio_transfer_internal+0x191/0x1c0 [usbio]
[ 4704.583401]  usbio_transfer+0x14/0x20 [usbio]
[ 4704.583407]  usbio_gpio_get_value+0x9f/0x120 [gpio_usbio]
[ 4704.583412]  gpiod_get_raw_value_commit+0x54/0x110
[ 4704.583417]  gpiod_get_value_cansleep+0x2c/0x50
[ 4704.583422]  cvs_release_camera_sensor_internal+0x90/0xf0 [intel_cvs]
[ 4704.583429]  cvs_get_fwver_vid_pid+0x56/0x70 [intel_cvs]
[ 4704.583437]  cvs_ctrl_data_pre_show+0x4f/0x70 [intel_cvs]
[ 4704.583445]  dev_attr_show+0x19/0x40
[ 4704.583450]  sysfs_kf_seq_show+0xa8/0xf0
[ 4704.583455]  seq_read_iter+0x11c/0x460
[ 4704.583460]  vfs_read+0x299/0x370
[ 4704.583468]  ksys_read+0x6d/0xf0
[ 4704.583475]  do_syscall_64+0x82/0x160
[ 4704.583483]  ? __x64_sys_poll+0xd0/0x180
[ 4704.583490]  ? syscall_exit_to_user_mode+0x10/0x210
[ 4704.583496]  ? do_syscall_64+0x8e/0x160
[ 4704.583502]  ? syscall_exit_to_user_mode+0x10/0x210
[ 4704.583506]  ? do_syscall_64+0x8e/0x160
[ 4704.583511]  ? vfs_write+0x28d/0x450
[ 4704.583519]  ? syscall_exit_to_user_mode+0x10/0x210
[ 4704.583523]  ? do_syscall_64+0x8e/0x160
[ 4704.583527]  ? syscall_exit_to_user_mode+0x10/0x210
[ 4704.583531]  ? do_syscall_64+0x8e/0x160
[ 4704.583535]  ? do_syscall_64+0x8e/0x160
[ 4704.583539]  ? do_syscall_64+0x8e/0x160
[ 4704.583543]  entry_SYSCALL_64_after_hwframe+0x76/0x7e
[ 4704.583547] RIP: 0033:0x7f52d5e219ca
[ 4704.583579] Code: 55 48 89 e5 48 83 ec 20 48 89 55 e8 48 89 75 f0 89 7d f8 e8 e8 56 f8 ff 48 8b 55 e8 48 8b 75 f0 41 89 c0 8b 7d f8 31 c0 0f 05 <48> 3d 00 f0 ff ff 77 2e 44 89 c7 48 89 45 f8 e8 42 57 f8 ff 48 8b
[ 4704.583583] RSP: 002b:00007ffd197a8b40 EFLAGS: 00000246 ORIG_RAX: 0000000000000000
[ 4704.583588] RAX: ffffffffffffffda RBX: 0000000031f475a0 RCX: 00007f52d5e219ca
[ 4704.583591] RDX: 000000000000001c RSI: 0000000031f48770 RDI: 0000000000000014
[ 4704.583593] RBP: 00007ffd197a8b60 R08: 0000000000000000 R09: 00007f52d5f23b20
[ 4704.583596] R10: 0000000000000000 R11: 0000000000000246 R12: 0000000031f0a580
[ 4704.583598] R13: 00007ffd197a9310 R14: 00007ffd197a9260 R15: 00007f52d695d670
[ 4704.583603]  </TASK>
[ 4704.583605] Modules linked in: intel_cvs(OE) i2c_dev overlay uinput rfcomm snd_seq_dummy snd_hrtimer nf_conntrack_netbios_ns nf_conntrack_broadcast nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject nft_ct nft_chain_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 ip_set sunrpc nf_tables qrtr bnep binfmt_misc vfat fat iwlmvm cs42l43_sdw regmap_sdw cs42l43 snd_hda_codec_hdmi snd_soc_dmic snd_sof_pci_intel_mtl mac80211 snd_sof_intel_hda_generic soundwire_intel soundwire_cadence snd_sof_intel_hda_common snd_soc_hdac_hda snd_sof_intel_hda_mlink snd_sof_intel_hda snd_sof_pci snd_sof_xtensa_dsp snd_sof libarc4 snd_sof_utils snd_hda_ext_core snd_soc_acpi_intel_match soundwire_generic_allocation snd_soc_acpi soundwire_bus snd_soc_core snd_compress x86_pkg_temp_thermal ac97_bus snd_pcm_dmaengine intel_powerclamp snd_hda_intel intel_ipu6_isys snd_intel_dspcfg coretemp snd_intel_sdw_acpi videobuf2_dma_sg snd_hda_codec videobuf2_memops videobuf2_v4l2 videobuf2_common
[ 4704.583700]  snd_hda_core v4l2_fwnode snd_hwdep v4l2_async dell_pc platform_profile videodev kvm_intel iTCO_wdt snd_ctl_led snd_seq hid_sensor_als intel_pmc_bxt mc spd5118 iTCO_vendor_support snd_seq_device btusb hid_sensor_trigger dell_laptop snd_pcm kvm processor_thermal_device_pci btrtl iwlwifi processor_thermal_device btintel processor_thermal_wt_hint processor_thermal_rfim hid_sensor_iio_common uas industrialio_triggered_buffer processor_thermal_rapl kfifo_buf dell_wmi btbcm intel_cstate intel_rapl_msr dell_smbios btmtk mei_gsc_proxy mei_wdt dcdbas dell_wmi_sysman usb_storage pcspkr dell_wmi_ddv intel_uncore bluetooth dell_smm_hwmon i2c_i801 snd_timer dell_wmi_descriptor firmware_attributes_class wmi_bmof intel_rapl_common e1000e industrialio i2c_smbus cfg80211 snd mei_me processor_thermal_wt_req thunderbolt mei soundcore intel_ipu6 processor_thermal_power_floor rfkill idma64 igen6_edac processor_thermal_mbox ipu_bridge i2c_usbio(OE) intel_pmc_core intel_skl_int3472_tps68470 gpio_usbio(OE) tps68470_regulator
[ 4704.583816]  int3403_thermal intel_vsec dptf_pch_fivr int340x_thermal_zone clk_tps68470 pmt_telemetry intel_skl_int3472_discrete int3400_thermal intel_hid pmt_class acpi_thermal_rel usbio(OE) intel_skl_int3472_common acpi_pad sparse_keymap acpi_tad joydev loop nfnetlink zram lz4hc_compress lz4_compress hid_sensor_hub intel_ishtp_hid xe drm_ttm_helper gpu_sched drm_suballoc_helper drm_gpuvm drm_exec i915 nvme nvme_core i2c_algo_bit drm_buddy ttm rtsx_pci_sdmmc mmc_core crct10dif_pclmul nvme_auth crc32_pclmul drm_display_helper crc32c_intel polyval_clmulni polyval_generic intel_ish_ipc ghash_clmulni_intel video sha512_ssse3 ucsi_acpi hid_multitouch sha256_ssse3 intel_vpu sha1_ssse3 typec_ucsi rtsx_pci intel_ishtp vmd typec cec i2c_hid_acpi i2c_hid wmi pinctrl_meteorlake serio_raw fuse
[ 4704.583916] Unloaded tainted modules: intel_cvs(OE):1 [last unloaded: intel_cvs(OE)]
[ 4704.583968] ---[ end trace 0000000000000000 ]---
[ 4704.583974] RIP: 0010:kfree+0x2f0/0x360
[ 4704.583982] Code: 5d 41 5e 41 5f 5d e9 0f d6 ff ff 4d 89 f1 41 b8 01 00 00 00 48 89 d9 48 89 da 4c 89 e6 4c 89 ef e8 a5 f7 ff ff e9 34 fe ff ff <0f> 0b 89 c8 4c 8d 04 03 40 f6 c6 80 0f 84 46 ff ff ff 83 e6 08 0f
[ 4704.583987] RSP: 0018:ffffa65ae85d3820 EFLAGS: 00010246
[ 4704.583992] RAX: ffff9aa5c443a180 RBX: ffff9aa5c443a180 RCX: ffff9aa5c443a188
[ 4704.583995] RDX: 0000000033006001 RSI: ffffffffc0e37334 RDI: ffff9aa5c443a180
[ 4704.583998] RBP: ffffa65ae85d3870 R08: 0000000000000000 R09: 0000000000000000
[ 4704.584001] R10: 000000000000000b R11: 0000000000000000 R12: ffffcdb844110e80
[ 4704.584004] R13: ffff9aa5c004ba00 R14: ffffffffc0e37334 R15: 000000000000000b
[ 4704.584007] FS:  00007f52d43c0b80(0000) GS:ffff9ab50fc80000(0000) knlGS:0000000000000000
[ 4704.584011] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 4704.584014] CR2: 00007f28b19fe000 CR3: 00000001e035c003 CR4: 0000000000f72ef0
[ 4704.584018] PKRU: 55555554
[ 4717.872431] usbio-bridge 3-4:1.0: header->cmd:2 != stub->cur_cmd:1
[ 4717.872443] usbio-bridge 3-4:1.0: failed to parse data: ret:-22 type:4 len: 6

It looks to me like some kind of refcounting problem -- as soon as the new i2c device appears fwupd queries the cvs_ctrl_data_pre file for the new version -- which seems to make the kernel explode -- and any subsequent access to /sys/devices/pci0000:00/0000:00:14.0/usb3/3-4/3-4:1.0/usbio-i2c.21.auto/i2c-14/i2c-INTC10E0:00 causes the same uninterruptable freeze of the fwupd daemon. Ideas?

Sub-issues

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions