2025-10-16 kernel meeting notes

[zachschuermann]

Summary

Attendees

@zachschuermann @nicklan @mightyshazam @hntd187 @OussamaSaoudi @rtyler @DrakeLin

notes stream

• is there need for different Metadata/Protocol constructors? we will make them just directly pub I think?
  • ask for DV/column mapping APIs? ask for more usable APIs in default engine instead of just kernel
  • rules unspecified for schema evolution? (not in protocol)
  • tradeoff between details and flexibility vs abstraction
• crate split?
• 0.17.0?
• kyjah: FFI engine
• @rtyler: initial regressions fixed in delta-rs (panics, memory issue)
  • starting more frequent release cycles
  • ask for higher-speed cadence? pull in kernel updates

todos from last time

• follow-up: do a test binary to hand to tyler

leftovers/follow-ups

• @zachschuermann to update summer 2025 kernel work #1184
• kernel side follow up: to check on checkpoint data iterator shouldn't return batches with different schemas #1099
• ask: why does daft/ray need serializing/deserializing ListedLogFiles: need Snapshot to be serializable/deserializable
  • basically just need some IO-less way to make these things
  • consumers: Daft (delta-rs) and Ray (deltalake/python)
• follow up on Urls discussion?