Surfaced by an automated multi-agent correctness audit; verified against the code.
Summary
The MySQL metadata provider unconditionally returns begin_snapshot: None, partial_max: None, schema_version: None for every data file, whereas the SQLite and Postgres providers probe for those catalog features and project the real values when present.
Location
src/metadata_provider_mysql.rs:54 (decode_table_file) and the duplicate inline decode near line 918- MySQL
get_table_files_for_select / get_table_file_metadata_page SQL never selects data.partial_max, a ducklake_schema_versions-derived schema_version, or projects data.begin_snapshot (only used in WHERE)
- contrast
src/metadata_provider_sqlite.rs (~332-410) and src/metadata_provider_postgres.rs (~261-336), which probe (pragma_table_info / information_schema) and degrade to NULL only when genuinely absent
Failure scenario
Against a MySQL-backed catalog that has undergone compaction (populating partial_max and ducklake_schema_versions), CDC reads (ducklake_table_changes / ducklake_table_deletions) and compaction candidate selection (which checks schema_version.is_some()) silently treat every file as non-partial / single-schema-version — producing incorrect CDC windows or skipped candidates, with no error and diverging from the identical catalog read via SQLite/Postgres.
Suggested fix
Mirror the SQLite/Postgres pattern in the MySQL provider: probe for the partial_max column and ducklake_schema_versions table, and project the real values in the SQL + decode_table_file when present.
Severity: High (silent wrong CDC/compaction metadata on the MySQL read backend).
Surfaced by an automated multi-agent correctness audit; verified against the code.
Summary
The MySQL metadata provider unconditionally returns
begin_snapshot: None,partial_max: None,schema_version: Nonefor every data file, whereas the SQLite and Postgres providers probe for those catalog features and project the real values when present.Location
src/metadata_provider_mysql.rs:54(decode_table_file) and the duplicate inline decode near line 918get_table_files_for_select/get_table_file_metadata_pageSQL never selectsdata.partial_max, aducklake_schema_versions-derivedschema_version, or projectsdata.begin_snapshot(only used inWHERE)src/metadata_provider_sqlite.rs(~332-410) andsrc/metadata_provider_postgres.rs(~261-336), which probe (pragma_table_info/information_schema) and degrade to NULL only when genuinely absentFailure scenario
Against a MySQL-backed catalog that has undergone compaction (populating
partial_maxandducklake_schema_versions), CDC reads (ducklake_table_changes/ducklake_table_deletions) and compaction candidate selection (which checksschema_version.is_some()) silently treat every file as non-partial / single-schema-version — producing incorrect CDC windows or skipped candidates, with no error and diverging from the identical catalog read via SQLite/Postgres.Suggested fix
Mirror the SQLite/Postgres pattern in the MySQL provider: probe for the
partial_maxcolumn andducklake_schema_versionstable, and project the real values in the SQL +decode_table_filewhen present.Severity: High (silent wrong CDC/compaction metadata on the MySQL read backend).