[pull] trunk from spiceai:trunk#859
Merged
Merged
Conversation
* feat: enhance metadata handling for datasets and views with comments * feat: propagate source comments into metadata * Enhance metadata handling for MySQL, Postgres, and Snowflake connectors - Updated MySQL and Postgres connectors to include source type metadata for columns. - Modified the schema enrichment process to incorporate partition and clustering metadata in Snowflake. - Refactored field metadata handling to support additional metadata keys across various connectors. - Improved the ADBC connector to retrieve and store schema metadata, including comments and source types. - Renamed functions related to metadata enrichment for clarity and consistency. * refactor: improve formatting and readability of metadata handling in MySQL, Postgres, Snowflake, and ADBC connectors * refactor: reorganize metadata key imports and improve test assertions for schema validation * refactor: update metadata handling to use metadata_value_to_string for improved consistency * feat: retrieve token from token provider in SqlWarehouseApi * refactor: simplify metadata enrichment by using metadata_enriched_table_provider function * refactor: streamline metadata retrieval in DatasetBuilder and ViewBuilder * feat: add data_components and tracing dependencies to MySQL and Postgres connectors * refactor: streamline MySQL and Postgres metadata handling by utilizing dedicated functions for comment extraction * feat: introduce metadata module with value serialization functions * refactor: enhance type definitions for MySQL schema metadata and improve SQLite connection documentation * refactor: rename ADBC factory field * fix: preserve federated adaptor pushdown * perf: avoid cloning field metadata maps * Fix test * Properly unwrap MetadataEnrichedTableProvider * Lint * Lint * Lint --------- Co-authored-by: Viktor Yershov <viktor@spice.ai>
* chbench-driver: Align schema & supplier loader with BenchBase Aligns `chbench-driver` DDL and supplier data generation with the [BenchBase CH-Benchmark reference](https://github.com/cmu-db/benchbase). ### Schema DDL - Add `NOT NULL` to all non-PK columns across all 12 tables - Add `DEFAULT CURRENT_TIMESTAMP` to `c_since`, `h_date`, `o_entry_d` - Set `ol_delivery_d` to `NULL DEFAULT NULL`, `o_carrier_id` to `DEFAULT NULL` - `ol_quantity`: `INT` → `DECIMAL(6,2)` - `n_name`, `r_name`, `su_name`, `su_phone`: `CHAR` → `VARCHAR` (avoids DuckDB/DataFusion trailing-space comparison issues) - `n_comment`, `r_comment`: add `NOT NULL` ### Supplier loader - `su_nationkey`: draw from all 62 CH-Benchmark nations (was 25 — left nations 25–61 with zero suppliers) - `su_acctbal`: 100.00–10,000,000.00 (was −999.99–9,999.99), matching BenchBase `fixedPoint(2, 10000, 1000000000)` - `n_comment`/`r_comment`: insert `''` instead of `NULL` * Exclude Q21 * Fix lint * Report tpmc * Fix lint
…l test spicepod (#10984)
The chat proc was sending messages before confirming the spice chat process was ready, causing a "spawn id not open" error when the process exited between responses.
* fix: handle EXISTS/NOT EXISTS subqueries in federation analyzer * Update * Update * Update sha
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
See Commits and Changes for more details.
Created by
pull[bot] (v2.0.0-alpha.4)
Can you help keep this open source service alive? 💖 Please sponsor : )