[pull] trunk from spiceai:trunk#653
Merged
Merged
Conversation
* Enhance error handling and sanitization in DataFusion integration - Added `sanitize_datafusion_error_message` utility to clean error messages by removing specific bug report suffixes. - Updated various error displays across multiple modules to utilize the new sanitization function, improving clarity and user-friendliness of error messages. - Included `format_datafusion_error` function to standardize error formatting in DataFusion-related errors. - Ensured consistent error handling in the `Error` enums across different components, enhancing maintainability and readability. - Updated `Cargo.toml` files to include the new `util` crate dependency where necessary. * fix: remove unnecessary newline in find_datafusion_root function * refactor: replace sanitize_datafusion_error_message with format_datafusion_error for improved error formatting * refactor: Organize imports and improve error message handling in DataFusion * test: Update table creation tests to return errors on configuration changes * fix: Update VortexConfig initialization to use struct syntax for clarity
* Add conditional logging for partition assignments Less noise, avoid `INFO runtime::cluster::partition::scheduler_task: Found unassigned partitions unassigned_count=0` * Update scheduler_task.rs
* use 'properly early exit on SIGTERM' * update mistral to spiceai branch
* Ensure we query one and only one partition per request * fix merge * clippy * clip * clippy again?? * fix * fix get_partitions_legacy * clippy * clippy Entire-Checkpoint: 236c651af457 * clippy * fix comments * fix merge * PR comments * clippy * docs * Use in_memory partition manager by default + timeout update * Fix table creation * Fix cayenne_catalog::tests::test_create_table_detects --------- Co-authored-by: Luke Kim <80174+lukekim@users.noreply.github.com> Co-authored-by: Viktor Yershov <viktor@spice.ai>
* feat: Add support for Spicepod version v2 - Updated spicepod.schema.json to include "v2" in the version enum. - Changed default version in SpicepodDefinition to v2. - Updated init command to create spicepod.yaml with version v2. - Added comprehensive tests for v1 and v2 versions, ensuring backward compatibility. - Created new test fixtures for v2 spicepods and updated existing v1 fixtures. - Implemented migration logic for deprecated fields from v1 to v2. - Enhanced caching configuration for v2, including new fields and defaults. - Updated documentation and comments to reflect changes in versioning and features. * feat: Remove support for v1beta1 and update examples to v2 * fix: ensure newline at end of iceberg-duckdb[file].yaml * Apply suggestion from @lukekim * feat: Add support for Spicepod version v2 - Updated spicepod.schema.json to include "v2" in the version enum. - Changed default version in SpicepodDefinition to v2. - Updated init command to create spicepod.yaml with version v2. - Added comprehensive tests for v1 and v2 versions, ensuring backward compatibility. - Created new test fixtures for v2 spicepods and updated existing v1 fixtures. - Implemented migration logic for deprecated fields from v1 to v2. - Enhanced caching configuration for v2, including new fields and defaults. - Updated documentation and comments to reflect changes in versioning and features. * feat: Remove support for v1beta1 and update examples to v2 * fix: ensure newline at end of iceberg-duckdb[file].yaml * Apply suggestion from @lukekim * fix: Update JSON schema for v2 features and fix remaining v1beta1 reference - Add read_write_create to AccessMode enum in schema - Add partition_management to Scheduler and PartitionManagement in schema - Update test/adbc/python/spicepod.yaml from v1beta1 to v2 * style: Refactor code for consistency and readability in version tests
…itial_partitions. (#9579) * make errors better * Avoid issues when executor requests allocate_initial_partitions earlier than scheduler can handle * fix lint * linting
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
See Commits and Changes for more details.
Created by
pull[bot] (v2.0.0-alpha.4)
Can you help keep this open source service alive? 💖 Please sponsor : )