[pull] trunk from spiceai:trunk#701
Merged
Merged
Conversation
… mode (#9838) * fix: Cayenne Catalog DDL requires a connected executor in distributed mode * chore: linting
* make cluster testing easier Entire-Checkpoint: 70992628f7b3 * some temporary improvements Entire-Checkpoint: ea1b793e1ea6 * fix: remove unused code * tests working * uncommnent * linting * do put to executors based on partitioning * write through cayenne catalog to executors * fix merge * used federated partition manager * write partition metadata for new data * PR comments * get partititon expressions from cayenne table * Jeadie/26 03 14/testing (#9752) * feat: Support PARTITION BY in Cayenne Catalog table creation * chore: clippy + fmt * review: Address comments + clippy * chore: fmt * Apply suggestions from code review Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * chore: clippy * review: Address comments * chore: fmt * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * fix lint * fix hash eq * fix merge; update spidapter * fix build --------- Co-authored-by: peasee <98815791+peasee@users.noreply.github.com> Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Co-authored-by: Luke Kim <80174+lukekim@users.noreply.github.com> * fixes needed * fixes * correct query exprs * fixes for partition types * fixes for bucketing * fixes for physical expression * fix table name issue with UNION ALL for scatter-gather * clippy * reduce object store calls * tests * clippy & comments * fixes, including for string literal values * fixes' * fix clippy * DML * formatting * compile * innter_catalog * clippy * lint again * clippy * lint * formatting * partitioning tests * more test snapshots * formatting * rename some things * Fix clippy and formatting lint issues * Address review feedback: correctness and safety fixes - Fix planner error messages to reference correct node types (DistributedCayenneDeleteNode/DistributedCayenneUpdateNode) - Make Hash/PartialEq consistent for DistributedCayenneDeleteNode and DistributedCayenneUpdateNode (PartialEq now includes all fields used in Hash) - Propagate extract_filter_sql errors in DELETE forwarding instead of silently dropping them via .ok().flatten() (could forward DELETE without WHERE clause) - Error when executor_registry is None in DistributedCayenneDeleteExec and DistributedCayenneUpdateExec instead of returning a misleading success batch - Replace ad-hoc adapt_batch_schema with arrow_tools::record_batch::try_cast_to which handles field name matching, missing nullable columns, and returns proper errors instead of panicking on column count mismatch --------- Co-authored-by: Luke Kim <80174+lukekim@users.noreply.github.com> Co-authored-by: peasee <98815791+peasee@users.noreply.github.com> Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Co-authored-by: Phillip LeBlanc <phillip@leblanc.tech>
…IRSA example (#9833) * feat: Add support for ServiceAccount annotations and AWS IRSA example * Address PR review: revert accidental cluster/mod.rs change, fix README defaults and Pod Identity docs
…#9842) LocalFileSystem returns NotImplemented for PutMode::Update (conditional writes), which broke scheduler registry heartbeats and async SQL job store state transitions when using file:// as the scheduler state location. Adds LocalConditionalPut, an ObjectStore wrapper that implements PutMode::Update for local filesystems by checking ETags under an async mutex then writing with PutMode::Overwrite if they match. This provides single-process OCC for file:// state locations (intended for single-node and development use). Wires LocalConditionalPut into build_object_store_internal() so file:// state locations use it instead of plain LocalFileSystem.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
See Commits and Changes for more details.
Created by
pull[bot] (v2.0.0-alpha.4)
Can you help keep this open source service alive? 💖 Please sponsor : )