Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Enhancement/8561507350/precompute output schema from processing #2233

Open
wants to merge 29 commits into
base: master
Choose a base branch
from
Open

Enhancement/8561507350/precompute output schema from processing #2233

wants to merge 29 commits into from

Conversation

alexowens90
Copy link
Collaborator

Reference Issues/PRs

Closes 8561507350

What does this implement or fix?

Adds a method to clauses to produce the correct StreamDescriptor and NormalizationMetadata given the same types as input, or throw an exception if the clause is not compatible with the input columns/types.

Checklist

Checklist for code changes...
  • Have you updated the relevant docstrings, documentation and copyright notice?
  • Is this contribution tested against all ArcticDB's features?
  • Do all exceptions introduced raise appropriate error messages?
  • Are API changes highlighted in the PR description?
  • Is the PR labelled as enhancement or bug so it appears in autogenerated release notes?

alexowens90 added 22 commits March 6, 2025 15:04
@alexowens90
Implement modify_schema for passthrough, row range, and date range cl…
45d9a43 
…auses
@alexowens90
Implement and test FilterClause::modify_schema based only on input_co…
c4ff8e9 
…lumns, not ExpressionContext
@alexowens90
Implement and test PartitionClause::modify_schema
9772e0e
@alexowens90
Implemented AggregationClause::modify_schema
191a101
@alexowens90
Aggregation column names tested
912c059
@alexowens90
Implement aggregation output types properly, test for sum
4b1cf76
@alexowens90
Tests for min and max aggregators
5970065
@alexowens90
Tests for mean aggregator
738c9b3
@alexowens90
Refactored tests
9938d77
@alexowens90
Implement modify_schema for ResampleClause
c68eda1
@alexowens90
ResampleClause::modify_schema fully tested
3923582
@alexowens90
Tests for RemoveCOlumnPartitioning, Split, Sort, and Merge clauses, a…
0ff5086 
…nd add check to Merge that data is a timeseries
@alexowens90
Throw if modify_schema is called on ColumnStatsGenerationClause
c103b92
@alexowens90
Same level of checks for FilterClause added to ProjectionClause
a0b0266
@alexowens90
Implemented type checking for ProjectClause (untested)
c419245
@alexowens90
Implemented type checking for FilterClause (untested)
0d3f2d5
@alexowens90
Fix filter and project tests
ac9e385
@alexowens90
Clone StreamDescriptors
2073432
@alexowens90
AST validity tests passing for projections
cb344ae
@alexowens90
AST validity tests passing for filters
e401c17
@alexowens90
Remove unneeded computed_data
fd5d85c
@alexowens90
Keep cache of column types
15ea08e
@alexowens90 alexowens90 self-assigned this Mar 12, 2025
@alexowens90 alexowens90 added the patch Small change, should increase patch version label Mar 12, 2025
@alexowens90 alexowens90 marked this pull request as draft March 12, 2025 13:22
@alexowens90 alexowens90 marked this pull request as ready for review March 12, 2025 16:45
@alexowens90 alexowens90 changed the title WIP: Enhancement/8561507350/precompute output schema from processing Enhancement/8561507350/precompute output schema from processing Mar 12, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@willdealtry willdealtry Awaiting requested review from willdealtry willdealtry is a code owner

@poodlewars poodlewars Awaiting requested review from poodlewars poodlewars is a code owner

At least 2 approving reviews are required to merge this pull request.

Assignees

@alexowens90 alexowens90

Labels
patch Small change, should increase patch version
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@alexowens90