Skip to content
This repository was archived by the owner on Nov 7, 2025. It is now read-only.

Conversation

@mieciu
Copy link
Member

@mieciu mieciu commented Mar 4, 2025

We're adding a partitioningStrategy to index configuration:

processors:
  - name: my-query-processor
    type: quesma-v1-processor-query
    config:
      indexes:
        '*':
          partitioningStrategy: "daily"
          target: [ my-clickhouse-data-source ]
        try_it:
          partitioningStrategy: "monthly"
          target: [ my-clickhouse-data-source ]

Having partitioningStrategy essentially adds PARTITION BY function( to CREATE TABLE statement executed by Quesma.

Key points:

  1. partitioningStrategy is applied only on table creation handled by Quesma. If the table already exists, Quesma won't modify it.
  2. partitioningStrategy can be set to hourly, daily, monthly and yearly.
  3. We're not adding common table partitioning support - so if index has useCommonTable: true specified together with any partitioningStrategy - Quesma is going to fail due to invalid configuration
  4. Leaving partitioningStrategy not specified means table is not partitioned at all (== ClickHouse default)

gitignore

cont'd

# Conflicts:
#	.gitignore
#	platform/ingest/processor.go
@mieciu mieciu force-pushed the table-partitioning-support branch from c07dddf to 5e7c203 Compare March 5, 2025 13:50
@mieciu mieciu marked this pull request as ready for review March 10, 2025 10:20
@mieciu mieciu requested a review from a team as a code owner March 10, 2025 10:20
@mieciu mieciu changed the title [WIP] Adding experimental (and limited) support for table partitioning Adding experimental (and limited) support for table partitioning Mar 10, 2025
@mieciu
Copy link
Member Author

mieciu commented Mar 10, 2025

/run-integration-tests

@cloudflare-workers-and-pages
Copy link

cloudflare-workers-and-pages bot commented Mar 10, 2025

Deploying quesma with  Cloudflare Pages  Cloudflare Pages

Latest commit: 5741569
Status: ✅  Deploy successful!
Preview URL: https://877e8ff0.quesma.pages.dev
Branch Preview URL: https://table-partitioning-support.quesma.pages.dev

View logs

@mieciu mieciu enabled auto-merge March 10, 2025 13:50
@mieciu mieciu added this pull request to the merge queue Mar 10, 2025
Merged via the queue into main with commit 0b1a835 Mar 10, 2025
7 checks passed
@mieciu mieciu deleted the table-partitioning-support branch March 10, 2025 13:57
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants