Skip to content

Support table-level auto_flush_interval #8340

Description

@killme2008

What type of enhancement is this?

Configuration / User experience

What does the enhancement do?

Make auto_flush_interval configurable at the table (region) level, in addition to the existing global setting.

Today auto_flush_interval is a global Mito engine config (src/mito2/src/config.rs, default 30min). It controls how long a region can stay un-flushed before the periodic flush job flushes it (see flush_periodically and flush_regions_on_engine_full in src/mito2/src/worker/handle_flush.rs, both read self.config.auto_flush_interval).

A single global value doesn't fit every workload. In practice you often want a different flush cadence for a few specific tables while leaving the rest on the default:

  • Low-traffic tables that should flush sooner so data lands in object storage / becomes queryable with bounded latency.
  • High-traffic or throwaway tables where you want a longer interval to reduce small-file pressure and compaction load.

Proposed behavior:

  • Add an auto_flush_interval table option (e.g. WITH ('auto_flush_interval' = '5m')), stored in the region options.
  • The periodic flush logic uses the per-region value when set, and falls back to the global engine config when unset.
  • Setting it via ALTER TABLE should take effect without restart, consistent with how other region options behave.

This mirrors options like ttl, which already support both a global default and a per-table override.

Implementation challenges

  • Add auto_flush_interval: Option<Duration> to RegionOptions (src/mito2/src/region/options.rs), with parsing/validation consistent with existing duration options.
  • In handle_flush.rs, resolve the effective interval per region: region_options.auto_flush_interval.unwrap_or(self.config.auto_flush_interval) instead of always using the global config.
  • Wire the option through SQL table options -> region create/alter requests so it can be set at CREATE TABLE and changed via ALTER TABLE.
  • Make sure the option is round-tripped in SHOW CREATE TABLE and covered by sqlness tests.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions