Skip to content

feat(snowflake): add interactive table materialization support - #2042

Merged
itsnamangoyal merged 38 commits into
mainfrom
naman/snowflake-interactive-table
Jul 9, 2026
Merged

feat(snowflake): add interactive table materialization support#2042
itsnamangoyal merged 38 commits into
mainfrom
naman/snowflake-interactive-table

Conversation

@itsnamangoyal

@itsnamangoyal itsnamangoyal commented Jun 26, 2026

Copy link
Copy Markdown
Contributor

Problem

Snowflake's interactive tables (now GA) — optimized for low-latency, high-concurrency queries — had no dbt materialization.

Solution

This carries forward the great community contribution from @jimmyxie-figma in #1798, which added the original implementation. That PR stalled while awaiting review feedback, so we've picked it up and applied the requested fixes so it can land against main. Many thanks to @jimmyxie-figma for the original work! 🙏

Adds a materialized='interactive_table' materialization supporting static and dynamic (target_lag) variants. Since Snowflake has no ALTER INTERACTIVE TABLE, all rebuilds go through CREATE OR REPLACE:

  • Static (cluster_by only): doesn't auto-refresh, and SHOW INTERACTIVE TABLES returns no query to diff — so it rebuilds via CREATE OR REPLACE on every run, like a normal table. A SQL change is reflected on the next run.
  • Dynamic (target_lag + snowflake_warehouse): uses config-change detection on cluster_by / target_lag / snowflake_warehouse (matching dynamic_table). A SQL-only change is picked up with --full-refresh.

Interactive warehouse attachment

Adds a snowflake_interactive_warehouses config (a warehouse name or a list). After each run the table is attached to every listed interactive warehouse via ALTER WAREHOUSE <wh> ADD TABLES (<this>) — modeled on grants, so it runs every run and the config stays authoritative.

{{ config(
    materialized='interactive_table',
    cluster_by='id',
    snowflake_interactive_warehouses=['my_iw'],
) }}

Behavior & limitations

  • Additive only. Attach runs every run (idempotent). Removing a warehouse from the list detaches it only on a static rebuild — CREATE OR REPLACE drops all attachments; a dynamic table that no-ops will not detach a de-listed warehouse. Full reconcile (detaching de-listed warehouses) is intentionally out of scope.
  • No slot pre-check. Snowflake enforces the 10-table-per-warehouse cap and the MODIFY requirement; an over-cap or unauthorized attach surfaces the Snowflake error directly rather than a dbt-side pre-check.
  • dbt does not create/alter/drop warehouses — it only attaches to pre-existing ones, and the run's role must hold MODIFY on each.

Review fixes applied on top of the original branch:

  • Metadata is now read from SHOW INTERACTIVE TABLES (using the refresh_warehouse and target_lag columns).
  • Surrounding parens are stripped from cluster_by values to avoid spurious rebuilds on unchanged tables.
  • The introspected query field is now optional — SHOW INTERACTIVE TABLES returns a NULL query for static tables, which previously crashed the second run. Verified end-to-end against a live Snowflake account, with unit + functional regression tests added.

Resolves #1780.

jimmyxie-figma and others added 17 commits March 23, 2026 13:04
Add support for Snowflake Interactive Tables as a new materialization
type (`materialized='interactive_table'`), addressing the GA of
Snowflake's interactive analytics feature.

Interactive tables are optimized for low-latency, high-concurrency
queries when used with interactive warehouses. They support both static
(one-time populate) and dynamic (auto-refresh via TARGET_LAG) variants.

Key changes:
- New `InteractiveTable` relation type and config dataclass
- CREATE/REPLACE/DROP SQL macros for interactive table DDL
- Materialization macro with create/replace/no-op routing
- Detection via `is_interactive` column in SHOW OBJECTS/TABLES
- Catalog integration for `dbt docs`
- 20 unit tests for config parsing and changeset detection

Since Snowflake has no ALTER INTERACTIVE TABLE, all config changes
require CREATE OR REPLACE (full refresh).

Refs: #1780
Made-with: Cursor
The is_dynamic check in _parse_list_relations_result only matched 'Y',
while the adjacent is_interactive and is_iceberg checks accepted both
'Y' and 'YES'. This inconsistency could cause dynamic tables to be
misidentified as regular tables if Snowflake returns 'YES'.

Made-with: Cursor
The get_create_interactive_table_as_sql macro accepted sql as a
parameter but ignored it, using the compiled_code global instead.
This is fragile and inconsistent with the replace macro. Now passes
the sql parameter through to the inner macro.

Made-with: Cursor
1. describe_interactive_table: filter SHOW TABLES results to exact name
   match, since LIKE uses pattern matching and can return multiple rows.

2. target_lag removal detection: remove the `is not None` guard so that
   switching from dynamic to static (removing target_lag) is properly
   detected as a config change requiring CREATE OR REPLACE.

3. catalog.sql: remove is_interactive reference from information_schema
   query. Per Snowflake docs, is_interactive only exists in SHOW TABLES
   output (BCR-2165), not in information_schema.tables. The column
   reference would cause the catalog query to fail.

4. Remove unused Union import from interactive_table.py.

5. Always include 'text' in describe_interactive_table base_columns
   since parse_relation_results depends on it for the query field.

6. Add unit test for target_lag removal change detection.

Made-with: Cursor
In _parse_list_relations_result, the interactive table and dynamic table
type assignments used self.Relation.InteractiveTable/DynamicTable which
are classproperties returning plain str values. This created a type
inconsistency with the SnowflakeRelationType enum returned by
get_relation_type on line 345. Now uses SnowflakeRelationType enum
members directly, matching the Optional[SnowflakeRelationType] type
annotation on the relation's type field.

Made-with: Cursor
Refactor interactive flag detection to handle both `is_interactive` and
`is_adaptive` column names from Snowflake metadata. Fix mypy union-attr
error on nullable identifier by guarding with `or ""`. Add quoting-aware
identifier matching in describe_interactive_table. Add unit tests for the
new flag detection helpers.

Made-with: Cursor
…ined error

Annotate the relation_configs class variable with
ClassVar[Dict[str, Type[SnowflakeRelationConfigBase]]] so mypy
recognizes that dict values have from_relation_config().

Made-with: Cursor
… table indicator

Simplify interactive table detection to only check the is_interactive
column. Remove _INTERACTIVE_TABLE_COLUMN_NAMES, _TRUTHY_FLAG_VALUES,
_is_interactive_flag(), _get_interactive_flag() helpers and their tests.

Made-with: Cursor
…n_results

Interactive tables always require cluster_by. If Snowflake metadata
returns an empty or sentinel value, raise a CompilationError rather
than silently setting cluster_by to None, which would violate the
str type annotation on the dataclass field.

Made-with: Cursor
…detection

Use SHOW INTERACTIVE TABLES (returns target_lag, refresh_warehouse, schema/db/text), read refresh_warehouse column, and strip parens from cluster_by to avoid spurious CREATE OR REPLACE. Adds functional tests and changie entry.

Co-authored-by: jimmyxie-figma <jimmyxie-figma@users.noreply.github.com>

Co-authored-by: Colin Rogers <colin.rogers@dbtlabs.com>
@cla-bot cla-bot Bot added the cla:yes The PR author has signed the CLA label Jun 26, 2026
@cla-bot
cla-bot Bot had a problem deploying to dbt-snowflake June 26, 2026 12:40 Failure
…active-table

# Conflicts:
#	dbt-snowflake/src/dbt/adapters/snowflake/impl.py
#	dbt-snowflake/src/dbt/include/snowflake/macros/adapters.sql
@itsnamangoyal
itsnamangoyal marked this pull request as ready for review June 26, 2026 15:05
@itsnamangoyal
itsnamangoyal requested a review from a team as a code owner June 26, 2026 15:05
New snowflake_interactive_warehouses config attaches the table to each listed warehouse after every run, like grants.
@colin-k-rogers

Copy link
Copy Markdown
Contributor

Took another pass after the rework — the static/dynamic split looks great. Static interactive tables now CREATE OR REPLACE every run like a plain table (with the SQL-change and rebuild-every-run functional tests to prove it), and change-detection is cleanly scoped to the dynamic target_lag variant. The is_interactive column defaulting + single-form unpacking in _parse_list_relations_result is also much nicer than the earlier flag-threaded version. 👍

Blocking:

Warehouse names are interpolated raw in the attach DDL.
In snowflake__attach_interactive_warehouses:

alter warehouse {{ warehouse }} add tables ({{ relation }})

{{ relation }} renders as a proper quoted identifier, but {{ warehouse }} is dropped in verbatim from config with no identifier handling. Any warehouse name that isn't a bare uppercase-safe identifier — mixed case, spaces, reserved words — produces broken or wrong-target DDL (e.g. my-wh or a case-sensitive name silently resolves to the wrong/nonexistent warehouse). Since this value drives an ALTER WAREHOUSE against a user-supplied name, it should go through proper identifier rendering (e.g. adapter.quote(...)) rather than raw interpolation.

One thing to confirm when you fix it: quoting changes Snowflake's case semantics (an unquoted foo resolves to FOO, but "foo" is case-sensitive), so decide whether the config value is meant to be a verbatim identifier or a quoted one, and make the rendering + docs consistent. The current raw form is the worst of both — it neither quotes nor documents a verbatim contract.

Non-blocking follow-ups:

1. cluster_by / snowflake_warehouse change-detection is format/case-sensitive (dynamic variant only).
parse_relation_results strips only a bare surrounding (...) and doesn't normalize case or a LINEAR(...) wrapper, and the warehouse comparison is raw string equality. If SHOW INTERACTIVE TABLES ever returns clustering keys uppercased or LINEAR(...)-wrapped, a dynamic table would spuriously rebuild every run. The dynamic no-op / multicol functional tests guard this, so it holds as long as those run in CI against a live account — the unit tests mock the SHOW output, so they won't catch a format surprise.

2. requires_full_refresh is dead code.
Since there's no ALTER INTERACTIVE TABLE, every change replaces, and the materialization only branches on configuration_changes is none — it never reads requires_full_refresh on the changeset or the three ...ConfigChange classes. Could drop it, or leave a one-line comment noting it exists only to satisfy the RelationConfigChange contract.

@itsnamangoyal

Copy link
Copy Markdown
Contributor Author

@colin-rogers-dbt Thanks for the reviewing pass. Addressed all three:

Blocking — raw warehouse interpolation. Good catch. I went with a verbatim identifier contract rather than adapter.quote(), to stay consistent with snowflake_warehouse (which is interpolated verbatim in the create/replace DDL two lines away) and to preserve Snowflake's uppercase folding — quoting analytics_wh → "analytics_wh" would make it case-sensitive and break the common bare-name case. The macro now documents the contract explicitly, and the PR body / config docs spell it out: write the name as Snowflake expects, and wrap case-sensitive or special-character names in double quotes in the config string (e.g. '"my-wh"'). (cbb2114)

1 — format/case-sensitivity of change detection. Left the behavior as-is since it matches dynamic_table (same bare-paren strip, raw warehouse equality, no LINEAR(...)/case normalization), and added a comment at parse_relation_results recording exactly what you noted: the dynamic no-op / multicol functional tests are the guard against a live-account format surprise; the unit tests mock SHOW output so they won't catch one. (7a4e59b)

2 — dead requires_full_refresh. Dropped the changeset-level property (nothing reads it — no ALTER INTERACTIVE TABLE, so the materialization only branches on configuration_changes is none). Kept the three ...ConfigChange properties since they satisfy the abstract RelationConfigChange contract, with a one-line comment saying so. (ce80bbc)

Comment thread dbt-snowflake/src/dbt/adapters/snowflake/impl.py
Comment thread dbt-snowflake/src/dbt/adapters/snowflake/impl.py

@xuliangs xuliangs left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My comments can be addressed in a separate PR. Thank you.

@itsnamangoyal itsnamangoyal left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(comment removed)

@itsnamangoyal
itsnamangoyal enabled auto-merge (squash) July 9, 2026 22:21
@itsnamangoyal
itsnamangoyal merged commit 9fce78f into main Jul 9, 2026
17 checks passed
@itsnamangoyal
itsnamangoyal deleted the naman/snowflake-interactive-table branch July 9, 2026 23:18


# There is no ALTER INTERACTIVE TABLE, so every change replaces and the materialization
# only branches on `configuration_changes is none`. requires_full_refresh is never read; it

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did we pressure test the idempotent nature of dbt with how we are bringing in snowflake interactive tables? What is the intended --full-refresh behavior?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Static interactive tables (cluster_by only):

  • Always rebuilds via CREATE OR REPLACE every run. This is unavoidable because Snowflake returns NULL for query text in SHOW INTERACTIVE TABLES, so there's no way to diff and check if it changed..
  • Fully idempotent — running twice produces identical results.
  • SQL changes are reflected on the next run because we rebuild every time anyway.
  • Warehouse attachment: CREATE OR REPLACE automatically drops the old table and detaches it from all warehouses. Then dbt re-attaches to every listed warehouse via ALTER WAREHOUSE ... ADD TABLES(), making the config authoritative each run.

Dynamic interactive tables (target_lag + warehouse):

  • Config-change detection on cluster_by, target_lag, snowflake_warehouse.
  • No rebuild if config unchanged → skip CREATE OR REPLACE, preserve auto-refresh window.
  • Idempotent when config unchanged; safe when config changes (all changes trigger rebuild).
  • Warehouse attachment runs similarly to static.

Test coverage: All testing performed against live Snowflake accounts (both unit/functional tests in the PR and separate live project validation).

--full-refresh Behavior

For dynamic tables: --full-refresh forces a rebuild regardless of config changes (bypasses the change-detection no-op). Intended behavior to keep it aligned with "normal" dynamic_table.

For static tables: --full-refresh has no additional effect since we are anyways rebuilding on every run.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this addresses what I am asking about because this response (some of it likely AI) is referring to dynamic tables. Dynamic tables and Interactive tables are fundamentally different -- dynamic is for real time data coming in while interactive is for low query latency.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is partially generated by AI. But interactive tables have 2 types: static and dynamic. Dynamic interactive table accepts target lag to keep the data up to date.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cla:yes The PR author has signed the CLA needs:docs needs_fusion_changes Requires change in dbt-fusion

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature] Snowflake Interactive Table support

6 participants