surfaces/adapter: in-place schema changes (ALTER MATERIALIZED VIEW)

[benesch]

Overview

Allow changing the definition of a materialized view in-place using ALTER MATERIALIZED VIEW.

Details

Pairing this with MaterializeInc/database-issues#4576 allows users to handle (compatible) schema changes end-to-end without disrupting downstream dependencies.

Discussion from original issue

From @morsapaes:

Recording a Slack discussion where we concluded that CREATE OR REPLACE MATERIALIZED VIEW might be the more appropriate command here. We already support the OR REPLACE syntax for materialized views:

If a materialized view exists with the same name, replace it with the view defined in this statement. You cannot replace views that other views or sinks depend on, nor can you replace a non-view object with a view.

, so we could extend the semantics to allow replacing the definition of the view also when it has downstream dependencies.

cc @benesch

From @benesch:

Here's how the semantics would look. The ALTER MATERIALIZED VIEW command would occur at some timestamp t_alter. At timestamps t < t_alter, the materialized view would contain the results from the old definition. At timestamps t >= t_alter, the materialized view would contain the results from the new definition. Subscribes and sinks that observe time t will see retractions of all data that was present in the old definition but is not present in the new definition, and insertions of all data that is present in the new definition but was not present in the old definition.

From @sthm:

Subscribes and sinks that observe time t will see retractions of all data that was present in the old definition but is not present in the new definition, and insertions of all data that is present in the new definition but was not present in the old definition.

Seems like I'm some missing context here. Doesn't that imply that if the schema is changed at t_alter, all data of the old definition is retracted and all data of the new definition (with the added column) is inserted?

From @benesch:

Ah, the comment above was specifically about changes to a view definition that do not change its schema. Schema changes require slightly different considerations. (Loosely: we introduce a new version of the matview in which it is as if the new column existed since the beginning of time.)

See also

• MaterializeInc/database-issues#4576
• MaterializeInc/database-issues#7819

cc @benesch

Decision log

✋