Skip to content

Commit 80e7659

Browse files
committed
Add documentation
1 parent 9475cde commit 80e7659

File tree

2 files changed

+114
-1
lines changed

2 files changed

+114
-1
lines changed

docs/guides/configuration.md

Lines changed: 113 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -381,6 +381,119 @@ Example showing default values:
381381
)
382382
```
383383

384+
385+
### Always comparing against production
386+
387+
By default, SQLMesh compares the current state of project files to the target `<env>` environment when `sqlmesh plan <env>` is run. However, a common expectation is that local changes should always be compared to the production environment.
388+
389+
The `always_compare_against_prod` boolean plan option can alter this behavior. When enabled, SQLMesh will always attempt to compare against the production environment; If that does not exist, SQLMesh will fall back to comparing against the target environment.
390+
391+
**NOTE:**: Upon succesfull plan application, changes are still promoted to the target `<env>` environment.
392+
393+
=== "YAML"
394+
395+
```yaml linenums="1"
396+
plan:
397+
always_compare_against_prod: True
398+
```
399+
400+
=== "Python"
401+
402+
The Python `auto_categorize_changes` argument takes `CategorizerConfig` object. That object's arguments take an `AutoCategorizationMode` enumeration with values of `AutoCategorizationMode.FULL`, `AutoCategorizationMode.SEMI`, or `AutoCategorizationMode.OFF`.
403+
404+
```python linenums="1"
405+
from sqlmesh.core.config import (
406+
Config,
407+
ModelDefaultsConfig,
408+
AutoCategorizationMode,
409+
CategorizerConfig,
410+
PlanConfig,
411+
)
412+
413+
config = Config(
414+
model_defaults=ModelDefaultsConfig(dialect=<dialect>),
415+
plan=PlanConfig(
416+
always_compare_against_prod=True,
417+
),
418+
)
419+
```
420+
421+
#### Change Categorization Example
422+
423+
Consider this scenario with `always_compare_against_prod` enabled:
424+
425+
1. Initial state in `prod`:
426+
```sql
427+
MODEL (name sqlmesh_example.test_model, kind FULL);
428+
SELECT 1 AS col
429+
```
430+
431+
1. First (breaking) change in `dev`:
432+
```sql
433+
MODEL (name test.a, kind FULL);
434+
SELECT 2 AS col
435+
```
436+
437+
??? "Output plan example #1"
438+
439+
```bash
440+
New environment `dev` will be created from `prod`
441+
442+
Differences from the `prod` environment:
443+
444+
Models:
445+
└── Directly Modified:
446+
└── sqlmesh_example__dev.test_model
447+
448+
---
449+
+++
450+
451+
452+
kind FULL
453+
)
454+
SELECT
455+
- 1 AS col
456+
+ 2 AS col
457+
```
458+
459+
3. Second (metadata) change in `dev`:
460+
```sql
461+
MODEL (name test.a, kind FULL, owner 'John Doe');
462+
SELECT 5 AS col
463+
```
464+
465+
??? "Output plan example #2"
466+
467+
```bash
468+
Differences from the `prod` environment:
469+
470+
Models:
471+
└── Directly Modified:
472+
└── sqlmesh_example__dev.test_model
473+
474+
---
475+
476+
+++
477+
478+
@@ -1,8 +1,9 @@
479+
480+
MODEL (
481+
name sqlmesh_example.test_model,
482+
+ owner "John Doe",
483+
kind FULL
484+
)
485+
SELECT
486+
- 1 AS col
487+
+ 2 AS col
488+
489+
Directly Modified: sqlmesh_example__dev.test_model (Breaking)
490+
Models needing backfill:
491+
└── sqlmesh_example__dev.test_model: [full refresh]
492+
```
493+
494+
Even though the second change should have been a metadata change (thus not requiring a backfill), it will still be classified as a breaking change because the comparison is against production instead of the previous development state. This is intentional and may cause additional backfills as more changes are accumulated.
495+
496+
384497
### Gateways
385498
386499
The `gateways` configuration defines how SQLMesh should connect to the data warehouse, state backend, and scheduler. These options are in the [gateway](../reference/configuration.md#gateway) section of the configuration reference page.

sqlmesh/core/context_diff.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -150,7 +150,7 @@ def create(
150150
s.snapshot_id for s in initial_env.promoted_snapshots
151151
}
152152

153-
# Find the proper environment to diff against, this might be different than the "initial" (i.e user provided) environment
153+
# Find the proper environment to diff against, this might be different than the initial (i.e user provided) environment
154154
# e.g it will default to prod if the plan option `always_compare_against_prod` is set.
155155
environment = _get_diff_environment(environment, state_reader, always_compare_against_prod)
156156
env = (

0 commit comments

Comments
 (0)