Purpose: stress the parts
example-walkthrough.mdexplicitly listed as not covered — two levels ofobject_list, the^.one-level-out scope,map/number/json, and above all theApplyediting path (secret retention,$idalignment, secrets staying aligned after an entry is deleted).This round found 8 gaps, one of them a design bug that would have caused data loss (see 6.1).
Data pipeline configuration. A pipeline has one source and several sinks, and each sink has several field mappings — naturally two levels of nesting, and both levels may hold secrets.
Requirements worth noting:
- Which fields a sink offers depends on its
type(kafka / http / s3) - A mapping's
partition_keyonly means anything when the owning sink is kafka → it must reference one level out - A mapping with
transform: maskneeds a salt (a secret) → the second level has secrets too - While the pipeline is running, the source type cannot change →
readonly_when - Turning alerts off clears the webhook secret along with everything else
{
"schema_version": 2,
"groups": [
{ "key": "source", "label": "Source", "mode": "always", "fields": [
{ "name": "is_running", "type": "boolean", "label": "Running",
"readonly": true, "default": false },
{ "name": "source_type", "type": "enum", "label": "Source type", "required": true,
"options": [ {"value":"postgres","label":"PostgreSQL"},
{"value":"mysql","label":"MySQL"},
{"value":"mongodb","label":"MongoDB"} ],
"readonly_when": { "field": "is_running", "is_true": true } },
{ "name": "source_dsn", "type": "string", "format": "uri", "required": true },
{ "name": "source_password", "type": "secret", "required": true },
{ "name": "sampling_rate", "type": "number", "label": "Sampling rate",
"min": 0, "max": 1, "default": 1 } ] },
{ "key": "advanced", "label": "Advanced", "mode": "collapsible",
"default_collapsed": true, "fields": [
{ "name": "labels", "type": "map", "label": "Labels",
"key_pattern": "^[a-z][a-z0-9_]{0,30}$",
"items": { "type": "string", "max_length": 64 } },
{ "name": "extra_config", "type": "json", "label": "Custom settings" } ] },
{ "key": "sinks_group", "label": "Sinks", "mode": "always", "fields": [
{ "name": "sinks", "type": "object_list", "label": "Sink",
"min_items": 1, "max_items": 8, "unique": ["name"],
"item_label": "{name}", "layout": "card",
"fields": [
{ "name": "name", "type": "string", "required": true,
"pattern": "^[a-z][a-z0-9_]*$" },
{ "name": "type", "type": "enum", "required": true,
"options": [ {"value":"kafka","label":"Kafka"},
{"value":"http","label":"HTTP"},
{"value":"s3","label":"S3"} ] },
{ "name": "brokers", "type": "list", "min_items": 1,
"items": { "type": "string", "format": "hostname" },
"visible_when": { "field": "type", "equals": "kafka" } },
{ "name": "topic", "type": "string",
"visible_when": { "field": "type", "equals": "kafka" },
"required_when": { "field": "type", "equals": "kafka" } },
{ "name": "endpoint", "type": "string", "format": "uri",
"visible_when": { "field": "type", "equals": "http" },
"required_when": { "field": "type", "equals": "http" } },
{ "name": "headers", "type": "map",
"key_pattern": "^[A-Za-z0-9-]+$",
"items": { "type": "string" },
"visible_when": { "field": "type", "equals": "http" } },
{ "name": "auth_token", "type": "secret",
"visible_when": { "field": "type", "equals": "http" } },
{ "name": "bucket", "type": "string",
"visible_when": { "field": "type", "equals": "s3" },
"required_when": { "field": "type", "equals": "s3" } },
{ "name": "access_key", "type": "secret",
"visible_when": { "field": "type", "equals": "s3" } },
{ "name": "mappings", "type": "object_list", "label": "Field mappings",
"min_items": 1, "unique": ["target_field"],
"item_label": "{source_field} -> {target_field}", "layout": "table",
"fields": [
{ "name": "source_field", "type": "string", "required": true },
{ "name": "target_field", "type": "string", "required": true },
{ "name": "transform", "type": "enum", "default": "none",
"options": [ {"value":"none","label":"None"},
{"value":"upper","label":"Uppercase"},
{"value":"mask","label":"Mask"} ] },
{ "name": "secret_salt", "type": "secret",
"visible_when": { "field": "transform", "equals": "mask" },
"required_when": { "field": "transform", "equals": "mask" } },
{ "name": "partition_key", "type": "boolean", "default": false,
"visible_when": { "field": "^.type", "equals": "kafka" } }
] }
] } ] },
{ "key": "notify", "label": "Alerts", "mode": "toggleable",
"toggle": { "field": "enable_alerts", "label": "Enable alerts", "default": false },
"fields": [
{ "name": "alert_emails", "type": "list", "min_items": 1,
"items": { "type": "string", "format": "email" } },
{ "name": "alert_webhook", "type": "string", "format": "uri" },
{ "name": "webhook_secret", "type": "secret" } ] }
]
}All three scope forms appear at once inside mappings:
| Condition | Reference | Resolves to |
|---|---|---|
secret_salt.visible_when |
transform |
the transform of the same mapping |
partition_key.visible_when |
^.type |
the type of the owning sink (one level out) |
| (registration scenario) | $.need_accommodation |
the root level |
{
"values": {
"is_running": true,
"source_type": "postgres",
"source_dsn": "postgres://db.internal:5432/orders",
"sampling_rate": 1,
"labels": { "env": "prod", "team": "data" },
"sinks": [
{ "$id": "s_k1", "name": "kafka_main", "type": "kafka",
"brokers": ["a.internal", "b.internal"], "topic": "orders",
"mappings": [
{ "$id": "m_1", "source_field": "id", "target_field": "order_id",
"transform": "none", "partition_key": true },
{ "$id": "m_2", "source_field": "email", "target_field": "buyer_email",
"transform": "mask", "partition_key": false },
{ "$id": "m_3", "source_field": "phone", "target_field": "buyer_phone",
"transform": "mask", "partition_key": false }
] },
{ "$id": "s_h1", "name": "http_audit", "type": "http",
"endpoint": "https://audit.internal/ingest",
"headers": { "X-Source": "plasma" },
"mappings": [
{ "$id": "m_9", "source_field": "id", "target_field": "id",
"transform": "none" }
] }
],
"enable_alerts": true,
"alert_emails": ["ops@example.com"],
"alert_webhook": "https://hooks.example.com/x"
},
"secrets": {
"source_password": "pg-secret",
"sinks.s_k1.mappings.m_2.secret_salt": "salt-for-email",
"sinks.s_k1.mappings.m_3.secret_salt": "salt-for-phone",
"sinks.s_h1.auth_token": "bearer-xyz",
"webhook_secret": "hook-abc"
}
}Note that m_9 (a mapping of the http sink) has no partition_key — its visible_when
references ^.type, the owning sink is http, so the field is invisible and its value does not
exist. That is precisely what cross-level scope should do.
Six things done in the UI:
- Sampling rate 1 → 0.5
- The source password is left alone
kafka_main: topic changed toorders_v2; theemailmapping deleted; acreated_atmapping added- The entire
http_auditsink deleted - A new s3 sink added, with an access key
- Alerts turned off
The patch the client sends:
{
"values": {
"sampling_rate": 0.5,
"sinks": [
{ "$id": "s_k1", "topic": "orders_v2",
"mappings": [
{ "$id": "m_2", "$deleted": true },
{ "$id": "m_4", "source_field": "created_at", "target_field": "ts",
"transform": "none", "partition_key": false }
] },
{ "$id": "s_h1", "$deleted": true },
{ "$id": "s_s1", "name": "s3_archive", "type": "s3", "bucket": "archive-prod",
"mappings": [
{ "$id": "m_5", "source_field": "id", "target_field": "id",
"transform": "none" }
] }
],
"enable_alerts": false
},
"secrets": {
"source_password": "",
"sinks.s_s1.access_key": "AKIA..."
}
}$deleted came out of this round of stress testing; see 6.1 for why.
{
"values": {
"is_running": true,
"source_type": "postgres",
"source_dsn": "postgres://db.internal:5432/orders",
"sampling_rate": 0.5,
"labels": { "env": "prod", "team": "data" },
"sinks": [
{ "$id": "s_k1", "name": "kafka_main", "type": "kafka",
"brokers": ["a.internal", "b.internal"], "topic": "orders_v2",
"mappings": [
{ "$id": "m_1", "source_field": "id", "target_field": "order_id",
"transform": "none", "partition_key": true },
{ "$id": "m_3", "source_field": "phone", "target_field": "buyer_phone",
"transform": "mask", "partition_key": false },
{ "$id": "m_4", "source_field": "created_at", "target_field": "ts",
"transform": "none", "partition_key": false }
] },
{ "$id": "s_s1", "name": "s3_archive", "type": "s3", "bucket": "archive-prod",
"mappings": [
{ "$id": "m_5", "source_field": "id", "target_field": "id",
"transform": "none" }
] }
],
"enable_alerts": false
},
"secrets": {
"source_password": "pg-secret",
"sinks.s_k1.mappings.m_3.secret_salt": "salt-for-phone",
"sinks.s_s1.access_key": "AKIA..."
}
}Item by item:
| # | Expected | Result | Because |
|---|---|---|---|
| 1 | sampling_rate overwritten |
0.5 | A new value is an overwrite |
| 2 | Source password retained | pg-secret still there |
A secret carrying "" is kept |
| 3 | m_1 and m_3 absent from the patch → kept |
Both there | object_list aligns by $id; absent means kept |
| 4 | m_3's salt did not shift |
…m_3.secret_salt = salt-for-phone |
Secret keys use $id, not the index |
| 5 | m_2 and its salt gone |
Both gone | $deleted → the entry and all its secrets are removed recursively |
| 6 | The whole of s_h1 and its auth_token gone |
Both gone | Same, recursing into the second level |
| 7 | The new sink's access_key written | Present | The new entry carries a client-generated $id |
| 8 | Three alert values plus webhook_secret gone |
All gone | The toggle is off → the whole group is removed, secrets included |
| 9 | is_running / source_type unchanged |
Unchanged | Absent from the patch → kept |
Row 4 is the one this entire design most needs verified. Once m_2 is deleted, m_3's
array position moves from [2] to [1] — if secret keys were
sinks[0].mappings[2].secret_salt, m_3's salt would be orphaned and m_1 would pick up a
value that is not its own. With $id as the key, that is structurally impossible.
The original design (design.md §12) specified that an object_list aligns by $id, and that
a $id present in current but absent from the patch is deleted.
The problem: as soon as the client sends the mappings key holding only the entries it
changed — the most natural dirty-tracking a client framework offers — every other mapping is
silently deleted, along with its secrets. The user renames one field and the whole mapping
table disappears.
Comparing the consequences of each mistake:
| Semantics | When the client gets it wrong | Severity |
|---|---|---|
| Absence means delete | Unsent entries are silently deleted, data lost | Fatal, and irreversible |
| Absence means keep, deletion explicit | The delete marker was forgotten → something that should have gone did not | The user sees it immediately and clicks again |
Fix: adopt the same "absent means kept" rule as scalars, and require deletion to be explicit:
{"$id": "…", "$deleted": true}. The client has to keep a tombstone list, which is a real cost —
but the cost lands on "a few more lines of client state management" rather than on "the user's
data is gone".
An error at sinks[0].mappings[1].secret_salt involves two $ids (s_k1 and m_3), and a
single item_id field cannot hold both.
Fix: replace it with item_path, a $id path (sinks.s_k1.mappings.m_3). Its shape is
identical to the prefix of a secret key, so the client can match it directly against
secrets_set.
Secret keys are paths joined with .. If a $id itself contained a ., sinks.a.b.token could
not be resolved back to either "sink a.b's token" or "sink a's b.token" — path parsing
becomes ambiguous.
Fix: restrict $id to [A-Za-z0-9_-]{1,64}. uuids and nanoids satisfy this naturally.
Compile cannot check it (it is a value, not schema), so Validate enforces it.
items describes the value. But HTTP header names, environment variable names, and label keys
all have format requirements, and the design had nowhere to express that.
Fix: add key_pattern (a regexp) and max_items to the map type. Errors carry
FieldError.Key to say which key is invalid.
readonly_when means "reject a change", and that needs a baseline. Validate(doc) has one
document and no "before" to compare against — it simply cannot tell.
Fix: state explicitly that readonly / readonly_when are checked only in Apply, never
in Validate. The two APIs have different validation scope, and that must be in the API
documentation — otherwise a caller will assume that running Validate blocked tampering with
read-only fields, and that is a security assumption.
"Toggle off → remove the group" stops being a matter of deleting a few keys once nesting exists:
a closed group may contain an object_list, whose entries may contain further object_lists,
and any level may hold secrets.
Fix: state that removal is recursive, and that it must clear the secrets of every
$id path under that subtree. Implemented as a batch delete by secret-key prefix — exactly what
the flat secrets container buys. Were secrets a nested mirror of the value tree, the tree walk
would have to be written out again here.
If the user changes kafka_main's type from kafka to http, every mapping's partition_key
(visible_when: ^.type == kafka) becomes invisible at once and its value is removed; brokers
and topic go too.
This is the correct behaviour (invisible means non-existent), but its scale is not what the user expects — one dropdown wipes out a whole level of data.
Fix: the design does not change, but this is listed as a case where the caller must provide
a UI warning. Compiled offers ImpactOf(current, patch) []string, returning which paths this
change would clear, so the caller can prompt for confirmation before submitting. Better than
finding out afterwards that the data is gone.
A type: "json" field holds arbitrary structure. If "unknown keys are always rejected" recursed
into it, extra_config could never pass validation — and the whole point of the type would be
gone.
Fix: state that the unknown-key check stops at the boundary of a json value. Inside
json, the only check is "is a valid JSON value".
- The behaviour of
Extend()andFromSchema()(conflict handling when several providers merge schemas) - The actual lookup order for
label_i18n - Three or more levels of nesting and the
MaxDepthboundary ^^.(two levels out) — this scenario only uses^.- Concurrency: two editors calling
Applyon the same document (optimistic locking belongs to the caller, but$idmakes a merge possible, which is worth evaluating on its own)