Background
The RowBinary codec has a dynamic reader: compileRowBinaryWithNamesAndTypes reads the RowBinaryWithNamesAndTypes header, folds each column's type string into a Reader (astToReader), and decodes the rest of the stream — a generic way to consume any response without knowing the schema ahead of time.
There is no equivalent dynamic writer today, and that's intentional. This issue exists mostly to explain why, and to help anyone who lands here decide whether they actually need one.
You probably don't need this
The read and write sides aren't symmetric in practice:
- When reading
RowBinaryWithNamesAndTypes, the server decides the schema and sends it to you in the header — the projection, the types after functions/aggregations, the column order. The client genuinely can't know them ahead of time, so a runtime-driven decoder is the natural fit.
- When writing, you're usually inserting your data into your table, whose schema you already know at build time. In that case the specialized, hand-written writer (or a generated one) is the better choice — it's faster and simpler, and the codec is designed to steer you there.
So if you're encoding application data into a known table, you most likely want the specialized writer path described in writer.md, not a dynamic one.
When a dynamic writer might genuinely help
A runtime-schema-driven encoder mainly makes sense when the code is decoupled from the schema it's encoding — i.e. generic infrastructure rather than application code. A few scenarios:
- Generic ingestion tools / connectors — a Kafka sink, ETL/CDC pipeline, or a "load this file into that table" utility that inserts into tables it doesn't know at build time. It would learn the schema at runtime (e.g. from
DESCRIBE TABLE / system.columns) and encode rows accordingly. This is the closest mirror to the dynamic reader; the only difference is the schema comes from a query rather than a response header.
RowBinaryWithNamesAndTypes as an insert format — ClickHouse can accept that format as input and match columns to the target table by name. Emitting the names+types header + body for an unknown or evolving table benefits from building the encoders at runtime.- Read-transform-write / copy — table-to-table copy, fan-out, backfills, or an anonymizing proxy that decodes one stream and re-encodes it elsewhere, ideally driven by the same parsed type strings on both sides.
- Multi-tenant or evolving schemas — one binary serving many slightly-different (or changing) tables, discovering and caching encoders per schema at runtime instead of being recompiled per schema.
What it could look like
The write path would mirror the read path: an astToWriter (the inverse of astToReader) that folds a parsed @clickhouse/datatype-parser AST into a Writer<unknown> using the existing generic combinators, plus a helper that emits the names+types header and returns a writeRows driver. As with the dynamic reader, it would be generic and unoptimized — the specialized path stays the recommendation whenever the types are fixed.
Status
Not currently planned. If one of the scenarios above describes your use case, a comment with the details would help gauge interest — especially the generic-ingestion case, which is the strongest mirror of the reader.
Background
The RowBinary codec has a dynamic reader:
compileRowBinaryWithNamesAndTypesreads theRowBinaryWithNamesAndTypesheader, folds each column's type string into aReader(astToReader), and decodes the rest of the stream — a generic way to consume any response without knowing the schema ahead of time.There is no equivalent dynamic writer today, and that's intentional. This issue exists mostly to explain why, and to help anyone who lands here decide whether they actually need one.
You probably don't need this
The read and write sides aren't symmetric in practice:
RowBinaryWithNamesAndTypes, the server decides the schema and sends it to you in the header — the projection, the types after functions/aggregations, the column order. The client genuinely can't know them ahead of time, so a runtime-driven decoder is the natural fit.So if you're encoding application data into a known table, you most likely want the specialized writer path described in
writer.md, not a dynamic one.When a dynamic writer might genuinely help
A runtime-schema-driven encoder mainly makes sense when the code is decoupled from the schema it's encoding — i.e. generic infrastructure rather than application code. A few scenarios:
DESCRIBE TABLE/system.columns) and encode rows accordingly. This is the closest mirror to the dynamic reader; the only difference is the schema comes from a query rather than a response header.RowBinaryWithNamesAndTypesas an insert format — ClickHouse can accept that format as input and match columns to the target table by name. Emitting the names+types header + body for an unknown or evolving table benefits from building the encoders at runtime.What it could look like
The write path would mirror the read path: an
astToWriter(the inverse ofastToReader) that folds a parsed@clickhouse/datatype-parserAST into aWriter<unknown>using the existing generic combinators, plus a helper that emits the names+types header and returns awriteRowsdriver. As with the dynamic reader, it would be generic and unoptimized — the specialized path stays the recommendation whenever the types are fixed.Status
Not currently planned. If one of the scenarios above describes your use case, a comment with the details would help gauge interest — especially the generic-ingestion case, which is the strongest mirror of the reader.