perf: optimize column metadata parsing and readTypeInfo()#780
perf: optimize column metadata parsing and readTypeInfo()#780dkropachev merged 2 commits intoscylladb:masterfrom
Conversation
There was a problem hiding this comment.
Pull request overview
This PR optimizes result/prepare metadata parsing on the receive path in the GoCQL driver by reducing redundant string allocations and adding a fast-path in type parsing, while also adding a unit test for prepared batch metadata involving multiple keyspaces/tables.
Changes:
- Optimize column metadata parsing by reusing keyspace/table where possible and adding
skipString()to avoid unnecessary allocations. - Add a fast-path in
readTypeInfo()for common “simple” native CQL types. - Add a unit test and test-server support for preparing a
BEGIN BATCH ... APPLY BATCHstatement with variables spanning multiple keyspaces/tables.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
frame.go |
Adds readTypeInfo() fast-path, introduces skipString(), and refactors column metadata parsing via readColWithSpec() plus metadata struct changes. |
conn_test.go |
Adds unit test for batch prepare metadata across multiple keyspaces/tables and updates test server PREPARE handling to support batch statements. |
There was a problem hiding this comment.
Pull request overview
This PR improves metadata parsing performance and correctness by reducing redundant keyspace/table reads, adding a fast-path for common native types in readTypeInfo(), and fixing prepared batch metadata parsing to read all statements’ metadata.
Changes:
- Cache keyspace/table at the result-metadata level and skip redundant per-column reads via
skipString(). - Add a fast-path in
readTypeInfo()for simple scalar native types. - Fix prepared batch metadata parsing to capture column metadata across all batch statements, with a new unit test.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.
| File | Description |
|---|---|
| frame.go | Adds type-info fast-path, optimizes metadata parsing by caching keyspace/table and skipping redundant per-column strings, and introduces skipString() plus prepared-metadata tracking for mixed table specs. |
| conn_test.go | Adds a unit test for prepared batch metadata across multiple keyspace/tables and extends the test server’s prepare handling to emit batch-metadata responses. |
23c8dfd to
467d091
Compare
|
Addressed review feedback: Fixed (this push):
Already fixed in prior revision:
Not changing (style preference):
Not a bug (Copilot false positive):
|
There was a problem hiding this comment.
Pull request overview
This PR improves receive-path efficiency and correctness in frame parsing for the GoCQL driver by reducing redundant allocations/reads in result metadata parsing, adding a fast-path for common type info parsing, and fixing prepared batch metadata handling (with a new unit test).
Changes:
- Optimize column metadata parsing by reusing keyspace/table at the result-metadata level and skipping redundant per-column strings via
skipString()andreadColWithSpec(). - Add a fast-path in
readTypeInfo()for simple native scalar types to avoid extra branching/work. - Fix/cover prepared batch metadata parsing for multi-table batches via a new unit test + test-server behavior.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
frame.go |
Adds readTypeInfo() fast-path, introduces skipString(), refactors column parsing into readColWithSpec(), and adjusts prepared/result metadata parsing behavior. |
conn_test.go |
Adds a unit test for multi-keyspace/table prepared batch metadata and updates the test server’s PREPARE handling to return appropriate synthetic metadata. |
467d091 to
52764ca
Compare
52764ca to
9fc5382
Compare
There was a problem hiding this comment.
Pull request overview
This PR improves receive-path performance in frame metadata parsing and fixes correctness for prepared batch metadata, especially when multiple keyspace/table pairs are involved.
Changes:
- Optimize column metadata parsing by reusing keyspace/table across columns (and skipping redundant per-column strings without allocating) when possible.
- Add a fast-path in
readTypeInfo()for native types that require no additional parsing. - Fix prepared batch metadata handling to correctly retain per-column keyspace/table information across mixed-table batches, with new unit coverage.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
frame.go |
Adds skipString(), reworks column spec parsing to avoid redundant allocations, adds readTypeInfo() fast-path, and improves prepared metadata handling for mixed batches. |
frame_test.go |
Adds a unit test validating per-column table spec parsing + skipString() alignment (buffer fully consumed). |
conn_test.go |
Adds a unit test ensuring prepared batch metadata preserves per-column keyspace/table for mixed-table batches; extends test server PREPARE handling to support it. |
…hort-circuit tracker
Read keyspace/table once before the column loop instead of per-column, add skipString() to skip wire strings without allocation, and handle the !globalSpec path correctly for prepared batch metadata where columns may reference different tables. - Add keyspaceTableTracker for prepared batch per-column spec - Replace readCol() with readColWithSpec() supporting both paths - Remove duplicate keyspace/table fields from resultMetadata - Add TestPrepareBatchMetadataMultipleKeyspaceTables - Add TestParseResultMetadata_PerColumnSpec Signed-off-by: Yaniv Kaul <yaniv.kaul@scylladb.com>
Add early return for native type IDs (0x0001 through TypeDuration), skipping the TypeCustom check and the collection/UDT/tuple switch. Also convert single-case type switch to a type assertion. Signed-off-by: Yaniv Kaul <yaniv.kaul@scylladb.com>
459700b to
c94bbe1
Compare
|
Error:
|
Summary
flagGlobalTableSpecis set) instead of per-column, and useskipString()to skip redundant per-column keyspace/table reads inreadColWithSpec()readTypeInfo()for simple native types (the common case), avoiding the switch statement overhead for types that need no further parsingExtracted from PR #699 (
rec_perf_improvementsbranch). Part 3 of 3 — the other two are:streamline_readHeader)rowdata_reflection_elimination)Commits
frame: optimize column metadata parsing by reading keyspace/table once— When the global table spec flag is set in result metadata, read keyspace and table name once and share them across all columns instead of re-reading per column. AddskipString()helper. Reduces per-column allocations.Optimize readTypeInfo() with fast-path for simple native types— For the most common type info kinds (varchar, int, bigint, blob, etc.), return aNativeTypedirectly without falling through the full switch. This avoids unnecessary branching for the ~15 simple types that need no sub-type parsing.Fix prepared batch metadata parsing— When preparing a batch, parse result metadata from all statements (not just the last one) so that column metadata is correctly captured for multi-table batches. Includes a unit test covering this case.Benchmark Results
Benchmarks measured with a temporary
metadata_bench_test.goexercisingparseResultMetadataandparsePreparedMetadatawith synthetic CQL frames.perColSpec(per-column keyspace/table — biggest improvement)~44% fewer allocations and ~55-60% faster for per-column spec results.
The reduction in allocations comes from
skipString()which avoids allocating strings that are immediately discarded (the per-column keyspace/table names when global spec is not set are still read but now skipped efficiently).globalSpec(global table spec — modest improvement from readTypeInfo fast-path)For globalSpec, allocations are unchanged (already optimal). The 100-column case shows clear improvement from the readTypeInfo fast-path; smaller sizes are within noise.
PreparedMetadata
Within noise — prepared metadata parsing was already efficient; the fix in commit 3 is a correctness change, not a performance one.
Testing
go test -tags unit -count=1 ./...— all tests passgo vet ./...— cleanTestPrepareBatchMetadataMultipleKeyspaceTablescovers the batch metadata fixNotes
globalSpecoptimization in commit 1 assumes CQL protocol v3+ where the global table spec flag is standard. All supported protocol versions (v3, v4, v5) use this flag.