fix(contrib/drivers/pgsql): preserve bytea data integrity on read and write#4678
fix(contrib/drivers/pgsql): preserve bytea data integrity on read and write#4678lingcoder wants to merge 1 commit intogogf:masterfrom
Conversation
… write Fix two bytea data corruption issues in PostgreSQL driver: 1. READ path (gogf#4677): `CheckLocalTypeForField` and `ConvertValueForLocal` had no case for plain "bytea" type, causing it to fall through to Core layer which incorrectly mapped it to `LocalTypeString`. Binary data was converted to string via `gconv.String()`, corrupting the bytes. 2. WRITE path (gogf#4231): `ConvertValueForField` applied PostgreSQL array syntax conversion (`[`->`{`, `]`->`}`) to all slice types including `[]byte` for bytea columns, corrupting bytes 0x5B(`[`) and 0x5D(`]`). closes gogf#4677, closes gogf#4231
ac1dba1 to
96f7299
Compare
There was a problem hiding this comment.
Pull request overview
This PR fixes two bytea data corruption paths in the PostgreSQL driver by ensuring bytea is treated as binary ([]byte) end-to-end rather than falling back to string conversion, and by preventing array-syntax rewriting from mutating raw []byte payloads.
Changes:
- Add explicit
bytea -> LocalTypeBytesmapping andbyteahandling inConvertValueForLocalto preserve raw bytes on reads. - Skip PostgreSQL slice/array syntax conversion (
[/]→{/}) for[]bytewhen the target field isbyteaon writes. - Add unit + integration/issue tests covering byte patterns that previously triggered corruption (including
0x00,0x5B,0x5D,0xFF).
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| contrib/drivers/pgsql/pgsql_convert.go | Ensures bytea maps to []byte on read and prevents write-time bracket replacement for []byte destined for bytea. |
| contrib/drivers/pgsql/pgsql_z_unit_convert_test.go | Adds focused unit tests validating the new bytea mapping/convert behavior. |
| contrib/drivers/pgsql/pgsql_z_unit_issue_test.go | Adds regression tests for #4677 and #4231 using real round-trip DB operations. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| @@ -154,6 +163,13 @@ func (d *Driver) ConvertValueForLocal(ctx context.Context, fieldType string, fie | |||
| // Basic types are mostly handled by Core layer, only handle array types here | |||
There was a problem hiding this comment.
The comment says this function only handles array types, but the newly-added case "bytea" is a basic (non-array) type. Please update the comment to reflect the current behavior so future changes don’t accidentally regress bytea handling.
| // Basic types are mostly handled by Core layer, only handle array types here | |
| // Basic types are mostly handled by Core layer; handle array types and special-case bytea here. |
| if v, ok := fieldValue.([]byte); ok { | ||
| return v, nil | ||
| } | ||
| return fieldValue, nil |
There was a problem hiding this comment.
In ConvertValueForLocal, the new bytea branch returns fieldValue unchanged when it’s not already a []byte. For consistency with the bytea -> []byte mapping (and to keep all conversions centralized), it may be clearer to delegate to the Core conversion or explicitly convert non-[]byte inputs to []byte instead of returning them as-is.
| return fieldValue, nil | |
| return d.Core.ConvertValueForLocal(ctx, fieldType, fieldValue) |
|
@gqcn 关于bytea处理这块的建议: 保留原始[]byte值不调用Core.ConvertDataForRecord是有意的设计决定,主要考虑:
这个设计让bytea的行为更透明可预测,也符合最小惊讶原则。感谢review!👍 |
Summary
Fix two bytea data corruption issues in the PostgreSQL driver:
READ path (fixes record.Get().Bytes() corrupts bytea data on retrieval from PostgreSQL #4677):
CheckLocalTypeForFieldandConvertValueForLocalhad no case for plainbyteatype, causing it to fall through to the Core layer which incorrectly mapped it toLocalTypeString. Binary data was then converted to string viagconv.String(), corrupting the bytes on retrieval.WRITE path (fixes contrib/drivers/pgsql/pgsql_convert.go: If the byte type data contains 0x5D, it will be forcibly converted to 0x7D, causing problems with the stored data. #4231):
ConvertValueForFieldapplied PostgreSQL array syntax conversion ([→{,]→}) to all slice types including[]bytefor bytea columns, corrupting bytes0x5B([) and0x5D(]) on insertion.Changes
contrib/drivers/pgsql/pgsql_convert.go:CheckLocalTypeForField: Addcase "bytea"→LocalTypeBytesConvertValueForLocal: Addcase "bytea"to preserve[]byteas-isConvertValueForField: Skip[]→{}replacement for[]bytewithbyteafield typecontrib/drivers/pgsql/pgsql_z_unit_convert_test.go:byteatype inCheckLocalTypeForField,ConvertValueForLocal, andConvertValueForFieldcontrib/drivers/pgsql/pgsql_z_unit_issue_test.go:Test_Issue4677: End-to-end round-trip test with various binary data (including 0x00, 0x5B, 0x5D, 0xFF)Test_Issue4231: Targeted test for 0x5D byte corruption on writeTest plan
Test_CheckLocalTypeForField- bytea returnsLocalTypeBytesTest_ConvertValueForLocal- bytea preserves[]byteas-isTest_ConvertValueForField- bytea skips array syntax replacementTest_Issue4677- full DB round-trip with binary dataTest_Issue4231- write path preserves 0x5B/0x5D bytescloses #4677
closes #4231
ref #4689