Skip to content

feat: support Firebird BLOB seek in proxy#39081

Open
Zhengcy05 wants to merge 3 commits into
apache:masterfrom
Zhengcy05:feat/support-firebird-seek-blob
Open

feat: support Firebird BLOB seek in proxy#39081
Zhengcy05 wants to merge 3 commits into
apache:masterfrom
Zhengcy05:feat/support-firebird-seek-blob

Conversation

@Zhengcy05

Copy link
Copy Markdown
Contributor

Special note: Since there are many design changes this time, I sincerely hope someone will carefully review this PR. Thx😘

Fixes #39051.

Changes proposed in this pull request:

  • Track BLOB read state per connection and per blob handle.
  • Make OPEN_BLOB register content and cursor state.
  • Make SEEK_BLOB return the resulting cursor position.
  • Make GET_SEGMENT read from the current cursor and return EOF correctly.
  • Make INFO_BLOB read the current BLOB length from the same state.
  • Clean up BLOB state on close, cancel, and connection release.

Validation:

  • ./mvnw -pl database/protocol/dialect/firebird -am -Dspotless.skip=true -Dtest=FirebirdBlobRegistryTest,FirebirdBlobInfoReturnPacketTest -Dsurefire.failIfNoSpecifiedTests=false test
  • ./mvnw -pl proxy/frontend/dialect/firebird -am -DskipITs -Dspotless.skip=true -Djacoco.skip=true -Dtest=FirebirdOpenBlobCommandExecutorTest,FirebirdCloseBlobCommandExecutorTest,FirebirdCancelBlobCommandExecutorTest,FirebirdGetBlobSegmentCommandExecutorTest,FirebirdSeekBlobCommandExecutorTest,FirebirdBlobInfoExecutorTest,FirebirdFrontendEngineTest,FirebirdAuthenticationEngineTest,FirebirdCommandExecuteEngineTest -Dsurefire.failIfNoSpecifiedTests=false -DargLine='-javaagent:/Users/didi/.m2/repository/net/bytebuddy/byte-buddy-agent/1.18.10/byte-buddy-agent-1.18.10.jar -Djdk.attach.allowAttachSelf=true -XX:+EnableDynamicAgentLoading' test
  • ./mvnw spotless:apply -Pcheck -T1C
  • ./mvnw checkstyle:check -Pcheck -T1C

Before committing this PR, I'm sure that I have checked the following options:
- [x] My code follows the [code of conduct](https://shardingsphere.apache.org/community/en/involved/conduct/code/) of this project.
- [x] I have self-reviewed the commit code.
- [ ] I have (or in comment I request) added corresponding labels for the pull request.
- [ ] I have passed maven check locally : `./mvnw clean install -B -T1C -Dmaven.javadoc.skip -Dmaven.jacoco.skip -e`.
- [ ] I have made corresponding changes to the documentation.
- [x] I have added corresponding unit tests for my changes.
- [ ] I have updated the Release Notes of the current development version. For more details, see [Update Release Note](https://shardingsphere.apache.org/community/en/involved/contribute/contributor/)

@makssent

Copy link
Copy Markdown
Contributor

I've also been working on BLOB support in the Firebird proxy (#39050). The original issue only covered op_seek_blob, but since this PR implements the full BLOB flow, I'd like to share two bugs I ran into while testing BLOB scenarios - both reproduce on this branch as well:

1. The deferred BLOB handle 0xFFFF (INVALID_OBJECT) is not resolved.

Jaybird 5.0.7+ pipelines BLOB operations: it sends op_put_segment / op_info_blob / op_get_segment right after op_create_blob2 / op_open_blob2 without waiting for the response, using the placeholder 0xFFFF (INVALID_OBJECT) in place of the not-yet-known handle. The Firebird server substitutes the handle of the most recently created object in this case, while the proxy looks up 0xFFFF literally:

  • on write, op_put_segment finds no blob and silently drops the segment → the row is inserted with empty BLOBs;
  • on read, op_get_segment returns empty data and never reports the end-of-blob state (p_resp_object = 2), so the client's read loop never terminates.

Reproducible with Jaybird 5.0.10: INSERT stores empty BLOBs, and a subsequent SELECT of a BLOB column hangs.

2. The op_get_segment response declares the alignment padding as part of the data.

The response data must be [2-byte little-endian length][segment bytes], with the 4-byte alignment padding written outside the declared length. Currently the declared length includes the padding, so Jaybird reads the padding as data and runs past the buffer on any segment whose length % 4 == 1 or 3:

java.lang.ArrayIndexOutOfBoundsException: Index 26 out of bounds for length 26
    at org.firebirdsql.gds.VaxEncoding.iscVaxInteger2(VaxEncoding.java:165)
    at org.firebirdsql.gds.ng.wire.version10.V10InputBlob.getSegment(V10InputBlob.java:136)
    at org.firebirdsql.jdbc.field.FBBlobField.getBytesInternal(FBBlobField.java:149)

Reproducible with Jaybird 5.0.6 by reading back a 21-byte BLOB (2 + 21 = 23 bytes of payload, 3 bytes of padding, declared length 26). Segments whose length % 4 == 0 or 2 survive by luck — that's why small ASCII test payloads often pass. Large BLOBs always fail: the client requests chunks of 32765 bytes (32765 % 4 == 1).

@Zhengcy05

Zhengcy05 commented Jul 14, 2026

Copy link
Copy Markdown
Contributor Author

I've also been working on BLOB support in the Firebird proxy (#39050). The original issue only covered op_seek_blob, but since this PR implements the full BLOB flow, I'd like to share two bugs I ran into while testing BLOB scenarios - both reproduce on this branch as well:

1. The deferred BLOB handle 0xFFFF (INVALID_OBJECT) is not resolved.

Jaybird 5.0.7+ pipelines BLOB operations: it sends op_put_segment / op_info_blob / op_get_segment right after op_create_blob2 / op_open_blob2 without waiting for the response, using the placeholder 0xFFFF (INVALID_OBJECT) in place of the not-yet-known handle. The Firebird server substitutes the handle of the most recently created object in this case, while the proxy looks up 0xFFFF literally:

  • on write, op_put_segment finds no blob and silently drops the segment → the row is inserted with empty BLOBs;
  • on read, op_get_segment returns empty data and never reports the end-of-blob state (p_resp_object = 2), so the client's read loop never terminates.

Reproducible with Jaybird 5.0.10: INSERT stores empty BLOBs, and a subsequent SELECT of a BLOB column hangs.

2. The op_get_segment response declares the alignment padding as part of the data.

The response data must be [2-byte little-endian length][segment bytes], with the 4-byte alignment padding written outside the declared length. Currently the declared length includes the padding, so Jaybird reads the padding as data and runs past the buffer on any segment whose length % 4 == 1 or 3:

java.lang.ArrayIndexOutOfBoundsException: Index 26 out of bounds for length 26
    at org.firebirdsql.gds.VaxEncoding.iscVaxInteger2(VaxEncoding.java:165)
    at org.firebirdsql.gds.ng.wire.version10.V10InputBlob.getSegment(V10InputBlob.java:136)
    at org.firebirdsql.jdbc.field.FBBlobField.getBytesInternal(FBBlobField.java:149)

Reproducible with Jaybird 5.0.6 by reading back a 21-byte BLOB (2 + 21 = 23 bytes of payload, 3 bytes of padding, declared length 26). Segments whose length % 4 == 0 or 2 survive by luck — that's why small ASCII test payloads often pass. Large BLOBs always fail: the client requests chunks of 32765 bytes (32765 % 4 == 1).

Thank you very much, I will make corrections

@Zhengcy05

Copy link
Copy Markdown
Contributor Author

Thank you very much for this detailed report — both bugs have been reproduced on my end and have now been fixed.

1. Unresolved delay handle '0xFFFF'

Root cause: Both 'FirebirdCreateBlobCommandExecutor' and 'FirebirdOpenBlobCommandExecutor' generate a new handle and return immediately, but neither records the handle for subsequent operations. When Jaybird uses '0xFFFF' to pipeline before the response returns, all subsequent executors (put-segment, get-segment, seek, close, cancel) directly look for '0xFFFF' and get an empty value—causing silent drops (data) during writing and deadlocks when reading.

Fix: 'FirebirdBlobRegistry' now tracks the last created/opened blob handle by connection. 'create_blob2' and 'open_blob2' are both recorded using 'setLastBlobHandle()'. All five subsequent executors now call 'resolveBlobHandle()' before each registry/cache lookup, and when the client sends '0xFFFF', it returns the last handle—which matches how a real Firebird server replaces the recently created object handle.

2. Incorrect fill length in the 'op_get_segment' response

Root cause: 'FirebirdGetBlobSegmentResponsePacket.write()' adds its own internal 4-byte alignment padding after a 2-byte prefix. Because 'FirebirdGenericResponsePacket' encapsulates response data with its XDR length prefix, the internal padding is included in that length. For a 21-byte fragment: declared length = 2 + 21 + 3 (padded) = 26, so Jaybird reads the 21-byte data and interprets the 3 padding bytes as the prefix -> for the next 2-byte fragment length, causing buffer overrun.

Fix: Removed internal padding from 'FirebirdGetBlobSegmentResponsePacket.write()'. The generic response encapsulation has applied the correct external alignment via '(4 - length) & 3', which lies outside the declared data length, which is exactly what the protocol requires.

Now, both fixes include dedicated unit tests, such as the 'assertExecuteWithDeferredHandle' test, which verifies '0xFFFF' parsing on all five subsequent executors, as well as updated response packet tests for misaligned fragments. If you wish to coordinate BLOB testing between these two PRs, I am happy to help.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support op_seek_blob operation in Firebird Proxy

2 participants