Skip to content

Commit bf2331e

Browse files
authored
perf: use a stream mark to read null-terminated string (GoogleCloudPlatform#3103)
Prepared statements and portals can have names in the PG wire-protocol. These are sent as null-terminated strings. Read these strings by setting a mark in the stream and then scan for the null-terminator, instead of always creating a 128-byte buffer and read into that buffer. Using a stream mark will instead re-use the existing buffer in the existing input stream, and so reduce the amount of memory that is being reserved and released for each statement that is received.
1 parent b6a71b7 commit bf2331e

1 file changed

Lines changed: 27 additions & 11 deletions

File tree

src/main/java/com/google/cloud/spanner/pgadapter/wireprotocol/WireMessage.java

Lines changed: 27 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,6 @@
2323
import java.io.IOException;
2424
import java.nio.charset.StandardCharsets;
2525
import java.text.MessageFormat;
26-
import java.util.Arrays;
2726
import java.util.logging.Level;
2827
import java.util.logging.Logger;
2928

@@ -149,30 +148,47 @@ public String read(int length) throws IOException {
149148
}
150149

151150
/**
152-
* Reads a null-terminated string from a {@link DataInputStream}. Note that though existing
153-
* solutions for this exist, they are either not keyed exactly for our use case, or would lead to
154-
* a more combersome addition to this codebase. Also note the 128 byte length is chosen from
155-
* profiling and determining that it exceeds the 90th percentile size for inbound messages.
151+
* The max number of characters to read when scanning for a null-terminator. Null-terminated
152+
* strings are used in the PG wire-protocol for names of prepared statements and portals, but also
153+
* for the SQL string of a statement. We therefore need to read potentially very long strings.
154+
*/
155+
private static final int MARK_READ_LIMIT = 100_000_000;
156+
157+
/**
158+
* Reads a null-terminated string from a {@link DataInputStream}.
156159
*
157160
* @return the string.
158161
* @throws IOException if an error occurs while reading from the stream, or if no null-terminator
159162
* is found before the end of the stream.
160163
*/
161164
public String readString() throws IOException {
162-
byte[] buffer = new byte[128];
165+
this.inputStream.mark(MARK_READ_LIMIT);
163166
int index = 0;
164-
while (true) {
167+
while (index < MARK_READ_LIMIT) {
165168
byte b = this.inputStream.readByte();
166169
if (b == 0) {
167170
break;
168171
}
169-
buffer[index] = b;
170172
index++;
171-
if (index == buffer.length) {
172-
buffer = Arrays.copyOf(buffer, buffer.length * 2);
173+
if (index == MARK_READ_LIMIT) {
174+
throw new IOException("No null terminator found");
173175
}
174176
}
175-
return new String(buffer, 0, index, StandardCharsets.UTF_8);
177+
// Reset the stream to the mark and read the name (if any).
178+
this.inputStream.reset();
179+
if (index == 0) {
180+
// No name, but we still need to skip the null-terminator.
181+
//noinspection StatementWithEmptyBody
182+
while (this.inputStream.skip(1) < 1) {}
183+
return "";
184+
}
185+
186+
byte[] result = new byte[index];
187+
this.inputStream.readFully(result);
188+
// Skip the null-terminator.
189+
//noinspection StatementWithEmptyBody
190+
while (this.inputStream.skip(1) < 1) {}
191+
return new String(result, StandardCharsets.UTF_8);
176192
}
177193

178194
/**

0 commit comments

Comments
 (0)