Skip to content

Commit a4a3d3f

Browse files
authored
GH-39265: [Java] Make it run well with the netty newest version 4.1.104 (#39266)
### Describe the enhancement requested When I used `netty arrow memory 14.0.1` and `netty 4.1.104.Final` in Spark, the following error occurred, After pr: netty/netty#13613, `PoolArena` no longer extends `SizeClasses`, but instead uses it as one of its fields, as follows: <img width="1051" alt="image" src="https://github.com/apache/arrow/assets/15246973/6112757b-b2b7-42aa-b4c1-6ab473b91a09"> in order to ensure that `netty arrow memory 14.0.1` works well with `netty 4.1.104.Final` version, I suggest making similar modifications here. 1.Compilation errors are as follows: https://github.com/panbingkun/spark/actions/runs/7237466030/job/19717162391 <img width="1005" alt="image" src="https://github.com/apache/arrow/assets/15246973/98edb6a1-f0e6-4d4e-b568-fbdbffe612f0"> 2.Some bugs have been fixed in `netty 4.1.104.Final` as follows: <img width="862" alt="image" src="https://github.com/apache/arrow/assets/15246973/12354a1e-cddd-4ab8-b168-e92712d84cea"> <img width="861" alt="image" src="https://github.com/apache/arrow/assets/15246973/bd7d27e1-3953-451c-8c9b-24ecb0d61efd"> 4.1.104.Final release note: https://netty.io/news/2023/12/15/4-1-104-Final.html 4.1.103.Final release note: https://netty.io/news/2023/12/13/4-1-103-Final.html 4.1.101.Final release note: https://netty.io/news/2023/11/09/4-1-101-Final.html ### Component(s) Java * Closes: #39265 Authored-by: panbingkun <[email protected]> Signed-off-by: David Li <[email protected]>
1 parent cd5a1bd commit a4a3d3f

File tree

2 files changed

+3
-6
lines changed

2 files changed

+3
-6
lines changed

java/memory/memory-netty/src/main/java/io/netty/buffer/PooledByteBufAllocatorL.java

Lines changed: 2 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -71,7 +71,7 @@ public UnsafeDirectLittleEndian allocate(long size) {
7171
}
7272

7373
public int getChunkSize() {
74-
return allocator.chunkSize;
74+
return allocator.chunkSize();
7575
}
7676

7777
public long getHugeBufferSize() {
@@ -137,7 +137,6 @@ private class InnerAllocator extends PooledByteBufAllocator {
137137

138138
private final PoolArena<ByteBuffer>[] directArenas;
139139
private final MemoryStatusThread statusThread;
140-
private final int chunkSize;
141140

142141
public InnerAllocator() {
143142
super(true);
@@ -150,8 +149,6 @@ public InnerAllocator() {
150149
throw new RuntimeException("Failure while initializing allocator. Unable to retrieve direct arenas field.", e);
151150
}
152151

153-
this.chunkSize = directArenas[0].chunkSize;
154-
155152
if (memoryLogger.isTraceEnabled()) {
156153
statusThread = new MemoryStatusThread(this);
157154
statusThread.start();
@@ -166,7 +163,7 @@ private UnsafeDirectLittleEndian newDirectBufferL(int initialCapacity, int maxCa
166163

167164
if (directArena != null) {
168165

169-
if (initialCapacity > directArena.chunkSize) {
166+
if (initialCapacity > chunkSize()) {
170167
// This is beyond chunk size so we'll allocate separately.
171168
ByteBuf buf = UnpooledByteBufAllocator.DEFAULT.directBuffer(initialCapacity, maxCapacity);
172169

java/pom.xml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -33,7 +33,7 @@
3333
<dep.junit.jupiter.version>5.10.1</dep.junit.jupiter.version>
3434
<dep.slf4j.version>2.0.9</dep.slf4j.version>
3535
<dep.guava-bom.version>32.1.3-jre</dep.guava-bom.version>
36-
<dep.netty-bom.version>4.1.100.Final</dep.netty-bom.version>
36+
<dep.netty-bom.version>4.1.104.Final</dep.netty-bom.version>
3737
<dep.grpc-bom.version>1.60.0</dep.grpc-bom.version>
3838
<dep.protobuf-bom.version>3.23.1</dep.protobuf-bom.version>
3939
<dep.jackson-bom.version>2.16.0</dep.jackson-bom.version>

0 commit comments

Comments
 (0)