Client lazy parse ByteBuf to avoid deeply memory copy

[swuferhong]

Search before asking

• I searched in the issues and found nothing similar.

Motivation

Currently, we will do one deeply memory copy when receive Response from server, see NettyClientHandler#channelRead() for more details:

This behavior would unnecessarily increase heap memory usage and negatively impact read performance. Therefore, if precise manual buffer release can be achieved, we can eliminate memory copying here.

Solution

No response

Anything else?

No response

Willingness to contribute

• I'm willing to submit a PR!

[kaori-seasons]

OK, thanks for the problem you found. I have made some summary. You are welcome to check if it meets your expectations.

Current situation analysis

The current system has the problem of deep memory copy when processing ByteBuf, which is mainly reflected in the following aspects:

1. Deep copy in RPC message conversion: During the conversion process from ByteBuf to ByteBuffer, when the buffer is neither a direct buffer nor an array-backed buffer, a deep copy will be performed

2. Existing lazy parsing mechanism: The system has implemented some lazy parsing mechanisms in protogen, and uses the isLazilyParsed method to determine whether the message retains a reference to the original ByteBuf

3. Buffer release management: The existing releaseBuffer mechanism is used to manage the life cycle of ByteBuf

Detailed transformation plan

1. Extend lazy parsing mechanism

Goal: Extend the existing protogen lazy parsing mechanism to more message types, especially the response messages received by the client

Implementation plan:

• Add lazy parsing support for more field types in the protogen generator
• Extend the isLazilyParsed check logic to cover the processing of client response messages

2. Optimize ByteBuf conversion strategy

Goal: Reduce deep copying in CommonRpcMessageUtils and use zero copy or reference passing instead.

Implementation plan:

• Modify the implementation logic of the toByteBuffer method and give priority to the nioBuffer() method for zero copy conversion
• When copying is necessary, implement a delayed copy mechanism and only copy when actually accessing data
• Leverage the existing SendWritableOutput zero copy mechanism

3. Introduce smart buffer management

Goal: Implement precise manual buffer release to ensure correct memory management in lazy parsing scenarios.

Implementation plan:

• Extend the existing buffer release mechanism and add reference count management in client request processing
• Refer to the server-side processing method and implement similar releaseBuffer logic on the client side 6
• Add the corresponding buffer release call in the client request handler

4. Implement lazy BytesView mechanism

Goal: Use the existing BytesView abstraction to implement lazy access to client data.

Implementation plan:

• Extend the implementation of the BytesView interface to support lazy access to ByteBuf 7
• Implement a client-specific BytesView implementation similar to MemorySegmentBytesView and FileRegionBytesView
• Use the getZeroCopyLength method to optimize zero-copy transmission

5. Client response processing optimization

Goal: Integrate lazy parsing mechanism in the client response processing flow.

Implementation plan:

• Modify the client response parser to support lazy parsing mode
• Apply lazy parsing mechanism in key components such as LogFetcher
• Ensure that the buffer is correctly released in the replica fetcher

Implementation steps

1. Phase 1: Extend the protogen generator to support lazy parsing of more fields
2. Phase 2: Optimize the ByteBuf conversion logic of CommonRpcMessageUtils
3. Phase 3: Implement intelligent buffer management mechanism on the client
4. Phase 4: Integrate all components and conduct end-to-end testing and performance verification

[kaori-seasons]

For the first phase, the current system has implemented a lazy parsing mechanism for the records field, which is handled by the ProtobufRecordsField class. ProtobufField.java:56-58 I need to extend this mechanism to more field types

for example:

public class LazyFieldFinder {  
  
    public static final String RECORDS_FIELD_NAME = "records";  
      
    // 扩展支持惰性解析的字段名称  
    public static final Set<String> LAZY_BYTES_FIELD_NAMES = Set.of(  
        "records",  
        "data",   
        "payload",  
        "content",  
        "body"  
    );  
      
    // 支持惰性解析的字符串字段（通常是大文本）  
    public static final Set<String> LAZY_STRING_FIELD_NAMES = Set.of(  
        "message",  
        "description",  
        "metadata",  
        "config"  
    );  
  
    public static boolean hasLazyField(Message message) {  
        return message.getFields().stream().anyMatch(LazyFieldFinder::hasLazyField);  
    }  
  
    private static boolean hasLazyField(Field<?> field) {  
        if (field instanceof MessageField) {  
            return hasLazyField(((MessageField) field).getMessage());  
        } else if (field instanceof Field.Bytes) {  
            return !field.isRepeated() && LAZY_BYTES_FIELD_NAMES.contains(field.getName());  
        } else if (field instanceof Field.String) {  
            return !field.isRepeated() && LAZY_STRING_FIELD_NAMES.contains(field.getName());  
        } else {  
            return false;  
        }  
    }  
      
    public static boolean isLazyBytesField(Field<?> field) {  
        return field instanceof Field.Bytes &&   
               !field.isRepeated() &&   
               LAZY_BYTES_FIELD_NAMES.contains(field.getName());  
    }  
      
    public static boolean isLazyStringField(Field<?> field) {  
        return field instanceof Field.String &&   
               !field.isRepeated() &&   
               LAZY_STRING_FIELD_NAMES.contains(field.getName());  
    }  
}