Skip to content

[feat] Add observability to OutOfMemoryErrors when OutOfMemoryPolicy is FallbackToHeap #4534

Open
@lhotari

Description

@lhotari

FEATURE REQUEST

Currently when direct memory allocation fails with OutOfMemoryError, the default OutOfMemoryPolicy is FallbackToHeap and the exception is catched silently:

} catch (OutOfMemoryError e) {
if (canFallbackToHeap && outOfMemoryPolicy == OutOfMemoryPolicy.FallbackToHeap) {
try {
return unpooledAllocator.heapBuffer(initialCapacity, maxCapacity);
} catch (OutOfMemoryError e2) {
consumeOOMError(e2);
throw e2;
}
} else {
// ThrowException
consumeOOMError(e);
throw e;
}
}

It would be great to be able to observe this condition. Pulsar uses ByteBufAllocatorImpl also for Pulsar client and it's hard to detect that Netty direct buffer direct access isn't properly configured unless there's a way to observe the condition.
Netty direct memory buffer access is explained in Pulsar client's "Java client Performance considerations", https://pulsar.apache.org/docs/4.0.x/client-libraries-java-setup/#java-client-performance .

After there's a counter in ByteBufAllocatorImpl, it would be possible to implement a metric in Pulsar client.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions