Summary:
A memory leak occurs in the EclipseLink identity map under high capacity conditions. When a bulk data load takes place and the majority of possible entities are loaded into the cache, the WeakCacheKey cleanup mechanism permanently ceases to execute. This happens because the number of new cache insertions never reaches the required threshold to trigger the cleanup, leading to an accumulated memory footprint of dead keys.
Detailed Failure Mechanism:
After a massive entity load where the identity map reaches near-full or full capacity of possible entities, the cleanup process gets permanently blocked by the combination of the following conditions in the EclipseLink source code:
WeakIdentityMap.checkCleanup: The cleanup is gated by the condition if (this.cleanupCount > this.cleanupSize). However, once most entities are already cached, the increment rate of cleanupCount drops significantly or stops completely because new entities are rarely being instantiated and inserted anymore. Furthermore, cleanupSize never drops.
|
protected void checkCleanup() { |
WeakIdentityMap.putCacheKeyIfAbsent: Inside this method, the increment and the subsequent call to checkCleanup() are guarded by the condition if (cacheKey == null). Since the majority of objects are already present in the identity map, cacheKey is almost always found (not null). As a result, the code path that increments the counter and triggers the cleanup is completely bypassed.Calls to putCacheKeyIfAbsent: Furthermore, most calls to this method are themselves guarded by conditions that check if the cache key is already present, preventing the method from being executed at all.
Consequently, the threshold required to initiate checkCleanup is never reached again. The dead keys remain referenced within the internal map structure, preventing the JVM from reclaiming the memory and causing a progressive memory leak. The memory leak appears to be roughly 500 MB per 1 million keys.
Steps to Reproduce:
- Initialize an EclipseLink session with an identity map configured to use
WeakCacheKey.
- Perform a bulk load operation that populates the cache with the vast majority of available database records (heavy cache load).
- Trigger entity garbage collection (simulate business objects becoming unreferenced/dead).
- Observe that further read operations or minor cache interactions do not trigger the
WeakCacheKey cleanup because cacheKey == null evaluates to false and cleanupCount never exceeds cleanupSize. The dead keys' memory footprint remains in the heap.
Expected Behavior:
The cleanup threshold logic should account for high-capacity states. Even when the majority of entities are already cached (resulting in cache hits), the system must ensure that the accumulated dead keys' footprint is periodically evaluated and managed, rather than indefinitely suppressing the cleanup task.
Temporary Workaround
Attached is a modified WeakIdentityMap.java to mitigate the leak. This patch is limited to specific high entity volumes; for a proper long-term solution, utilizing a ReferenceQueue and potentially considering CacheKey reuse would be more robust.
WeakIdentityMap.java
Summary:
A memory leak occurs in the EclipseLink identity map under high capacity conditions. When a bulk data load takes place and the majority of possible entities are loaded into the cache, the
WeakCacheKeycleanup mechanism permanently ceases to execute. This happens because the number of new cache insertions never reaches the required threshold to trigger the cleanup, leading to an accumulated memory footprint of dead keys.Detailed Failure Mechanism:
After a massive entity load where the identity map reaches near-full or full capacity of possible entities, the cleanup process gets permanently blocked by the combination of the following conditions in the EclipseLink source code:
WeakIdentityMap.checkCleanup: The cleanup is gated by the conditionif (this.cleanupCount > this.cleanupSize). However, once most entities are already cached, the increment rate ofcleanupCountdrops significantly or stops completely because new entities are rarely being instantiated and inserted anymore. Furthermore,cleanupSizenever drops.eclipselink/foundation/org.eclipse.persistence.core/src/main/java/org/eclipse/persistence/internal/identitymaps/WeakIdentityMap.java
Line 96 in b81f620
WeakIdentityMap.putCacheKeyIfAbsent: Inside this method, the increment and the subsequent call tocheckCleanup()are guarded by the conditionif (cacheKey == null). Since the majority of objects are already present in the identity map,cacheKeyis almost always found (not null). As a result, the code path that increments the counter and triggers the cleanup is completely bypassed.Calls to putCacheKeyIfAbsent: Furthermore, most calls to this method are themselves guarded by conditions that check if the cache key is already present, preventing the method from being executed at all.Consequently, the threshold required to initiate
checkCleanupis never reached again. The dead keys remain referenced within the internal map structure, preventing the JVM from reclaiming the memory and causing a progressive memory leak. The memory leak appears to be roughly 500 MB per 1 million keys.Steps to Reproduce:
WeakCacheKey.WeakCacheKeycleanup becausecacheKey == nullevaluates to false andcleanupCountnever exceedscleanupSize. The dead keys' memory footprint remains in the heap.Expected Behavior:
The cleanup threshold logic should account for high-capacity states. Even when the majority of entities are already cached (resulting in cache hits), the system must ensure that the accumulated dead keys' footprint is periodically evaluated and managed, rather than indefinitely suppressing the cleanup task.
Temporary Workaround
Attached is a modified WeakIdentityMap.java to mitigate the leak. This patch is limited to specific high entity volumes; for a proper long-term solution, utilizing a ReferenceQueue and potentially considering CacheKey reuse would be more robust.
WeakIdentityMap.java