Pooling Deep Dive

The engine uses extensive object pooling to reduce GC pressure and improve performance. This page covers the pooling infrastructure in detail.

Overview

flowchart TB
    subgraph Core["Core Pool Infrastructure"]
        ObjectPool((ObjectPool&lt;T&gt;))
        BucketedArrayPool((BucketedArrayPool&lt;T&gt;))
    end
    
    subgraph Specialized["Specialized Pools"]
        JsSlotArrayPool((JsSlotArrayPool))
        IteratorResultObjectPool((IteratorResultObjectPool))
        EnvPool["JsEnvironment Pool"]
        DriverPools["Iterator Driver Pools"]
    end
    
    ObjectPool --> JsSlotArrayPool
    BucketedArrayPool --> JsSlotArrayPool
    ObjectPool --> IteratorResultObjectPool
    ObjectPool --> EnvPool
    ObjectPool --> DriverPools

Key Files

File	Purpose
`ObjectPool.cs`	Generic lock-free object pool
`BucketedArrayPool.cs`	Fixed-size bucket array pool
`JsSlotArrayPool.cs`	Specialized pool for slot arrays
`JsTypes/IteratorResultObjectPool.cs`	Pool for iterator results
`PoolDebug.cs`	DEBUG-only leak detection
`PoolGuard.cs`	Runtime leak detection

ObjectPool

A fast, lock-free object pool using a fixed-size array.

Implementation

internal sealed class ObjectPool<T>(int size, Func<T> factory)
    where T : class
{
    private static readonly bool IsRentable = typeof(IRentable).IsAssignableFrom(typeof(T));
    private readonly T?[] _items = new T?[size];

    public T Rent(ILogger? logger = null)
    {
#if DISABLE_POOLING
        // Kill switch: always create fresh instance
        var freshItem = factory();
        if (IsRentable)
            ((IRentable)freshItem).OnRent(logger);
        return freshItem;
#else
        var items = _items;
        for (var i = 0; i < items.Length; i++)
        {
            var item = items[i];
            if (item is not null && 
                Interlocked.CompareExchange(ref items[i], null, item) == item)
            {
                if (IsRentable)
                    ((IRentable)item).OnRent(logger);
                PoolDebug.MarkLeased(item);
                return item;
            }
        }

        var newItem = factory();
        if (IsRentable)
            ((IRentable)newItem).OnRent(logger);
        PoolDebug.MarkLeased(newItem);
        return newItem;
#endif
    }

    public void Return(T item, ILogger? logger = null)
    {
#if DISABLE_POOLING
        // Kill switch: do nothing, let GC handle it
        return;
#else
        if (IsRentable)
            ((IRentable)item).OnReturn(logger);
        PoolDebug.MarkReturned(item);

        var items = _items;
        for (var i = 0; i < items.Length; i++)
        {
            if (items[i] is null && 
                Interlocked.CompareExchange(ref items[i], item, null) is null)
            {
                return;
            }
        }
        // Pool full, item will be GC'd
#endif
    }
}

Lock-Free Algorithm

flowchart TD
    Rent((Rent)) --> Scan["Scan array slots"]
    Scan --> Found{Found item?}
    Found -->|Yes| CAS["CompareExchange\n(item, null)"]
    CAS --> Success{CAS Success?}
    Success -->|Yes| Return["Return item"]
    Success -->|No| Scan
    Found -->|No| Create["Create new item"]
    Create --> Return

The pool uses Interlocked.CompareExchange for thread-safe access without locks:

Rent: Find non-null slot, atomically swap to null
Return: Find null slot, atomically swap to item

IRentable Interface

public interface IRentable
{
    void OnRent(ILogger? logger);
    void OnReturn(ILogger? logger);
}

Types implementing IRentable get callbacks for:

OnRent: Initialize/reset state when rented
OnReturn: Clean up references when returned

BucketedArrayPool

Fixed-size bucket pool for arrays to avoid repeated allocations.

Bucket Sizes

flowchart LR
    Request["Request Size"] --> Round["Round Up"]
    Round --> B8["8"]
    Round --> B16["16"]
    Round --> B32["32"]
    Round --> B64["64"]
    Round --> B128["128"]
    Round --> B256["256"]
    Round --> B512["512"]
    Round --> B1024["1024"]
    Round --> Alloc["Allocate (>1024)"]

Implementation

internal sealed class BucketedArrayPool<T>
{
    private const int PoolSize = 100;  // Pre-heat size per bucket

    private readonly ObjectPool<T[]> _pool8 = new(PoolSize, static () => new T[8]);
    private readonly ObjectPool<T[]> _pool16 = new(PoolSize, static () => new T[16]);
    private readonly ObjectPool<T[]> _pool32 = new(PoolSize, static () => new T[32]);
    private readonly ObjectPool<T[]> _pool64 = new(PoolSize, static () => new T[64]);
    private readonly ObjectPool<T[]> _pool128 = new(PoolSize, static () => new T[128]);
    private readonly ObjectPool<T[]> _pool256 = new(PoolSize, static () => new T[256]);
    private readonly ObjectPool<T[]> _pool512 = new(PoolSize, static () => new T[512]);
    private readonly ObjectPool<T[]> _pool1024 = new(PoolSize, static () => new T[1024]);

    public T[] Rent(int minimumLength)
    {
        return minimumLength switch
        {
            <= 8 => _pool8.Rent(),
            <= 16 => _pool16.Rent(),
            <= 32 => _pool32.Rent(),
            <= 64 => _pool64.Rent(),
            <= 128 => _pool128.Rent(),
            <= 256 => _pool256.Rent(),
            <= 512 => _pool512.Rent(),
            <= 1024 => _pool1024.Rent(),
            _ => new T[minimumLength]  // Rare case, just allocate
        };
    }

    public void Return(T[]? array)
    {
        if (array is null) return;

        switch (array.Length)
        {
            case 8: _pool8.Return(array); break;
            case 16: _pool16.Return(array); break;
            case 32: _pool32.Return(array); break;
            case 64: _pool64.Return(array); break;
            case 128: _pool128.Return(array); break;
            case 256: _pool256.Return(array); break;
            case 512: _pool512.Return(array); break;
            case 1024: _pool1024.Return(array); break;
            // Non-pooled sizes are just discarded
        }
    }
}

No Array.Clear()

The pool intentionally skips Array.Clear() because:

/// NOTE: We skip Array.Clear() here because callers are expected to
/// reinitialize all elements which overwrites all entries anyway.

This is safe when:

The caller always initializes all used elements
Stale data in unused elements doesn't leak sensitive info

JsSlotArrayPool

Specialized static pool for JsSlot[] arrays used in environments.

internal static class JsSlotArrayPool
{
    private static readonly BucketedArrayPool<JsSlot> Pool = new();

    [MethodImpl(JsEngineConstants.Inlining)]
    public static JsSlot[] Rent(int minimumLength) => Pool.Rent(minimumLength);

    [MethodImpl(JsEngineConstants.Inlining)]
    public static void Return(JsSlot[]? array) => Pool.Return(array);
}

Usage Pattern

// In JsEnvironment
public void InitializeSlots(int count)
{
    _slots = JsSlotArrayPool.Rent(count);
    for (var i = 0; i < count; i++)
        _slots[i] = new JsSlot(JsValue.Undefined, false);
}

public void Dispose()
{
    JsSlotArrayPool.Return(_slots);
    _slots = null;
}

IteratorResultObjectPool

Pool for iterator result objects ({ value, done }).

The IsCaptured Pattern

flowchart TD
    Create((Create Result)) --> Pool["Rent from Pool"]
    Pool --> Use["Use in Iterator"]
    Use --> Check{Captured?}
    Check -->|No| Return["Return to Pool"]
    Check -->|Yes| Keep["Keep Alive\n(GC'd later)"]

Implementation

internal static class IteratorResultObjectPool
{
    private static readonly ObjectPool<IteratorResultObject> Pool = new(64,
        static () => new IteratorResultObject(JsValue.Undefined, false));

    public static IteratorResultObject Rent(JsValue value, bool done)
    {
        var result = Pool.Rent();
        result.Reset(value, done);
        return result;
    }

    public static void Return(IteratorResultObject result)
    {
        // Don't return captured objects or the shared singleton
        if (result.IsCaptured || ReferenceEquals(result, IteratorResultObject.DoneUndefined))
            return;

        // Clear value reference to avoid holding onto objects
        result.Reset(JsValue.Undefined, false);
        Pool.Return(result);
    }
}

IsCaptured Flag

When a pooled object escapes the fast path (e.g., user stores it in a variable), it's marked as "captured":

public sealed class IteratorResultObject
{
    public JsValue Value { get; private set; }
    public bool Done { get; private set; }
    public bool IsCaptured { get; set; }

    public void Reset(JsValue value, bool done)
    {
        Value = value;
        Done = done;
        IsCaptured = false;  // Reset on reuse
    }
}

When to set IsCaptured:

Object is stored in a user-accessible variable
Object is returned from a user-visible function
Object escapes the iterator hot path

Why it matters:

Captured objects can't be safely returned to pool
User code might still hold references
Returning captured objects causes use-after-return bugs

Pooling Kill Switch

For debugging, pooling can be completely disabled:

#if DISABLE_POOLING
public T Rent(ILogger? logger = null)
{
    // Kill switch: always create fresh instance
    var freshItem = factory();
    if (IsRentable)
        ((IRentable)freshItem).OnRent(logger);
    return freshItem;
}

public void Return(T item, ILogger? logger = null)
{
    // Kill switch: do nothing, let GC handle it
    _ = item;
    _ = logger;
}
#endif

Enable with: dotnet build -p:DefineConstants=DISABLE_POOLING

Pool Debug (DEBUG only)

Zero-overhead leak detection in DEBUG builds.

internal static class PoolDebug
{
#if DEBUG
    private static readonly ConcurrentDictionary<object, StackTrace> _leased = new();

    public static void MarkLeased(object item)
    {
        _leased[item] = new StackTrace(skipFrames: 2, fNeedFileInfo: true);
    }

    public static void MarkReturned(object item)
    {
        _leased.TryRemove(item, out _);
    }

    public static void AssertNoLeaks()
    {
        if (_leased.Count > 0)
        {
            var sb = new StringBuilder();
            sb.AppendLine($"Pool leak detected: {_leased.Count} items not returned");
            foreach (var (item, stack) in _leased)
            {
                sb.AppendLine($"  {item.GetType().Name}:");
                sb.AppendLine($"    Rented at: {stack}");
            }
            throw new InvalidOperationException(sb.ToString());
        }
    }
#else
    [MethodImpl(MethodImplOptions.AggressiveInlining)]
    public static void MarkLeased(object item) { }

    [MethodImpl(MethodImplOptions.AggressiveInlining)]
    public static void MarkReturned(object item) { }

    public static void AssertNoLeaks() { }
#endif
}

Usage in Tests

[TearDown]
public void TearDown()
{
    PoolDebug.AssertNoLeaks();
}

Pool Guard (Runtime)

Runtime leak detection controlled by environment variable.

internal static class PoolGuard
{
    private static readonly bool Enabled = 
        Environment.GetEnvironmentVariable("JSENGINE_DEBUG_POOL_GUARDS") == "true";

    private static readonly ConcurrentDictionary<object, string> _active = new();

    public static void OnRent(object item, string context)
    {
        if (!Enabled) return;
        _active[item] = context;
    }

    public static void OnReturn(object item)
    {
        if (!Enabled) return;
        if (!_active.TryRemove(item, out _))
        {
            throw new InvalidOperationException(
                $"Double-return detected for {item.GetType().Name}");
        }
    }
}

Enable with: JSENGINE_DEBUG_POOL_GUARDS=true

Pre-heating

Pools are pre-heated at construction to avoid allocation spikes:

private const int PoolSize = 100;  // Pre-heat size per bucket

private readonly ObjectPool<T[]> _pool8 = new(PoolSize, static () => new T[8]);

This creates 100 arrays per bucket size upfront, reducing first-use allocation latency.

Common Pooled Types

Type	Pool Location	IsCaptured	Notes
`JsEnvironment`	Static pool	N/A	Scope/execution context
`JsSlot[]`	JsSlotArrayPool	N/A	Variable storage
`IteratorResultObject`	IteratorResultObjectPool	Yes	`{ value, done }`
`IteratorDriverState`	Static pool	N/A	for-of loop state
`ForInDriverState`	Static pool	N/A	for-in loop state
`List<T>` builders	Various	N/A	Temporary collections

Performance Considerations

Inlining

Pool operations are marked for aggressive inlining:

[MethodImpl(JsEngineConstants.Inlining)]
public static JsSlot[] Rent(int minimumLength) => Pool.Rent(minimumLength);

Contention

Lock-free design minimizes contention, but under high load:

Multiple threads may scan the same slots
CAS failures cause retries
Pool may temporarily appear full

Memory Trade-off

Memory Usage = PoolSize * BucketSizes * ElementSize

For JsSlot[]:
  100 * (8 + 16 + 32 + 64 + 128 + 256 + 512 + 1024) * sizeof(JsSlot)
= 100 * 2040 * ~16 bytes
= ~3.2 MB per BucketedArrayPool instance

This is acceptable for the allocation savings during execution.

Best Practices

Do

Always return pooled objects when done
Reset state in OnRent/Reset methods
Clear references in OnReturn to avoid holding objects
Set IsCaptured = true when objects escape hot paths
Use PoolDebug.AssertNoLeaks() in test teardown

Don't

Store pooled objects in long-lived collections
Return objects that might still be referenced
Assume pool is empty (always handle allocation fallback)
Rely on stale data in pooled objects

Uh oh!

Pooling Deep Dive

Pooling Deep Dive

Overview

Key Files

ObjectPool

Implementation

Lock-Free Algorithm

IRentable Interface

BucketedArrayPool

Bucket Sizes

Implementation

No Array.Clear()

JsSlotArrayPool

Usage Pattern

IteratorResultObjectPool

The IsCaptured Pattern

Implementation

IsCaptured Flag

Pooling Kill Switch

Pool Debug (DEBUG only)

Usage in Tests

Pool Guard (Runtime)

Pre-heating

Common Pooled Types

Performance Considerations

Inlining

Contention

Memory Trade-off

Best Practices

Do

Don't

See Also

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!