[PERFORMANCE]: Plugin Framework Memory Optimization: Copy-on-Write for Context State

[crivetimihai]

Summary

The plugin framework in mcpgateway/plugins/framework/manager.py performs deep copies of GlobalContext.state and GlobalContext.metadata dictionaries for each plugin in the execution chain. For plugins that store large objects in state/metadata and chains with many plugins, this creates memory overhead.

Important: The payload itself is NOT deep copied - only the context's state and metadata dicts are copied.

Impact

• Memory: Each plugin execution duplicates the state and metadata dicts
• GC Pressure: Many short-lived dict copies increase garbage collection overhead
• Latency: Deep copy overhead scales with state/metadata size × number of plugins
• Scalability: With 10 plugins and 100KB state = 1MB temporary allocations per request

Memory Analysis

State/Metadata Size	Plugins	Current Memory Overhead	Optimized Overhead
1KB	5	5KB	~0KB (if read-only)
10KB	10	100KB	~0KB (if read-only)
100KB	10	1MB	~0KB (if read-only)
100KB	20	2MB	~0KB (if read-only)

Affected Code

File: mcpgateway/plugins/framework/manager.py

The deepcopy calls are inside the for hook_ref in hook_refs: loop (lines 148-165), meaning they execute for EVERY plugin:

tmp_global_context = GlobalContext(
    request_id=global_context.request_id,
    user=global_context.user,
    tenant_id=global_context.tenant_id,
    server_id=global_context.server_id,
    state={} if not global_context.state else deepcopy(global_context.state),      # Deep copy per plugin!
    metadata={} if not global_context.metadata else deepcopy(global_context.metadata),  # Deep copy per plugin!
)

Root Cause

The deep copy approach provides:

1. Plugin Isolation: Plugins shouldn't see each other's state modifications until after execution
2. Rollback Safety: If a plugin fails, the original state is preserved
3. State Aggregation: After plugin execution, state changes are merged back

However, most plugins only read the state/metadata or make small modifications.

Proposed Solution

Implement a Copy-on-Write (COW) Dict Wrapper that only materializes a copy when modification is detected:

class CopyOnWriteDict:
    """Dict wrapper that defers deep copy until modification."""
    
    def __init__(self, original: Dict[str, Any]):
        self._original = original
        self._local_changes: Dict[str, Any] = {}
        self._deleted_keys: set = set()
        self._modified = False

    def __getitem__(self, key: str) -> Any:
        if key in self._deleted_keys:
            raise KeyError(key)
        if key in self._local_changes:
            return self._local_changes[key]
        return self._original[key]

    def __setitem__(self, key: str, value: Any) -> None:
        self._local_changes[key] = value
        self._modified = True
    
    # ... additional dict methods ...
    
    def to_dict(self) -> Dict[str, Any]:
        if not self._modified:
            return self._original  # No copy needed!
        result = dict(self._original)
        for key in self._deleted_keys:
            result.pop(key, None)
        result.update(self._local_changes)
        return result

Implementation Tasks

• Profile current memory usage with plugins that have large state
• Identify which built-in plugins actually modify state/metadata
• Create CopyOnWriteDict class in mcpgateway/plugins/framework/cow_dict.py
• Update PluginExecutor.execute() to use COW wrappers (lines 158-165)
• Update state merge logic (lines 227-229) to use to_dict()
• Add unit tests for COW dict behavior
• Add memory profiling tests
• Verify plugin isolation still works
• Benchmark before/after

Acceptance Criteria

• Memory usage reduced for multi-plugin chains with read-only plugins
• No regression in plugin isolation
• State/metadata modifications still work correctly
• All existing plugin tests pass
• New tests verify COW behavior
• Passes make verify

References

• Detailed analysis: todo/performance/plugin-framework-memory-optimization.md
• Python Copy-on-Write Patterns
• collections.ChainMap (alternative approach)

[terylt]

@crivetimihai I think this is a good extension to the plugin framework. I thought about it a bit during the initial revision of the plugin framework but didn't go too deeply into it as we didn't have a good sense of performance and state at the time.

One thing we could also do is think a little bit more about metadata and whether we want to make parts of it read-only. For example, part of the metadata comes from the metadata about tools, prompts, and resources.. This kind of stuff we probably don't want or need to update. and can just ignore edits to it. I talk about this a bit in the spec, but it's not fleshed out as much as it could be.

[araujof]

+1 to this proposal.

A common approach is to wrap the original dictionary and only store modifications in the new wrapper. This is similar to how collections.ChainMap works in Python 3, which handles multiple dictionaries as a single view.

https://docs.python.org/3/library/collections.html#collections.ChainMap

You can implement a form of "copy-on-write" behavior for dictionary operations using collections.ChainMap by wrapping an existing dictionary and performing all new writes to a new, empty dictionary placed at the front of the chain.

e.g.: cow_map = collections.ChainMap({}, context)

[araujof]

@terylt Please take a look at this draft PR when you get a chance: #1636

Would be cool to include some performance profiling for deepcopy vs copyonwrite so we can measure the impact.