fix(serve): resource governor for multi-instance system freezes

[rajkumarsakthivel]

Summary

Fixes #139. Multiple cce serve processes (one per project per AI session) exhaust system memory and cause 10–20 minute desktop freezes on Linux. The reported case had 68 CCE processes with 2,179 threads consuming 5.4 GiB RSS, triggering kernel OOM and PSI pressure spikes above 85%.

This PR adds a resource governor (resource_governor.py) with four mechanisms:

• ONNX Runtime thread caps: each cce serve process defaults to 2 intra-op threads instead of cpu_count. Override via CCE_ORT_THREADS env var or serve.max_ort_threads config key. Also sets TOKENIZERS_PARALLELISM=false to prevent the tokenizers library from spawning additional threads.

• Per-project index lock: file-lock (fcntl.flock) so when multiple AI sessions spawn duplicate cce serve processes for the same project, only one indexes at a time. Others skip and retry on the next file-change event. No-op on Windows (advisory lock not available).

• Linux PSI memory-pressure detection: reads /proc/pressure/memory and defers re-indexing when some avg10 exceeds 25%. This prevents CCE from contributing to the cascading page-cache thrashing described in the issue. No-op on non-Linux platforms.

• Idle auto-shutdown: cce serve exits after 30 minutes of MCP inactivity (no call_tool invocations). This prevents zombie processes from accumulating when AI sessions disconnect or go idle. Configurable via CCE_IDLE_TIMEOUT_MINUTES env var, serve.idle_timeout_minutes config key, or set to 0 to disable.

New config keys

serve:
  idle_timeout_minutes: 30   # 0 = disabled
  max_ort_threads: 2         # 0 = use ONNX default

Impact on the reported scenario

With the defaults from this PR, the 68-process scenario from the issue would see:

• Thread count reduced from ~2,179 to ~340 (68 × ~5 threads each vs. 68 × ~32)
• Idle processes auto-shutdown after 30 minutes, reducing the steady-state count
• Concurrent indexing serialized per-project, preventing I/O amplification
• Memory-pressure detection pausing heavy work before the system enters OOM territory

[Copilot]

Pull request overview

Adds a “resource governor” to cce serve to reduce multi-instance host overload (threads, indexing contention, and idle zombie processes), addressing issue #139’s reported Linux desktop freezes under heavy multi-process usage.

Changes:

• Introduces resource_governor.py with ONNX/thread caps, per-project indexing lock, Linux PSI memory-pressure backoff, and idle auto-shutdown utilities.
• Wires the governor into cce serve startup/reindex flow (thread caps, index lock, PSI backoff, and an idle watchdog).
• Adds new serve.* config keys (idle_timeout_minutes, max_ort_threads) and maps them into Config.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 5 comments.

File	Description
src/context_engine/resource_governor.py	New governor utilities (thread caps, lock, PSI check, idle tracking).
src/context_engine/integration/mcp_server.py	Tracks MCP tool activity to support idle shutdown.
src/context_engine/config.py	Adds and maps new serve.* configuration keys.
src/context_engine/cli.py	Applies governor behavior in cce serve (caps, lock/backoff, idle watchdog).

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (2)

src/context_engine/resource_governor.py:81

• When CCE_ORT_THREADS (or serve.max_ort_threads) is set to 0/negative to disable ORT caps, this returns early and never sets TOKENIZERS_PARALLELISM=false. That contradicts the PR’s stated goal of preventing tokenizers from spawning extra threads, and can reintroduce high thread counts even when ORT caps are disabled.

    if n <= 0:
        return 0  # 0 = "let ONNX pick"

src/context_engine/cli.py:3525

• If the per-project index lock is held by another process, this worker drops the reindex request entirely. If the other process misses the same filesystem event (watcher queue overflow, transient watcher failure, etc.), the project index can remain stale until a later change happens to the same file. Consider re-queuing with a small backoff (similar to the PSI-pressure path) so every change is eventually indexed once the lock becomes available.

                if not index_lock.try_acquire():
                    _log.debug(
                        "Another process is indexing this project; "
                        "skipping %s", rel,
                    )