Skip to content

[PERFORMANCE]: Implement Lazy Service Initialization #1681

@crivetimihai

Description

@crivetimihai

Implement lazy service initialization for 50-80% faster cold start times.

Problem

Location: mcpgateway/main.py lines 110-125

All services imported at module load time:

from mcpgateway.services.a2a_service import A2AAgentService
from mcpgateway.services.completion_service import CompletionService
from mcpgateway.services.export_service import ExportService
from mcpgateway.services.gateway_service import GatewayService
from mcpgateway.services.import_service import ImportService
from mcpgateway.services.prompt_service import PromptService
from mcpgateway.services.resource_service import ResourceService
from mcpgateway.services.server_service import ServerService
from mcpgateway.services.tag_service import TagService
from mcpgateway.services.tool_service import ToolService
# ... more services and error classes

Current Startup Timeline

Phase Time Notes
Python import 200ms All dependencies
Database check 50-500ms Varies by DB
Service init 100-300ms 10+ services
Redis check 50-200ms If enabled
Plugin load 50-100ms Plugin manifests
Total 450-1300ms Cold start

Proposed Solution

Lazy Service Singleton Pattern

from functools import lru_cache

@lru_cache
def get_tool_service():
    from mcpgateway.services.tool_service import ToolService
    return ToolService()

# In routes:
@router.get("/tools")
async def list_tools(
    tool_service: ToolService = Depends(get_tool_service)
):
    return await tool_service.list_tools()

Feature-Based Service Loading

def get_a2a_service():
    if not settings.a2a_enabled:
        raise HTTPException(501, "A2A not enabled")
    from mcpgateway.services.a2a_service import A2AService
    return ServiceRegistry.get(A2AService)

Performance Targets

Metric Current Target
Cold start (minimal) 450ms 150ms
Cold start (full) 1300ms 400ms
First request latency +0ms +100ms (one-time)

Acceptance Criteria

  • Services instantiated on first use, not import
  • Database check deferred to first database operation
  • Redis check deferred to first cache operation
  • Disabled features don't load their services
  • Startup time reduced by 50%+
  • All existing tests pass
  • Passes make verify

Metadata

Metadata

Assignees

No one assigned

    Labels

    SHOULDP2: Important but not vital; high-value items that are not crucial for the immediate releasedatabaseenhancementNew feature or requestperformancePerformance related items
    No fields configured for Feature.

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions