Common utilities for serializing/deserializing and estimating size

[strawgate]

Enhancement

In a bunch of places in the code and tests we have snippets for serializing and deserializing common response types and in logging middleware and soon caching middleware we want to estimate the size of objects

We should create/consolidate utility functions for serializing/deserializing and sizing the common MCP and FastMCP response types

[strawgate]

/marvin can you identify all the places across code and tests we're doing this today?

[marvin-context-protocol]

Claude finished @strawgate's task —— View job

---

Analysis Complete: Serialization/Deserialization and Sizing Patterns

I've identified all the places across code and tests where serialization/deserialization and object sizing is currently being done. Here's my comprehensive analysis:

🔍 Serialization/Deserialization Patterns Found

Core Library Functions (Most Important)

These are the main utilities that handle MCP/FastMCP response types:

1. src/fastmcp/server/middleware/logging.py:16 - default_serializer()

   • Uses pydantic_core.to_json(data, fallback=str).decode()
   • This is the primary serializer for MCP objects

2. src/fastmcp/tools/tool.py:63 - Default tool serializer

   • Uses pydantic_core.to_json(data, fallback=str).decode()
   • Handles tool results

3. src/fastmcp/resources/resource.py:218 - Resource content serializer

   • Uses pydantic_core.to_json(result, fallback=str).decode()

4. src/fastmcp/prompts/prompt.py:365 - Prompt message serializer

   • Uses pydantic_core.to_json(msg, fallback=str).decode()

Inspection & Utilities (CLI/Debug)

5. src/fastmcp/utilities/inspect.py - Multiple uses for server inspection

   • Lines 125, 141, 161, 182, 261: Uses .model_dump() for annotations
   • Lines 385, 418: Uses pydantic_core.to_json(result, indent=2) for output

6. src/fastmcp/utilities/components.py:122 - Component comparison

   • Uses .model_dump() for equality checks

Client-Side Serialization

7. src/fastmcp/client/client.py:714 - Client argument serialization
   • Uses pydantic_core.to_json(value).decode() for tool arguments

Configuration & Server

8. src/fastmcp/mcp_config.py - Configuration handling

   • Lines 281, 286, 328, 330: Uses .model_dump() and .model_dump_json()

9. src/fastmcp/server/server.py:240 - Server dependency injection

   • Uses json.dumps(dependencies) for template generation

📏 Object Sizing Patterns Found

Middleware Sizing (Key Areas)

1. src/fastmcp/server/middleware/logging.py:72-79 - Primary payload sizing
   • payload_length = len(payload) - Gets serialized payload size
   • payload_tokens = payload_length // 4 - Token estimation
   • len(payload) > self.max_payload_length - Payload truncation check
   • This is the main place where object sizing happens for logging

Rate Limiting Middleware

2. src/fastmcp/server/middleware/rate_limiting.py:86 - Request counting
   • len(self.requests) < self.max_requests

Collection Size Checks (Throughout Codebase)

Hundreds of instances using len() for:

• Tool/resource/prompt collection counting
• Parameter validation
• Loop boundaries and array processing
• Response validation in tests

🎯 Recommended Consolidation Areas

Based on this analysis, here are the common utilities that should be created:

1. Core Serialization Utils (src/fastmcp/utilities/serialization.py)

# Consolidate these patterns:
- pydantic_core.to_json(data, fallback=str).decode()  # Used 8+ times
- json.dumps() with consistent parameters
- .model_dump() for Pydantic models

2. Object Sizing Utils (src/fastmcp/utilities/sizing.py)

# Consolidate these patterns:  
- Payload size calculation (len of serialized)
- Token estimation (size // 4)
- Size-based truncation logic
- Memory usage estimation for caching middleware

3. Response Type Handlers (src/fastmcp/utilities/response_utils.py)

# Specialized handlers for:
- Tool results (CallToolResult)
- Resource content (ReadResourceResult) 
- Prompt messages (GetPromptResult)
- Server inspection data

📍 Specific Files Needing Consolidation

High Priority:

• src/fastmcp/server/middleware/logging.py - Main sizing logic
• src/fastmcp/tools/tool.py - Tool serialization
• src/fastmcp/resources/resource.py - Resource serialization
• src/fastmcp/prompts/prompt.py - Prompt serialization

Medium Priority:

• src/fastmcp/utilities/inspect.py - Inspection serialization
• src/fastmcp/client/client.py - Client-side serialization
• OpenAPI integration files - Response processing

The logging middleware is the most critical area since it's where the caching middleware will also need similar functionality for object sizing and serialization.

Todo List:

• Search for serialization patterns (json.dumps, json.loads, .model_dump(), etc.)
• Search for object sizing patterns (len(), sys.getsizeof(), etc.)
• Examine middleware files for sizing/serialization
• Review test files for serialization patterns
• Analyze findings and categorize by location and purpose
• Create comprehensive report