Sampling implementation

[IsraelShok]

Hi, there is an implementation for sampling logic?
I can see code that related to elicitation, but cant see for sampling.

Thanks

[crivetimihai]

Hi @IsraelShok, thanks for the question. Could you share what client/SDK path you are testing against and the exact sampling behavior you expect versus observed so I can align with the current protocol support and document any gaps.

[IsraelShok]

Hi @crivetimihai ,

We want to use GitHub Copilot (VS Code) as the client.
As we are checking in this days, there is no option to activate sampling with stateless mode, so we are trying to find a way to solve it.

From your experience, is it an option to activate mcp's in stateful mode? with thousands of users.

Anyway, I can't see how sampling is possible in the current version of Forge, or am I mistaken?

Thanks!

[IsraelShok]

Hi @crivetimihai , any update?
Thanks!

[crivetimihai]

Hi @IsraelShok, to answer directly:

Sampling is partially implemented. The framework is in place but the actual LLM invocation is stubbed out:

• What exists: A SamplingHandler class with a POST /sampling/createMessage endpoint, model selection based on preferences (cost/speed/intelligence), and message validation — all matching the MCP spec's sampling/createMessage request type.

• What's missing: The handler currently returns mock responses instead of calling an actual LLM. The code has TODO markers for implementing real model sampling and context gathering.

• Elicitation is more complete: The ElicitationService is fully implemented with real timeout handling, cleanup, and the three-action model (accept/decline/cancel) per the MCP spec.

In practice, sampling (where an MCP server requests the client to generate an LLM completion) is a less commonly used MCP feature compared to tools and resources, and many MCP deployments don't require it. The gateway prioritizes the server->client elicitation flow which is the more common interaction pattern.

If you have a specific use case for sampling, please share the client/SDK path and the expected behavior — that would help prioritize making the implementation fully functional.

[crivetimihai]

Hi @IsraelShok, apologies for the delay on this.

Current state of sampling — confirmed by code review:

The SamplingHandler class exists with a POST /sampling/createMessage endpoint, model selection, and message validation — all matching the MCP spec. However, the actual LLM invocation is stubbed with mock responses. The code has explicit TODO: Implement actual model sampling - currently returns mock response markers.

Aspect	Status
Endpoint (POST /sampling/createMessage)	Implemented
Message validation	Implemented
Model selection (cost/speed/intelligence)	Implemented
Actual LLM invocation	Stubbed (mock)
Context gathering	Stubbed (no-op)

Regarding your specific questions:

1. Stateful mode with thousands of users: ContextForge supports stateful sessions (USE_STATEFUL_SESSIONS=true) with Redis-backed session affinity for multi-worker deployments. However, the sampling handler doesn't maintain state across requests — each createMessage call is independent. For scale, stateless mode with Redis caching is recommended.

2. GitHub Copilot (VS Code): ContextForge integrates with Copilot via SSE or streamable HTTP. However, GitHub Copilot does not currently support the MCP sampling capability — it calls tools, reads resources, and uses prompts, but doesn't expose the sampling/createMessage callback that servers need. This is a client-side limitation, not a gateway one.

3. When will real sampling ship?: This is tracked in [FEATURE][PROTOCOL]: Implement missing MCP protocol methods #758 (Implement missing MCP protocol methods). Sampling is not prioritized for v1.0 because most MCP clients (Claude Desktop, Copilot, Cursor) don't use sampling yet. If this is critical for your workflow, please comment on [FEATURE][PROTOCOL]: Implement missing MCP protocol methods #758 with your use case so we can prioritize accordingly.

Alternative: If you need LLM completions as part of your MCP workflow, consider using a tool that calls an LLM API directly rather than relying on the sampling protocol.