Skip to content

Commit b625bfb

Browse files
authored
Clarify live API conformance gap in Future Direction (#23)
## Summary - Replaces misleading "E2E SDK tests" bullet with accurate description: we have `api-conformance` tests that validate response format structure, but the real gap is tests that run against actual LLM APIs (gated behind API keys) to catch format drift as providers evolve ## Test plan - [x] Docs-only change, no code impact
2 parents 47dfd64 + b169bad commit b625bfb

1 file changed

Lines changed: 1 addition & 1 deletion

File tree

README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -631,7 +631,7 @@ Areas where llmock could grow, and explicit non-goals for the current scope.
631631

632632
### Testing
633633

634-
- **E2E SDK tests**: The test suite uses raw HTTP and WebSocket frames, not real OpenAI/Anthropic/Gemini client SDKs.
634+
- **Live API conformance**: The `api-conformance` tests validate response format structure but do not run against real LLM APIs. A subset of tests that hit actual OpenAI/Anthropic/Gemini endpoints (gated behind API keys) would catch format drift as providers evolve their APIs.
635635
- **Token counts**: Usage fields are always zero across all providers.
636636
- **Vision/image content**: Image content parts are not handled by any provider.
637637

0 commit comments

Comments
 (0)