Motivation
The langchain-tests package provides the official conformance suite for chat models (ChatModelUnitTests / ChatModelIntegrationTests, ~31 test methods covering invoke/stream/batch + async, tool calling, structured output across Pydantic/TypedDict/JSON Schema, usage metadata accumulation, multimodal inputs, and the v3 stream-events protocol). Capability flags (has_tool_choice, supports_image_inputs, supports_json_mode, supported_usage_metadata_details, …) make feature gaps visible and prevent regressions.
See: https://docs.langchain.com/oss/python/contributing/standard-tests-langchain
Current state
libs/oracledb already runs standard tests for embeddings (tests/unit_tests/embeddings/test_standard.py, tests/integration_tests/embeddings/test_standard.py) and the vector store (tests/integration_tests/vectorstores/test_standard.py).
libs/oci declares langchain-tests in pyproject.toml but has no ChatModelUnitTests / ChatModelIntegrationTests subclasses for any of ChatOCIGenAI, ChatOCIModelDeployment, or ChatOCIOpenAI.
Proposed scope
- Add standard unit + integration test classes for
ChatOCIGenAI (one per provider family where behavior differs: Cohere, Meta/Generic, Gemini, OpenAI), ChatOCIOpenAI, and ChatOCIModelDeployment.
- Set capability flags honestly; each
False flag doubles as a tracked feature gap.
- Wire integration variants behind the existing integration-test env gating.
Acceptance criteria
- Standard unit tests run in CI for all chat model classes.
- Integration tests runnable against OCI GenAI with documented env vars.
- Capability-flag matrix documented in the package README.
Motivation
The
langchain-testspackage provides the official conformance suite for chat models (ChatModelUnitTests/ChatModelIntegrationTests, ~31 test methods covering invoke/stream/batch + async, tool calling, structured output across Pydantic/TypedDict/JSON Schema, usage metadata accumulation, multimodal inputs, and the v3 stream-events protocol). Capability flags (has_tool_choice,supports_image_inputs,supports_json_mode,supported_usage_metadata_details, …) make feature gaps visible and prevent regressions.See: https://docs.langchain.com/oss/python/contributing/standard-tests-langchain
Current state
libs/oracledbalready runs standard tests for embeddings (tests/unit_tests/embeddings/test_standard.py,tests/integration_tests/embeddings/test_standard.py) and the vector store (tests/integration_tests/vectorstores/test_standard.py).libs/ocideclareslangchain-testsinpyproject.tomlbut has noChatModelUnitTests/ChatModelIntegrationTestssubclasses for any ofChatOCIGenAI,ChatOCIModelDeployment, orChatOCIOpenAI.Proposed scope
ChatOCIGenAI(one per provider family where behavior differs: Cohere, Meta/Generic, Gemini, OpenAI),ChatOCIOpenAI, andChatOCIModelDeployment.Falseflag doubles as a tracked feature gap.Acceptance criteria