Skip to content

test(oci): adopt langchain-tests standard ChatModel conformance suite for ChatOCIGenAI, ChatOCIOpenAI, ChatOCIModelDeployment #261

Description

@fede-kamel

Motivation

The langchain-tests package provides the official conformance suite for chat models (ChatModelUnitTests / ChatModelIntegrationTests, ~31 test methods covering invoke/stream/batch + async, tool calling, structured output across Pydantic/TypedDict/JSON Schema, usage metadata accumulation, multimodal inputs, and the v3 stream-events protocol). Capability flags (has_tool_choice, supports_image_inputs, supports_json_mode, supported_usage_metadata_details, …) make feature gaps visible and prevent regressions.

See: https://docs.langchain.com/oss/python/contributing/standard-tests-langchain

Current state

  • libs/oracledb already runs standard tests for embeddings (tests/unit_tests/embeddings/test_standard.py, tests/integration_tests/embeddings/test_standard.py) and the vector store (tests/integration_tests/vectorstores/test_standard.py).
  • libs/oci declares langchain-tests in pyproject.toml but has no ChatModelUnitTests / ChatModelIntegrationTests subclasses for any of ChatOCIGenAI, ChatOCIModelDeployment, or ChatOCIOpenAI.

Proposed scope

  • Add standard unit + integration test classes for ChatOCIGenAI (one per provider family where behavior differs: Cohere, Meta/Generic, Gemini, OpenAI), ChatOCIOpenAI, and ChatOCIModelDeployment.
  • Set capability flags honestly; each False flag doubles as a tracked feature gap.
  • Wire integration variants behind the existing integration-test env gating.

Acceptance criteria

  • Standard unit tests run in CI for all chat model classes.
  • Integration tests runnable against OCI GenAI with documented env vars.
  • Capability-flag matrix documented in the package README.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions