Generalize OpenAI-compatible provider adapters

[bglusman]

Summary

• replace the Helicone-only gateway path with one shared OpenAI-compatible HTTP core plus small engine policy overlays
• add first-class backend type parsing/validation for litellm, portkey, tensorzero, future-agi, and openrouter alongside helicone, http, and mock
• move Helicone streaming parsing to a generic OpenAI-compatible streaming parser and wire LiteLLM smoke coverage through the named adapter
• split oversized gateway/validator tests so architecture ratchets stay green and update docs/ADRs around provider adapters vs future observability sinks

Verification

• cargo test -p calciforge proxy::gateway_tests
• cargo test -p calciforge proxy::routing::tests
• python3 scripts/check-boundary-surfaces.py && python3 scripts/check-scenarios.py && ruby scripts/check-architecture-ratchets.rb && git diff --check
• python3 scripts/model-gateway-helicone-smoke.py && python3 scripts/model-gateway-litellm-smoke.py
• cargo test -p calciforge
• push hook: fmt, clippy, workspace unit tests, loom tests

Adversarial Review Notes

• Architecture: Helicone is no longer a separate router implementation. It is now one GatewayType policy overlay on the shared OpenAI-compatible HTTP core. This avoids the next adapter copying Helicone plumbing.
• Test quality: the added tests assert behavior that would fail before this change: named non-Helicone engines parse/validate, LiteLLM uses the shared streaming path, and routing preserves engine metadata through the shared gateway construction.
• Residual risk: Portkey/TensorZero/Future-AGI/OpenRouter are registered as OpenAI-compatible adapter shapes but are not live-smoked here. The docs call out that practical engine-specific validation still needs user/provider confirmation.
• Residual risk: pure observability sinks are documented as the next axis but are not implemented in this PR.

[Copilot]

Pull request overview

This PR generalizes Calciforge’s model-gateway “Helicone-only” path into a shared OpenAI-compatible HTTP core, with small engine/policy overlays keyed by backend_type. It expands root/provider backend parsing + validation to include named OpenAI-compatible gateways (LiteLLM/Portkey/TensorZero/Future-AGI/OpenRouter), and updates tests/scripts/docs to match the new architecture.

Changes:

• Introduces new GatewayType variants + shared OpenAI-compatible header overlay logic, and routes Helicone/LiteLLM/etc through the same HTTP backend/gateway core.
• Adds SSE parsing support to the HTTP backend using a generalized OpenAI-compatible streaming parser; updates routing + doctor reporting accordingly.
• Splits oversized validator tests into multiple modules, updates boundary scripts + scenario definitions, and refreshes docs/ADRs around the provider-adapter boundary.

Reviewed changes

Copilot reviewed 25 out of 25 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
tests/scenarios/high-risk-scenarios.json	Renames the streaming-tools high-risk scenario to reflect generalized OpenAI-compatible streaming coverage.
tests/boundaries/integration-surfaces.json	Updates boundary surface inventory and automation commands to use openai_streaming tests and new validator test modules.
scripts/model-gateway-litellm-smoke.py	Switches LiteLLM smoke config to backend_type = "litellm" to exercise named engine parsing/metadata.
scripts/boundary-explore-long.sh	Updates long boundary exploration to run proxy::openai_streaming::tests.
scripts/boundary-aggression.sh	Updates boundary aggression suite to run proxy::openai_streaming::tests.
docs/roadmap/litellm-gateway-owned-provider.md	Updates LiteLLM recipe to use the named litellm engine overlay and shared core framing.
docs/model-gateway.md	Reframes gateway engines as OpenAI-compatible overlays, expands supported backends, and documents gateway-vs-observability separation.
docs/adr/0002-provider-adapter-boundary.md	Notes the shared OpenAI-compatible HTTP core and engine overlays as an architectural decision.
docs/adr/0001-model-gateway-and-agent-boundaries.md	Updates diagrams/text to reflect a generalized OpenAI-compatible engine set and shared core.
crates/calciforge/src/proxy/routing.rs	Generalizes provider gateway construction by parsing GatewayType and applying engine-specific header overlays.
crates/calciforge/src/proxy/openai_streaming.rs	Renames/repurposes streaming parsing and error messages to be OpenAI-compatible rather than Helicone-specific.
crates/calciforge/src/proxy/mod.rs	Exposes gateway module internally, adds openai_streaming, and switches root backend allowlist to GatewayType constants.
crates/calciforge/src/proxy/helicone_router.rs	Deleted: removes the dedicated Helicone router implementation in favor of shared HTTP core.
crates/calciforge/src/proxy/helicone_router_tests.rs	Deleted: removes Helicone-router-specific tests (replaced by shared-core tests).
crates/calciforge/src/proxy/gateway.rs	Expands GatewayType, adds OpenAI-compatible header overlays, and routes Helicone/etc through the shared HTTP gateway implementation.
crates/calciforge/src/proxy/gateway_tests.rs	New consolidated gateway tests covering parsing, overlays, retry behavior, and streaming parsing via shared core.
crates/calciforge/src/proxy/backend.rs	Removes Helicone backend type and adds SSE parsing support to the HTTP backend.
crates/calciforge/src/doctor.rs	Updates provider-boundary reporting to recognize named OpenAI-compatible provider adapters and improves messaging.
crates/calciforge/src/config/validator.rs	Expands proxy/provider backend validation to use GatewayType parsing and splits test modules out of this file.
crates/calciforge/src/config/validator_tests_1.rs	New: first chunk of validator behavioral tests (moved from validator.rs).
crates/calciforge/src/config/validator_tests_2.rs	New: second chunk of validator behavioral tests (moved from validator.rs).
crates/calciforge/src/config/validator_tests_3.rs	New: third chunk of validator behavioral tests (moved from validator.rs).
crates/calciforge/src/config/validator_test_support.rs	New shared fixture/helpers for validator behavioral tests.
crates/calciforge/src/config.rs	Updates config docs/comments to reflect the expanded root/provider backend types and shared OpenAI-compatible core.
crates/calciforge/Cargo.toml	Removes default helicone feature and updates retry dependency comment to be engine-agnostic.

Comments suppressed due to low confidence (1)

crates/calciforge/src/proxy/backend.rs:381

• HttpBackend applies headers twice (as ClientBuilder::default_headers in new() and again per-request in send_chat_completion_request). This can duplicate headers and also allows user-configured headers (e.g., Authorization) to override/append after the API-key header, which can break auth and is risky. Prefer applying configured headers only once and ensure the computed Authorization header cannot be overridden by user headers (e.g., strip/ignore authorization from configured headers or apply Authorization last with replacement semantics).

        // Add default headers if provided
        if let Some(headers) = &headers {
            let mut header_map = reqwest::header::HeaderMap::new();
            for (key, value) in headers {
                if let Ok(header_name) = reqwest::header::HeaderName::from_bytes(key.as_bytes())
                    && let Ok(header_value) = reqwest::header::HeaderValue::from_str(value)
                {
                    header_map.insert(header_name, header_value);
                }
            }
            client_builder = client_builder.default_headers(header_map);
        }

        let client = client_builder.build().expect("Failed to build HTTP client");

        Self {
            client,
            base_url,
            api_key,
            timeout_seconds,
            headers: headers.unwrap_or_default(),
        }
    }

    async fn send_chat_completion_request(
        &self,
        request: crate::proxy::openai::ChatCompletionRequest,
    ) -> Result<ChatCompletionResponse, BackendError> {
        let url = format!("{}/chat/completions", self.base_url);

        let model = request.model.clone();
        let mut request_body = serde_json::to_value(&request).map_err(|e| {
            BackendError::InvalidResponse(format!("Failed to serialize request: {e}"))
        })?;
        apply_kimi_compat(&self.base_url, &model, &mut request_body);

        let mut request_builder = self
            .client
            .post(&url)
            .header("Content-Type", "application/json");

        if !self.api_key.is_empty() {
            request_builder =
                request_builder.header("Authorization", format!("Bearer {}", self.api_key));
        }

        for (key, value) in &self.headers {
            request_builder = request_builder.header(key, value);
        }