Praxis's core differentiator: a full AI inference gateway built on the filter-first, body-first architecture, with per-provider control over binary size and attack surface.
Scope Boundary
This epic covers the core AI inference framework: provider abstraction, filter registry, streaming inspection, circuit breaking, pipeline validation, and GenAI observability. Related work tracked elsewhere:
Crate Architecture
AI inference filters live under filter/src/builtins/http/ai/. The current module structure is preferred over a separate praxis-ai crate unless binary size measurements demonstrate a need for crate-level feature gating. Per-provider Cargo feature flags at the workspace level provide equivalent control without the complexity of a separate crate.
Provider Abstraction Layer
Unified interface for OpenAI, Anthropic, Google, AWS Bedrock, and Azure OpenAI. Each provider is behind its own Cargo feature flag. The abstraction layer provides:
- Common request/response envelope types for translation
- Per-provider serialization/deserialization
- Streaming response normalization across SSE format differences
- Passthrough mode for provider-specific features
Streaming Inspection
SSE event parsing for inference responses:
- Per-event filter hooks for inspection and modification
- Streaming token extraction from delta content fields
- Cross-chunk event reassembly
- Backpressure-aware buffering
GenAI Observability
OpenTelemetry GenAI semantic conventions for standardized AI observability:
gen_ai.client.* span attributes (system, request.model, response.model)- Token usage metrics (
gen_ai.client.token.usage)
- Operation duration metrics
- Streaming-aware span lifecycle
Foundation
Body Access
Advanced
Praxis's core differentiator: a full AI inference gateway built on the filter-first, body-first architecture, with per-provider control over binary size and attack surface.
Scope Boundary
This epic covers the core AI inference framework: provider abstraction, filter registry, streaming inspection, circuit breaking, pipeline validation, and GenAI observability. Related work tracked elsewhere:
Crate Architecture
AI inference filters live under
filter/src/builtins/http/ai/. The current module structure is preferred over a separatepraxis-aicrate unless binary size measurements demonstrate a need for crate-level feature gating. Per-provider Cargo feature flags at the workspace level provide equivalent control without the complexity of a separate crate.Provider Abstraction Layer
Unified interface for OpenAI, Anthropic, Google, AWS Bedrock, and Azure OpenAI. Each provider is behind its own Cargo feature flag. The abstraction layer provides:
Streaming Inspection
SSE event parsing for inference responses:
GenAI Observability
OpenTelemetry GenAI semantic conventions for standardized AI observability:
gen_ai.client.*span attributes (system, request.model, response.model)gen_ai.client.token.usage)Foundation
Body Access
Advanced