Pre-filing checklist

• I searched existing issues and didn't find a duplicate

Component(s)

Rust OTAP dataflow (rust/otap-dataflow/)

Is your feature request related to a problem?

This is the planned Phase 2 follow-up to #2514 (implemented in #2529). Phase 1 delivered the ConfigProvider trait abstraction along with file: and env: providers. The original issue explicitly deferred these additional providers to a follow-up once the abstraction was in place.

Proposed Solution

Extend the provider abstraction introduced in #2529 with two additional providers and improved fallback behaviour:

New providers

yaml: provider

Follows the OTel Collector yaml: convention — the content after yaml: is treated as a literal YAML string, with :: used as a key path separator for inline overrides:

# Full inline YAML config
df_engine --config=yaml:version::otel_dataflow/v1

# Used to override a specific field from a base config (once merge support lands)
df_engine --config=file:/etc/base.yaml --config=yaml:engine::{}

http: provider

Performs an unauthenticated HTTP GET to fetch the config content. The response body is treated as YAML (or JSON if the Content-Type is application/json). A configurable timeout should be applied (default: 30s). Standard HTTP redirects are followed.

df_engine --config=http://config.internal:8080/pipeline.yaml

https:, authentication support (Bearer token, mTLS), and multi-config merge are intentionally deferred to Phase 3.

Acceptance Criteria

• --config=yaml:<inline-yaml> parses inline YAML using :: key path separator (OTel Collector convention)
• --config=http://... fetches config via unauthenticated HTTP GET with 30s default timeout
• HTTP: follows redirects, detects JSON vs YAML from Content-Type
• Helpful error message listing searched paths when all fallbacks fail
• Unit tests for both new providers
• parse_scheme updated to handle http:// (note :// vs : separator)

Alternatives Considered

• https: support was considered but deferred to Phase 3 alongside authentication (Bearer, mTLS) since the two concerns go together and add TLS backend complexity (cert validation, CA bundles, feature flags).
• Multi-config merge (--config specified multiple times with deep merge semantics) is also deferred to Phase 3.

Additional Context

• Phase 1 tracking issue: Feature: Add OTel Collector-compatible config URI providers (file:, env:) #2514
• Phase 1 implementation: feat(config): add URI providers for file: and env: config sources #2529
• OTel Collector yaml: provider docs: https://opentelemetry.io/docs/collector/configuration/#yaml-provider
• OTel Collector multiple config docs: https://opentelemetry.io/docs/collector/configuration/#multiple-config-files