feat: Add S3 remote provider configs to LLS midstream container.

[Elbehery]

This PR adds S3 remote provider configs to LLS midstream container.

Signed-off-by: Mustafa Elbehery melbeher@redhat.com

Closes https://issues.redhat.com/browse/RHAIENG-2131

Summary by CodeRabbit

• New Features

  • Added an optional S3-backed remote file provider with configurable bucket, region, credentials, endpoint, and auto-create bucket behavior.
  • S3 support is disabled by default and must be explicitly enabled via configuration.

• Documentation

  • Docs updated to explain S3 opt-in and how to configure the provider and metadata storage (SQLite).

_{✏️ Tip: You can customize this high-level summary in your review settings.}

[coderabbitai]

Walkthrough

Adds optional S3 support for the files API: distribution/run.yaml gains a guarded remote::s3 provider block (activated via ENABLE_S3) with S3 config fields and a metadata_store pointing to sql_files (table files_metadata); distribution/README.md adds a table row indicating S3 is disabled by default and requires ENABLE_S3.

Changes

Cohort / File(s)	Summary
S3 Provider Documentation distribution/README.md	Adds a table row indicating remote::s3 support for the files API is not enabled by default and requires the ENABLE_S3 environment variable.
S3 Provider Configuration distribution/run.yaml	Adds an optional providers/files entry for remote::s3 guarded by ENABLE_S3 (uses provider_id ${env.ENABLE_S3:+s3}). Introduces fields: bucket_name, region, aws_access_key_id, aws_secret_access_key, endpoint_url, auto_create_bucket, and a metadata_store using sql_files with table_name: files_metadata. Block is additive and only active when ENABLE_S3 is set.

Sequence Diagram(s)

sequenceDiagram
  participant Env as Runtime Env
  participant Runner as distribution/run.yaml
  participant FilesSvc as Files API
  participant S3 as remote::s3
  participant DB as sql_files (files_metadata)

  Env->>Runner: READ ENABLE_S3
  alt ENABLE_S3 set
    Runner->>FilesSvc: register provider_id "s3" (remote::s3) with config
    FilesSvc->>S3: authenticate (aws_access_key_id / aws_secret_access_key) and set region/endpoint
    FilesSvc->>S3: ensure bucket (auto_create_bucket?) and perform object operations
    S3-->>FilesSvc: object store/retrieve responses
    FilesSvc->>DB: persist metadata to `sql_files.table_name=files_metadata`
  else ENABLE_S3 not set
    Runner->>FilesSvc: do not configure remote::s3
    FilesSvc-->>FilesSvc: use existing/local file provider and metadata store
  end

Loading

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

• Verify ENABLE_S3 gating expression and exact env var usage (provider_id ${env.ENABLE_S3:+s3}).
• Check YAML indentation and field names (bucket_name, region, aws_access_key_id, aws_secret_access_key, endpoint_url, auto_create_bucket).
• Confirm metadata_store uses sql_files with table_name: files_metadata and that migrations/permissions exist.

Poem

🐇 I hopped a flag and found a key,

ENABLE_S3 beneath a tree.
Buckets hum and records hide,
Carrot crumbs of metadata inside.
A little hop, a cloud-bound leap. 🥕✨

Pre-merge checks and finishing touches

✅ Passed checks (3 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately describes the main change: adding S3 remote provider configurations to the LLS midstream container, which aligns with modifications in both the README and run.yaml files.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✨ Finishing touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch RHAIENG-2131

---

📜 Recent review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 9cc0595 and 5bfe9f7.

📒 Files selected for processing (2)

• distribution/README.md (1 hunks)
• distribution/run.yaml (1 hunks)

🚧 Files skipped from review as they are similar to previous changes (2)

• distribution/README.md
• distribution/run.yaml

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (2)

• GitHub Check: build-test-push (linux/amd64)
• GitHub Check: Summary

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 1

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between c20784c and 7b00db4.

📒 Files selected for processing (2)

• distribution/README.md (1 hunks)
• distribution/run.yaml (1 hunks)

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (2)

• GitHub Check: build-test-push (linux/amd64)
• GitHub Check: Summary

🔇 Additional comments (1)

distribution/README.md (1)

20-20: Verify README reflects generated output from script.

Line 1 indicates this file is auto-generated by scripts/gen_distro_doc.py. Confirm that the S3 entry at line 20 resulted from running the generation script rather than manual editing, ensuring consistency with distribution/run.yaml.

[Elbehery]

@nathan-weinberg ptal

[nathan-weinberg]

@skamenan7 PTAL

[Elbehery]

@nathan-weinberg @derekhiggins ptal, rebased and updated 👍🏽

[Elbehery]

@derekhiggins @skamenan7 @leseb ptal 👍🏽

[skamenan7]

sql_default is missing backend reference.

type: sqlite ?

[skamenan7]

No connect_timeout and read_timeout. Please check bedrock for example timeouts.