RHAIENG-2131: Add S3 remote provider configs to LLS midstream container.

This MR adds S3 remote provider configs to LLS midstream container.

Signed-off-by: Mustafa Elbehery <melbeher@redhat.com>

Closes https://issues.redhat.com/browse/RHAIENG-2131

Author: Elbehery

distribution/README.md

@@ -18,6 +18,7 @@ You can see an overview of the APIs and Providers the image ships with in the ta
 | eval | remote::trustyai_lmeval | Yes (version 0.4.1) | ✅ | N/A |
 | eval | remote::trustyai_ragas | Yes (version 0.5.1) | ❌ | Set the `KUBEFLOW_LLAMA_STACK_URL` environment variable |
 | files | inline::localfs | No | ✅ | N/A |
+| files | remote::s3 | No | ❌ | Set the `ENABLE_S3` environment variable |
 | inference | inline::sentence-transformers | No | ✅ | N/A |
 | inference | remote::azure | No | ❌ | Set the `AZURE_API_KEY` environment variable |
 | inference | remote::bedrock | No | ❌ | Set the `AWS_ACCESS_KEY_ID` environment variable |

distribution/run.yaml

@@ -181,6 +181,18 @@ providers:
       metadata_store:
         backend: sql_files
         table_name: files_metadata
+  - provider_id: ${env.ENABLE_S3:+s3}
+    provider_type: remote::s3
+    config:
+      bucket_name: ${env.S3_BUCKET_NAME:=}
+      region: ${env.AWS_DEFAULT_REGION:=us-east-1}
+      aws_access_key_id: ${env.AWS_ACCESS_KEY_ID:=}
+      aws_secret_access_key: ${env.AWS_SECRET_ACCESS_KEY:=}
+      endpoint_url: ${env.S3_ENDPOINT_URL:=}
+      auto_create_bucket: ${env.S3_AUTO_CREATE_BUCKET:=false}
+      metadata_store:
+        backend: sql_files
+        table_name: files_metadata
   batches:
   - provider_id: reference
     provider_type: inline::reference