Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,13 +53,23 @@ KUBEFLOW_LLAMA_STACK_URL=<your-llama-stack-url>
KUBEFLOW_PIPELINES_ENDPOINT=<your-kfp-endpoint>
KUBEFLOW_NAMESPACE=<your-namespace>
KUBEFLOW_BASE_IMAGE=quay.io/diegosquayorg/my-ragas-provider-image:latest
KUBEFLOW_RESULTS_S3_PREFIX=s3://my-bucket/ragas-results
KUBEFLOW_S3_CREDENTIALS_SECRET_NAME=<secret-name>
```

Where:
- `KUBEFLOW_LLAMA_STACK_URL`: The URL of the llama stack server that the remote provider will use to run the evaluation (LLM generations and embeddings, etc.). If you are running Llama Stack locally, you can use [ngrok](https://ngrok.com/) to expose it to the remote provider.
- `KUBEFLOW_PIPELINES_ENDPOINT`: You can get this via `kubectl get routes -A | grep -i pipeline` on your Kubernetes cluster.
- `KUBEFLOW_NAMESPACE`: The name of the data science project where the Kubeflow Pipelines server is running.
- `KUBEFLOW_BASE_IMAGE`: The image used to run the Ragas evaluation in the remote provider. See `Containerfile` for details. There is a public version of this image at `quay.io/diegosquayorg/my-ragas-provider-image:latest`.
- `KUBEFLOW_RESULTS_S3_PREFIX`: S3 location (bucket and prefix folder) where evaluation results will be stored, e.g., `s3://my-bucket/ragas-results`.
- `KUBEFLOW_S3_CREDENTIALS_SECRET_NAME`: Name of the Kubernetes secret containing AWS credentials with write access to the S3 bucket. Create with:
```bash
oc create secret generic <secret-name> \
--from-literal=AWS_ACCESS_KEY_ID=your-access-key \
--from-literal=AWS_SECRET_ACCESS_KEY=your-secret-key \
--from-literal=AWS_DEFAULT_REGION=us-east-1
```

Run the server:
```bash
Expand Down
20 changes: 20 additions & 0 deletions docs/modules/ROOT/pages/remote-provider.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -142,6 +142,10 @@ KUBEFLOW_NAMESPACE=<your-namespace>

# Container image for remote execution
KUBEFLOW_BASE_IMAGE=quay.io/diegosquayorg/my-ragas-provider-image:latest

# S3 configuration for storing evaluation results
KUBEFLOW_RESULTS_S3_PREFIX=s3://my-bucket/ragas-results
KUBEFLOW_S3_CREDENTIALS_SECRET_NAME=<secret-name>
----

=== Environment Variable Details
Expand All @@ -166,6 +170,22 @@ The name of the data science project where the Kubeflow Pipelines server is runn
`KUBEFLOW_BASE_IMAGE`::
The container image used to run the Ragas evaluation in the remote provider. See the `Containerfile` in the repository root for details on building a custom image.

`KUBEFLOW_RESULTS_S3_PREFIX`::
The S3 location (bucket and prefix) where evaluation results will be stored. This should be a folder path, e.g., `s3://my-bucket/ragas-results`. The remote provider will write evaluation outputs to this location.

`KUBEFLOW_S3_CREDENTIALS_SECRET_NAME`::
The name of the Kubernetes secret containing AWS credentials with write access to the S3 bucket specified in `KUBEFLOW_RESULTS_S3_PREFIX`. This secret will be mounted as environment variables in the Kubeflow pipeline components.
+
To create the secret:
+
[,bash]
----
oc create secret generic <secret-name> \
--from-literal=AWS_ACCESS_KEY_ID=your-access-key \
--from-literal=AWS_SECRET_ACCESS_KEY=your-secret-key \
--from-literal=AWS_DEFAULT_REGION=us-east-1
----

=== Distribution Configuration

The repository includes a sample Llama Stack distribution configuration that uses Ollama as a provider for inference and embeddings.
Expand Down
2 changes: 1 addition & 1 deletion src/llama_stack_provider_ragas/remote/ragas_remote_eval.py
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ class RagasEvaluationJob(Job):

@property
def result_s3_location(self) -> str:
return f"{self.runtime_config.kubeflow_config.results_s3_prefix}/{self.job_id}/results.jsonl"
return f"{self.runtime_config.kubeflow_config.results_s3_prefix.rstrip('/')}/{self.job_id}/results.jsonl"


@json_schema_type
Expand Down