feat(autox): Add auto-discovery support for managed MinIO DSPAs (opendatahub-io#7256)

* feat(automl/bff): Add auto-discovery support for managed MinIO DSPAs

Enable BFF to automatically discover and inject S3 credentials when DSPA
uses managed MinIO (minio.deploy: true), eliminating the need for frontend
to explicitly pass secretName parameter for MinIO-based deployments.

Changes:
- Add MinioStorage model to DSPipelineApplication ObjectStorage spec
- Enhance injectDSPAObjectStorageIfAvailable() to detect managed MinIO
- Update AttachPipelineServerClient middleware for MinIO auto-discovery
- Add mock MinIO DSPA to test data (minio-test namespace)
- Add unit tests for MinIO auto-discovery functionality

When DSPA uses managed MinIO, the BFF now:
1. Detects minio.deploy: true in DSPA spec
2. Constructs secret name: ds-pipeline-s3-{dspa-name}
3. Constructs endpoint: http://minio-{dspa-name}.{namespace}.svc.cluster.local:9000
4. Injects DSPAObjectStorage into request context
5. S3 handlers use auto-discovered config (no secretName needed)

External storage is still preferred when both external and MinIO exist.

Resolves the 400 Bad Request error when accessing S3 endpoints in
namespaces with managed MinIO DSPAs.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* feat(autorag/bff): Add auto-discovery support for managed MinIO DSPAs

Enable BFF to automatically discover and inject S3 credentials when DSPA
uses managed MinIO (minio.deploy: true), eliminating the need for frontend
to explicitly pass secretName parameter for MinIO-based deployments.

Changes:
- Add MinioStorage model to DSPipelineApplication ObjectStorage spec
- Enhance injectDSPAObjectStorageIfAvailable() to detect managed MinIO
- Update AttachPipelineServerClient middleware for MinIO auto-discovery
- Add mock MinIO DSPA to test data (minio-test namespace)
- Add unit tests for MinIO auto-discovery functionality

When DSPA uses managed MinIO, the BFF now:
1. Detects minio.deploy: true in DSPA spec
2. Constructs secret name: ds-pipeline-s3-{dspa-name}
3. Constructs endpoint: http://minio-{dspa-name}.{namespace}.svc.cluster.local:9000
4. Injects DSPAObjectStorage into request context
5. S3 handlers use auto-discovered config (no secretName needed)

External storage is still preferred when both external and MinIO exist.

Resolves the 400 Bad Request error when accessing S3 endpoints in
namespaces with managed MinIO DSPAs.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* docs(automl,autorag): document managed MinIO local dev setup

Add instructions for port-forwarding the MinIO service and
configuring /etc/hosts when working with DSPAs that use managed
MinIO in local development.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(automl,autorag/bff): Allow HTTP for in-cluster MinIO endpoints

- Updated S3 client validation to permit HTTP scheme for in-cluster endpoints
  (*.svc.cluster.local)
- In-cluster endpoints skip DNS resolution and IP validation as they are
  trusted cluster-internal services
- External endpoints still require HTTPS to prevent credentials in cleartext
- Added tests for HTTP in-cluster endpoints (MinIO service scenarios)
- Updated test names to clarify HTTP is rejected only for external endpoints
- Fixed autorag test to check for invalid scheme instead of invalid URL

This fixes the issue where auto-discovered managed MinIO configurations
were being rejected because they use HTTP internally within the cluster.

Both automl and autorag packages updated with identical changes.

* security(automl,autorag/bff): Require 5-label FQDN for in-cluster endpoints

Enhanced isInCluster validation to match gen-ai pattern:
- Require fully-qualified Kubernetes service DNS name with 5+ labels
  (format: <service>.<namespace>.svc.cluster.local)
- Prevents overly-broad matches like 'evil.svc.cluster.local' (4 labels)
  or 'evil.cluster.local' (3 labels)
- Added comprehensive tests for invalid cluster-local hostnames

This aligns with the security pattern established in the gen-ai package
and prevents potential SSRF attacks via malicious hostnames with partial
.svc.cluster.local or .cluster.local suffixes.

* feat(automl,autorag/bff): Read endpoint and bucket from MinIO secret

Enhanced GetS3CredentialsFromDSPA to prefer secret values over constructed defaults:
- Try to read AWS_S3_ENDPOINT from the secret first
- Fall back to constructed endpoint (http://minio-{name}.{ns}.svc.cluster.local:9000) if not present
- Similarly, prefer AWS_S3_BUCKET from secret over DSPA spec bucket
- Allows custom endpoint configurations while maintaining backward compatibility

This enables:
1. Custom MinIO endpoint configurations via secret
2. Non-standard ports or hostnames
3. External MinIO instances referenced from DSPA
4. Full flexibility for different deployment scenarios

Addresses feedback to read endpoint details from the secret instead of
hardcoding, while maintaining backward compatibility with the default
DSPA operator behavior.

* fix: address coderabbit

* fix: address comments

---------

Co-authored-by: Daniel Duong <danielduong@ibm.com>
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
Co-authored-by: Chris Jones <chrjones@redhat.com>

Author: 4 people

packages/automl/bff/README.md

@@ -247,6 +247,45 @@ curl -H "kubeflow-userid: user@example.com" \
   "http://localhost:4003/api/v1/s3/file?namespace=<namespace>&secretName=<secret>&bucket=<bucket>&key=<key>"
 ```
 
+#### Managed MinIO in port-forward mode
+
+When the DSPA uses managed MinIO (`minio.deploy: true`) instead of external S3, the BFF auto-discovers the MinIO configuration and constructs an in-cluster endpoint URL of the form `http://minio-<dspa-name>.<namespace>.svc.cluster.local:9000`. For this to work locally you need to port-forward the MinIO service and add a `/etc/hosts` entry so the BFF can resolve the in-cluster hostname.
+
+**1. Identify the MinIO service and DSPA name**
+
+```shell
+# Find the MinIO service in your namespace
+oc get svc -n <namespace> | grep minio
+
+# The service is typically named "minio-<dspa-name>", e.g. "minio-dspa"
+```
+
+**2. Port-forward the MinIO service**
+
+```shell
+oc port-forward -n <namespace> svc/minio-<dspa-name> 9000:9000
+```
+
+**3. Add a `/etc/hosts` entry**
+
+The BFF constructs the MinIO endpoint using the in-cluster DNS name. Add an entry to `/etc/hosts` so this resolves to your local port-forward:
+
+```shell
+# Add to /etc/hosts (requires sudo)
+echo "127.0.0.1 minio-<dspa-name>.<namespace>.svc.cluster.local" | sudo tee -a /etc/hosts
+```
+
+For example, if your DSPA is named `dspa` in namespace `my-project`:
+
+```shell
+oc port-forward -n my-project svc/minio-dspa 9000:9000
+echo "127.0.0.1 minio-dspa.my-project.svc.cluster.local" | sudo tee -a /etc/hosts
+```
+
+After this, the BFF will auto-discover the MinIO storage config from the DSPA spec and connect to MinIO via your port-forward. No `secretName` query parameter is needed.
+
+> **Cleanup:** Remember to remove the `/etc/hosts` entry when you're done testing.
+
 ### Federated development with a live cluster
 
 To run the AutoML module as a federated micro-frontend against the main ODH Dashboard with a real pipeline server, you need three things running: