Failed to initialize Minio Client for Minio Provider: Error: Encountered error when trying to fetch provider secret mlpipeline-minio-artifact.

[Leoauro]

When I deployed Kubeflow on Kubernetes using all default configurations, including the default Minio storage, I encountered an error when previewing artifacts in the Kubeflow Pipeline:

Error: Failed to initialize Minio Client for Minio Provider: Error: Encountered error when trying to fetch provider secret mlpipeline-minio-artifact.

kubeflow version: v1.9.1

I checked the API request and found that the error comes from:

http://8.148.230.236/pipeline/artifacts/get?

The following is the decoded request payload sent to the endpoint

source=minio  
namespace=kubeflow-user-example-com  
providerInfo={
  "Provider": "minio",
  "Params": {
    "accessKeyKey": "accesskey",
    "disableSSL": "true",
    "endpoint": "minio-service.kubeflow:9000",
    "fromEnv": "false",
    "region": "minio",
    "secretKeyKey": "secretkey",
    "secretName": "mlpipeline-minio-artifact"
  }
}  
peek=256  
bucket=mlpipeline  
key=v2/artifacts/manual-metrics/2dd62a94-b3da-416f-9b3d-7c0f9fef003c/evaluate-classification/classification_metrics  

When I manually changed the fromEnv field in providerInfo to true, the API worked as expected.

Further investigation led me to the source code:

pipelines/frontend/server/minio-helper.ts

Line 116 in ada935a

async function parseS3ProviderInfo(

To debug, I used an AI-generated test script and executed it inside the ml-pipeline-ui pod (kubectl exec -it -n kubeflow ml-pipeline-ui-54c7d4d5fb-b4b94 -- /bin/sh). The script successfully retrieved the accessKey and secretKey:

const { getK8sSecret } = require('./k8s-helper');  

async function testRealK8sSecret() {  
  const providerInfo = {  
    Params: {  
      secretName: 'mlpipeline-minio-artifact',  
      accessKeyKey: 'accesskey',  
      secretKeyKey: 'secretkey'  
    }  
  };  
  const namespace = 'kubeflow-user-example-com';  

  try {  
    const config = {};  

    config.accessKey = await getK8sSecret(  
      providerInfo.Params.secretName,  
      providerInfo.Params.accessKeyKey,  
      namespace  
    );  

    config.secretKey = await getK8sSecret(  
      providerInfo.Params.secretName,  
      providerInfo.Params.secretKeyKey,  
      namespace  
    );  

    console.log('✅ success');  
    console.log('secret:', {  
      accessKey: config.accessKey ? config.accessKey : 'NULL',  
      secretKey: config.secretKey ? config.secretKey : 'NULL'  
    });  

    if (!config.accessKey || !config.secretKey) {  
      throw new Error('fail');  
    }  
  } catch (e) {  
    console.error('❌ test fail:', e.message);  
    process.exit(1);  
  }  
}  

testRealK8sSecret();  

Questions:

• What could be the root cause of this issue?
• Where can I configure fromEnv to default to true?
• Are there any alternative solutions to resolve this problem?

Any help would be greatly appreciated!

[zazulam]

I would suggest updating your kubeflow version to the latest version, 1.10.2 and trying again. Let us know if the issue persists!

[juliusvonkohout]

#12851