Skip to content

Commit 292c000

Browse files
hmtosijesuinoCopilot
authored
issue 680: configure kale to use different kfp server (kubeflow#702)
* initial changes Signed-off-by: Hannah Tosi <htosi@redhat.com> * update to include all Client parameters Signed-off-by: Hannah Tosi <htosi@redhat.com> * remove UI components Signed-off-by: Hannah Tosi <htosi@redhat.com> * remove ui components Signed-off-by: Hannah Tosi <htosi@redhat.com> * remove test until functionality is confirmed Signed-off-by: Hannah Tosi <htosi@redhat.com> * finish removing ui functionality Signed-off-by: Hannah Tosi <htosi@redhat.com> * add unit test for kfp server config Signed-off-by: Hannah Tosi <htosi@redhat.com> * update paramter descriptions Signed-off-by: Hannah Tosi <htosi@redhat.com> * fix linting Signed-off-by: Hannah Tosi <htosi@redhat.com> * change to factory architecture Signed-off-by: Hannah Tosi <htosi@redhat.com> * add factory to git tracking Signed-off-by: Hannah Tosi <htosi@redhat.com> * update documentation Signed-off-by: Hannah Tosi <htosi@redhat.com> * update so e2e passes Signed-off-by: Hannah Tosi <htosi@redhat.com> * update default config path Signed-off-by: Hannah Tosi <htosi@redhat.com> * update kfp authentication to allow credentials Signed-off-by: Hannah Tosi <htosi@redhat.com> * add authenticator file Signed-off-by: Hannah Tosi <htosi@redhat.com> * Fix circular import Signed-off-by: William Siqueira <william.fatecsjc@gmail.com> * Apply suggestion from @Copilot Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: Hannah Tosi <htosi@redhat.com> * add config env var so it can be explicitly set Signed-off-by: Hannah Tosi <htosi@redhat.com> * add atomic rename suggested by copilot Signed-off-by: Hannah Tosi <htosi@redhat.com> * update documentation Signed-off-by: Hannah Tosi <htosi@redhat.com> * separate configuration (persisted) from credentials (resolved at runtime) Signed-off-by: Hannah Tosi <htosi@redhat.com> * prevent tokens from being saved to disk Signed-off-by: Hannah Tosi <htosi@redhat.com> * add authenticator class and standardize authentictaion order Signed-off-by: Hannah Tosi <htosi@redhat.com> --------- Signed-off-by: Hannah Tosi <htosi@redhat.com> Signed-off-by: William Siqueira <william.fatecsjc@gmail.com> Co-authored-by: William Siqueira <william.fatecsjc@gmail.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
1 parent 37578f1 commit 292c000

7 files changed

Lines changed: 1101 additions & 14 deletions

File tree

README.md

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -135,6 +135,50 @@ make docker-run
135135
- **KFP UI links** pointing to `localhost:8080` (so pipeline links open in your browser)
136136
- **Wheel server** connectivity for compiled pipelines
137137

138+
## KFP Server Configuration
139+
140+
Kale stores connection settings in `~/.config/kale/kfp_server_config.json` while keeping credentials secure. **Tokens and secrets are never saved to disk** — only references to environment variables or file paths are stored.
141+
142+
### Quick Setup
143+
144+
**Using environment variables (recommended):**
145+
```bash
146+
# Set your token
147+
export KF_PIPELINES_TOKEN=your-token-here
148+
149+
# Configure Kale to use it
150+
python -c "
151+
from kale.config import kfp_server_config
152+
kfp_server_config.save_config({
153+
'host': 'http://ml-pipeline.kubeflow:8888',
154+
'auth_type': 'existing_bearer_token',
155+
'auth_config': {'env_var': 'KF_PIPELINES_TOKEN'}
156+
})
157+
"
158+
```
159+
160+
**Using Kubernetes mounted secrets:**
161+
```python
162+
from kale.config import kfp_server_config
163+
164+
kfp_server_config.save_config({
165+
"host": "http://ml-pipeline.kubeflow:8888",
166+
"auth_type": "existing_bearer_token",
167+
"auth_config": {"file_path": "/var/run/secrets/kfp-token"}
168+
})
169+
```
170+
171+
### Authentication Types
172+
173+
| Type | Description | Config Example |
174+
|------|-------------|----------------|
175+
| `none` | No authentication | `{"auth_type": "none"}` |
176+
| `existing_bearer_token` | Bearer token | `{"auth_type": "existing_bearer_token", "auth_config": {"env_var": "KF_PIPELINES_TOKEN"}}` |
177+
| `dex` | DEX cookies | `{"auth_type": "dex", "auth_config": {"env_var": "KF_PIPELINES_COOKIES"}}` |
178+
| `kubernetes_service_account_token` | K8s service account | `{"auth_type": "kubernetes_service_account_token", "auth_config": {"token_path": "/var/run/secrets/kubernetes.io/serviceaccount/token"}}` |
179+
180+
**Security:** Configuration stores only references (`env_var`, `file_path`), never actual credentials. Tokens are read fresh from the environment or filesystem at runtime.
181+
138182
## Cell Types
139183

140184
Kale uses special cell types (tags) to organize your notebook into pipeline components. You can assign these types to cells using the Kale JupyterLab extension or by adding tags directly in the notebook metadata.

kale/common/kfp_authenticator.py

Lines changed: 254 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,254 @@
1+
# Copyright 2026 The Kubeflow Authors.
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
"""KFP authentication module for creating credentials at runtime."""
16+
17+
from abc import ABC, abstractmethod
18+
import logging
19+
import os
20+
from typing import Any
21+
22+
log = logging.getLogger(__name__)
23+
24+
25+
class AuthResult:
26+
"""Result from authentication containing credentials for kfp.Client.
27+
28+
This class holds the various authentication artifacts that can be passed
29+
to kfp.Client(). Only one authentication method should be set at a time.
30+
31+
Attributes:
32+
credentials: ServiceAccountTokenVolumeCredentials object for K8s service account auth
33+
cookies: Cookie string for DEX-based authentication
34+
existing_token: Bearer token string for token-based authentication
35+
"""
36+
37+
def __init__(
38+
self,
39+
credentials: Any | None = None,
40+
cookies: str | None = None,
41+
existing_token: str | None = None,
42+
):
43+
self.credentials = credentials
44+
self.cookies = cookies
45+
self.existing_token = existing_token
46+
47+
48+
class Authenticator(ABC):
49+
"""Base class for KFP authentication strategies."""
50+
51+
@abstractmethod
52+
def authenticate(self, config: dict[str, Any] | None = None) -> AuthResult:
53+
"""Create credentials for KFP authentication.
54+
55+
Args:
56+
config: Optional configuration dictionary specific to auth type
57+
58+
Returns:
59+
AuthResult with resolved credentials
60+
"""
61+
pass
62+
63+
64+
class K8sServiceAccountTokenAuthenticator(Authenticator):
65+
"""Authenticator for Kubernetes service account token-based authentication.
66+
67+
Creates a ServiceAccountTokenVolumeCredentials object that reads the
68+
service account token from a file path (typically mounted by Kubernetes).
69+
"""
70+
71+
def authenticate(self, config: dict[str, Any] | None = None) -> AuthResult:
72+
"""Create credentials from Kubernetes service account token.
73+
74+
Args:
75+
config: Optional dictionary containing:
76+
- token_path: Path to service account token file.
77+
If not provided, uses KF_PIPELINES_SA_TOKEN_PATH env var or standard location.
78+
79+
Returns:
80+
AuthResult with ServiceAccountTokenVolumeCredentials
81+
82+
Raises:
83+
FileNotFoundError: If token file doesn't exist
84+
ValueError: If token file is empty
85+
"""
86+
from kfp.client import KF_PIPELINES_SA_TOKEN_PATH, ServiceAccountTokenVolumeCredentials
87+
88+
config = config or {}
89+
token_path = config.get(
90+
"token_path",
91+
os.getenv("KF_PIPELINES_SA_TOKEN_PATH", KF_PIPELINES_SA_TOKEN_PATH),
92+
)
93+
94+
# Validate token file exists and is non-empty
95+
if not os.path.exists(token_path):
96+
raise FileNotFoundError(
97+
f"Service account token file not found at {token_path}. "
98+
"Ensure you're running in a Kubernetes pod with a service account token mounted."
99+
)
100+
101+
with open(token_path) as f:
102+
token_content = f.read().strip()
103+
if not token_content:
104+
raise ValueError(f"Service account token file at {token_path} is empty")
105+
106+
log.info("Using Kubernetes service account token from %s", token_path)
107+
credentials = ServiceAccountTokenVolumeCredentials(path=token_path)
108+
return AuthResult(credentials=credentials)
109+
110+
111+
class ExistingBearerTokenAuthenticator(Authenticator):
112+
"""Authenticator for pre-existing bearer token authentication.
113+
114+
Resolves token from environment variable or file, never stores it directly in config.
115+
"""
116+
117+
def authenticate(self, config: dict[str, Any] | None = None) -> AuthResult:
118+
"""Create credentials from an existing bearer token.
119+
120+
Token is resolved from environment variable or file at runtime.
121+
122+
Args:
123+
config: Dictionary containing ONE of:
124+
- file_path: Path to file containing the token
125+
- env_var: Name of environment variable containing the token
126+
If neither provided, checks KF_PIPELINES_TOKEN env var by default
127+
128+
Returns:
129+
AuthResult with bearer token
130+
131+
Raises:
132+
ValueError: If token cannot be resolved
133+
"""
134+
config = config or {}
135+
136+
file_path = config.get("file_path")
137+
if file_path:
138+
if not os.path.exists(file_path):
139+
raise FileNotFoundError(f"Token file not found at {file_path}")
140+
141+
with open(file_path) as f:
142+
token = f.read().strip()
143+
if not token:
144+
raise ValueError(f"Token file at {file_path} is empty")
145+
146+
log.info("Using bearer token from file %s", file_path)
147+
return AuthResult(existing_token=token)
148+
149+
env_var = config.get("env_var", "KF_PIPELINES_TOKEN")
150+
token = os.getenv(env_var)
151+
if token:
152+
log.info("Using bearer token from environment variable %s", env_var)
153+
return AuthResult(existing_token=token.strip())
154+
155+
raise ValueError(
156+
f"Bearer token not found. Set {env_var} environment variable "
157+
f"or provide file_path in auth_config"
158+
)
159+
160+
161+
class DexAuthenticator(Authenticator):
162+
"""Authenticator for DEX-based authentication using cookies.
163+
164+
Resolves cookies from environment variable or file, never stores them directly in config.
165+
"""
166+
167+
def authenticate(self, config: dict[str, Any] | None = None) -> AuthResult:
168+
"""Create credentials from DEX session cookies.
169+
170+
Cookies are resolved from environment variable or file at runtime.
171+
172+
Args:
173+
config: Dictionary containing ONE of:
174+
- env_var: Name of environment variable containing the cookies
175+
- file_path: Path to file containing the cookies
176+
If neither provided, checks KF_PIPELINES_COOKIES env var by default
177+
178+
Returns:
179+
AuthResult with cookies
180+
181+
Raises:
182+
ValueError: If cookies cannot be resolved
183+
"""
184+
config = config or {}
185+
186+
file_path = config.get("file_path")
187+
if file_path:
188+
if not os.path.exists(file_path):
189+
raise FileNotFoundError(f"Cookies file not found at {file_path}")
190+
191+
with open(file_path) as f:
192+
cookies = f.read().strip()
193+
if not cookies:
194+
raise ValueError(f"Cookies file at {file_path} is empty")
195+
196+
log.info("Using DEX cookies from file %s", file_path)
197+
return AuthResult(cookies=cookies)
198+
199+
env_var = config.get("env_var", "KF_PIPELINES_COOKIES")
200+
if env_var:
201+
cookies = os.getenv(env_var)
202+
if cookies:
203+
log.info("Using DEX cookies from environment variable %s", env_var)
204+
return AuthResult(cookies=cookies.strip())
205+
206+
raise ValueError(
207+
f"DEX cookies not found. Set {env_var} environment variable "
208+
f"or provide file_path in auth_config"
209+
)
210+
211+
212+
class NoAuthAuthenticator(Authenticator):
213+
"""Authenticator for unsecured KFP endpoints (no authentication required)."""
214+
215+
def authenticate(self, config: dict[str, Any] | None = None) -> AuthResult:
216+
"""Return empty credentials for unsecured endpoints.
217+
218+
Args:
219+
config: Ignored
220+
221+
Returns:
222+
AuthResult with no credentials set
223+
"""
224+
log.info("Using no authentication (unsecured endpoint)")
225+
return AuthResult()
226+
227+
228+
def get_authenticator(auth_type: str) -> Authenticator:
229+
"""Factory function to get the appropriate authenticator for an auth type.
230+
231+
Args:
232+
auth_type: Authentication type. Supported values:
233+
- "kubernetes_service_account_token": K8s service account token
234+
- "existing_bearer_token": Pre-existing bearer token
235+
- "dex": DEX cookie-based authentication
236+
- "none": No authentication
237+
238+
Returns:
239+
Authenticator instance for the specified type.
240+
Defaults to NoAuthAuthenticator for unknown types.
241+
"""
242+
authenticators = {
243+
"kubernetes_service_account_token": K8sServiceAccountTokenAuthenticator(),
244+
"existing_bearer_token": ExistingBearerTokenAuthenticator(),
245+
"dex": DexAuthenticator(),
246+
"none": NoAuthAuthenticator(),
247+
}
248+
249+
authenticator = authenticators.get(auth_type)
250+
if authenticator is None:
251+
log.warning("Unknown auth_type '%s', defaulting to no authentication", auth_type)
252+
return NoAuthAuthenticator()
253+
254+
return authenticator

kale/common/kfp_client_factory.py

Lines changed: 82 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,82 @@
1+
# Copyright 2026 The Kubeflow Authors.
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
"""Factory for creating KFP client instances with configuration support."""
16+
17+
from typing import TYPE_CHECKING
18+
19+
import kfp
20+
21+
from kale.common import kfp_authenticator
22+
from kale.config import kfp_server_config
23+
24+
if TYPE_CHECKING:
25+
from kfp import Client
26+
27+
28+
def get_kfp_client(
29+
host: str | None = None,
30+
auth_type: str | None = None,
31+
auth_config: dict | None = None,
32+
namespace: str | None = None,
33+
ssl_ca_cert: str | None = None,
34+
) -> "Client":
35+
"""Create a KFP client with configuration.
36+
37+
Loads saved configuration from ~/.config/kale/kfp_server_config.json and allows
38+
parameter overrides. Explicit parameters override saved config if they are provided.
39+
40+
Authentication is handled by creating credentials at runtime using the authenticator
41+
module. Credentials are NEVER stored in config - only references to where they
42+
can be found (env vars, file paths).
43+
44+
Args:
45+
host: KFP API server host
46+
auth_type: Authentication type. Supported values:
47+
- "kubernetes_service_account_token": K8s service account token
48+
- "existing_bearer_token": Pre-existing bearer token
49+
- "dex": DEX cookie-based authentication
50+
- "none": No authentication (default)
51+
auth_config: Configuration references for authentication:
52+
- {"env_var": "VAR_NAME"}: Read credential from environment variable
53+
- {"file_path": "/path/to/file"}: Read credential from file
54+
- {"token_path": "/path/to/token"}: K8s SA token path (SA auth only)
55+
namespace: Kubernetes namespace
56+
ssl_ca_cert: Path to CA certificate file
57+
58+
Returns:
59+
kfp.Client instance configured with provided parameters or saved config
60+
"""
61+
# Load saved configuration
62+
config = kfp_server_config.load_config()
63+
64+
# Use parameter if provided, otherwise fall back to config
65+
host = host or config.host
66+
auth_type = auth_type or config.auth_type or "none"
67+
auth_config = auth_config or config.auth_config or {}
68+
namespace = namespace or config.namespace or "kubeflow"
69+
ssl_ca_cert = ssl_ca_cert or config.ssl_ca_cert
70+
71+
# Create credentials at runtime using authenticator
72+
authenticator = kfp_authenticator.get_authenticator(auth_type)
73+
auth_result = authenticator.authenticate(auth_config)
74+
75+
return kfp.Client(
76+
host=host,
77+
credentials=auth_result.credentials,
78+
cookies=auth_result.cookies,
79+
existing_token=auth_result.existing_token,
80+
namespace=namespace,
81+
ssl_ca_cert=ssl_ca_cert,
82+
)

0 commit comments

Comments
 (0)