Skip to content

Commit 506da30

Browse files
tsebastianitsebastiani
committed
krkn container permissions fix
fix again service account
1 parent 9cd7ad7 commit 506da30

File tree

4 files changed

+218
-7
lines changed

4 files changed

+218
-7
lines changed

‎PROGRESS.md‎

Lines changed: 182 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -2,8 +2,8 @@
22

33
## Current Status
44

5-
**Last updated:** 2026-01-13
6-
**Current phase:** REST API Enhancement - POST /scenarios/globals COMPLETED
5+
**Last updated:** 2026-01-14
6+
**Current phase:** Scenario Job Management - COMPLETED
77

88
### Completed
99
- Project scaffolding with Kubebuilder
@@ -80,6 +80,23 @@
8080
- ✅ Returns 404 if global environment not found
8181
- ✅ Type fields converted to strings (not enum integers)
8282
- ✅ Handler registered at POST /scenarios/globals/{scenario_name}
83+
- **Scenario Job Management endpoints completed:**
84+
- ✅ POST /scenarios/run - Create and start scenario job
85+
- ✅ GET /scenarios/run/{jobId} - Get job status
86+
- ✅ GET /scenarios/run/{jobId}/logs - Stream logs (stub, requires clientset)
87+
- ✅ GET /scenarios/run - List all jobs with filtering
88+
- ✅ DELETE /scenarios/run/{jobId} - Stop and delete job
89+
- ✅ Router handler for endpoint dispatching
90+
- ✅ Complete kubeconfig retrieval and mounting
91+
- ✅ File mounts via ConfigMaps (base64 encoded)
92+
- ✅ Private registry support with ImagePullSecrets
93+
- ✅ Pod lifecycle management with cleanup
94+
- ✅ RBAC permissions updated
95+
- **Code refactoring completed:**
96+
- ✅ Extracted getKubeconfigFromTarget() helper function
97+
- ✅ Refactored GET /nodes to use helper
98+
- ✅ Refactored POST /scenarios/run to use helper
99+
- ✅ Eliminated ~130 lines of duplicated code
83100

84101
### In Progress
85102
- None
@@ -837,6 +854,159 @@ curl -X POST http://localhost:8080/scenarios/globals/pod-scenarios \
837854
}
838855
```
839856

857+
### Phase 12: Scenario Job Management - /scenarios/run Endpoints ✅
858+
**Status:** COMPLETED
859+
860+
1. ✅ Type Definitions (internal/api/types.go)
861+
- Created `FileMount` struct for user file uploads
862+
- Created `ScenarioRunRequest` with embedded `ScenariosRequest`
863+
- Created `ScenarioRunResponse` for job creation
864+
- Created `JobStatusResponse` with targetId, clusterName metadata
865+
- Created `JobsListResponse` for listing jobs
866+
867+
2. ✅ POST /scenarios/run Implementation (internal/api/handlers.go)
868+
- Validates required fields: targetId, clusterName, scenarioImage, scenarioName
869+
- Uses `getKubeconfigFromTarget()` helper to retrieve kubeconfig
870+
- Creates ConfigMap for kubeconfig (decoded from base64)
871+
- Creates ConfigMaps for user-provided files (base64 decoded)
872+
- Supports private registry with ImagePullSecret creation
873+
- Creates pod with:
874+
- RestartPolicy: Never (one-time execution)
875+
- Labels: app, krkn-job-id, krkn-scenario-name, krkn-target-id, krkn-cluster-name
876+
- Environment variables from request
877+
- Volume mounts for kubeconfig and user files
878+
- ImagePullSecrets if registry credentials provided
879+
- Returns 201 Created with jobId, status "Pending", podName
880+
- Complete cleanup on error (deletes created ConfigMaps/Secrets)
881+
882+
3. ✅ GET /scenarios/run/{jobId} Implementation
883+
- Finds pod by label `krkn-job-id`
884+
- Maps pod phase to job status
885+
- Extracts metadata from pod labels
886+
- Returns timestamps (startTime, completionTime)
887+
- Returns error messages for failed jobs
888+
889+
4. ✅ GET /scenarios/run/{jobId}/logs Implementation (STUB)
890+
- Endpoint registered and routed correctly
891+
- Returns 501 Not Implemented
892+
- TODO: Requires Kubernetes clientset integration for actual log streaming
893+
- Query parameter parsing ready (follow, timestamps, tailLines)
894+
895+
5. ✅ GET /scenarios/run Implementation
896+
- Lists all pods with label `app=krkn-scenario`
897+
- Supports filtering by: status, scenarioName, targetId, clusterName
898+
- Returns array of JobStatusResponse
899+
- Includes timestamps and metadata for each job
900+
901+
6. ✅ DELETE /scenarios/run/{jobId} Implementation
902+
- Finds and deletes pod with 5 second grace period
903+
- Cleanup associated ConfigMaps (by label `krkn-job-id`)
904+
- Cleanup associated Secrets (ImagePullSecrets)
905+
- Returns status "Stopped" with success message
906+
907+
7. ✅ Router Handler (ScenariosRunRouter)
908+
- Dispatches requests based on path and method
909+
- POST /scenarios/run → PostScenarioRun
910+
- GET /scenarios/run → ListScenarioRuns
911+
- GET /scenarios/run/{jobId} → GetScenarioRunStatus
912+
- GET /scenarios/run/{jobId}/logs → GetScenarioRunLogs
913+
- DELETE /scenarios/run/{jobId} → DeleteScenarioRun
914+
- Returns 405 Method Not Allowed for invalid methods
915+
916+
8. ✅ RBAC Updates (config/rbac/role.yaml)
917+
- pods: added create, delete verbs
918+
- pods/log: added get verb
919+
- configmaps: added get, list, create, delete verbs
920+
- secrets: added create, delete verbs
921+
922+
9. ✅ Build Verification
923+
- Successful compilation with no errors
924+
- All handlers properly integrated
925+
- Routes registered correctly
926+
927+
**Implementation Highlights:**
928+
- **Pod naming**: `krkn-job-{jobId}`
929+
- **ConfigMap naming**: `krkn-job-{jobId}-kubeconfig`, `krkn-job-{jobId}-file-{sanitized-name}`
930+
- **Default kubeconfig path**: `/home/krkn/.kube/config` (configurable)
931+
- **Error handling**: Comprehensive cleanup on failures
932+
- **Label-based tracking**: No CRD required, uses pod labels
933+
- **Private registry**: Full support with ImagePullSecret creation
934+
935+
**Request Example:**
936+
```json
937+
{
938+
"targetId": "550e8400-e29b-41d4-a716-446655440000",
939+
"clusterName": "my-cluster-1",
940+
"scenarioImage": "quay.io/krkn-chaos/krkn-hub:pod-scenarios",
941+
"scenarioName": "pod-scenarios",
942+
"kubeconfigPath": "/home/krkn/.kube/config",
943+
"environment": {
944+
"NAMESPACE": "default",
945+
"LABEL_SELECTOR": "app=myapp"
946+
},
947+
"files": [
948+
{
949+
"name": "config.yaml",
950+
"content": "base64-encoded-content",
951+
"mountPath": "/config/scenario.yaml"
952+
}
953+
]
954+
}
955+
```
956+
957+
**Response Example:**
958+
```json
959+
{
960+
"jobId": "generated-uuid",
961+
"status": "Pending",
962+
"podName": "krkn-job-generated-uuid"
963+
}
964+
```
965+
966+
### Code Refactoring: getKubeconfigFromTarget() Helper ✅
967+
**Status:** COMPLETED
968+
969+
**Problem:** Kubeconfig retrieval logic was duplicated in GET /nodes and POST /scenarios/run (~130 lines total)
970+
971+
**Solution:** Extracted common helper function `getKubeconfigFromTarget(ctx, targetId, clusterName)`
972+
973+
**Location:** internal/api/handlers.go:275-318
974+
975+
**Functionality:**
976+
1. Fetches secret named `targetId` from operator namespace
977+
2. Parses `managed-clusters` JSON from secret data
978+
3. Navigates structure: `["krkn-operator-acm"][clusterName]["kubeconfig"]`
979+
4. Returns base64-encoded kubeconfig string
980+
5. Returns descriptive errors for all failure cases
981+
982+
**Refactored Endpoints:**
983+
1. GET /nodes (lines ~137-156): Replaced ~65 lines with helper call
984+
2. POST /scenarios/run (lines ~820-838): Replaced ~70 lines with helper call
985+
986+
**Benefits:**
987+
- **DRY Principle**: Single source of truth for kubeconfig retrieval
988+
- **Maintainability**: Changes only needed in one place
989+
- **Consistency**: Identical error handling across endpoints
990+
- **Testability**: Helper can be unit tested independently
991+
- **Code reduction**: ~130 lines eliminated
992+
993+
**Before:**
994+
```go
995+
// Duplicated in GET /nodes and POST /scenarios/run
996+
var secret corev1.Secret
997+
err := h.client.Get(ctx, ...)
998+
// ... 60+ lines of identical code ...
999+
```
1000+
1001+
**After:**
1002+
```go
1003+
// Both endpoints now use:
1004+
kubeconfigBase64, err := h.getKubeconfigFromTarget(ctx, targetId, clusterName)
1005+
if err != nil {
1006+
// Error handling
1007+
}
1008+
```
1009+
8401010
## Technical Notes
8411011

8421012
### KrknTargetRequest Structure
@@ -931,12 +1101,19 @@ type KrknTargetRequestStatus struct {
9311101
9. ✅ Implement POST /scenarios endpoint with krknctl integration
9321102
10. ✅ Implement POST /scenarios/detail/{scenario_name} endpoint
9331103
11. ✅ Implement POST /scenarios/globals endpoint
934-
12. Create API documentation (OpenAPI/Swagger spec)
935-
13. Add additional endpoints as requirements evolve
936-
14. Implement controller logic for KrknTargetRequest
1104+
12. ✅ Implement Scenario Job Management (/scenarios/run endpoints)
1105+
13. ✅ Code refactoring - extract getKubeconfigFromTarget() helper
1106+
14. Complete GET /scenarios/run/{jobId}/logs endpoint (requires Kubernetes clientset)
1107+
15. Create API documentation (OpenAPI/Swagger spec)
1108+
16. Add additional endpoints as requirements evolve
1109+
17. Implement controller logic for KrknTargetRequest
9371110

9381111
## Future Work (Not in Current Scope)
9391112

1113+
- Complete log streaming implementation for GET /scenarios/run/{jobId}/logs
1114+
- Requires Kubernetes clientset integration (rest.Config)
1115+
- Need to add clientset to Handler struct
1116+
- Implement actual streaming with io.Copy from pod logs API
9401117
- Controller implementation for KrknTargetRequest
9411118
- Controller implementation for KrknOperatorTargetProvider
9421119
- Migration from environment variable to ConfigMap for namespace configuration

‎config/rbac/kustomization.yaml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,7 @@ resources:
55
# runtime. Be sure to update RoleBinding and ClusterRoleBinding
66
# subjects if changing service account names.
77
- service_account.yaml
8+
- scenario_serviceaccount.yaml
89
- role.yaml
910
- role_binding.yaml
1011
- leader_election_role.yaml

‎config/rbac/scenario_serviceaccount.yaml‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
apiVersion: v1
2+
kind: ServiceAccount
3+
metadata:
4+
name: krkn-scenario-runner
5+
namespace: system
6+
labels:
7+
app.kubernetes.io/name: krkn-operator
8+
app.kubernetes.io/component: scenario-runner

‎internal/api/handlers.go‎

Lines changed: 27 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1048,6 +1048,25 @@ func (h *Handler) PostScenarioRun(w http.ResponseWriter, r *http.Request) {
10481048
})
10491049
}
10501050

1051+
// Add writable tmp volume for scenarios that need to write temporary files
1052+
volumes = append(volumes, corev1.Volume{
1053+
Name: "tmp",
1054+
VolumeSource: corev1.VolumeSource{
1055+
EmptyDir: &corev1.EmptyDirVolumeSource{},
1056+
},
1057+
})
1058+
1059+
volumeMounts = append(volumeMounts, corev1.VolumeMount{
1060+
Name: "tmp",
1061+
MountPath: "/tmp",
1062+
})
1063+
1064+
// SecurityContext for running as krkn user (UID 1000)
1065+
// Requires ServiceAccount with anyuid SCC permissions
1066+
var runAsUser int64 = 1000
1067+
var runAsGroup int64 = 1000
1068+
var fsGroup int64 = 1000
1069+
10511070
// Create the pod
10521071
pod := &corev1.Pod{
10531072
ObjectMeta: metav1.ObjectMeta{
@@ -1062,8 +1081,14 @@ func (h *Handler) PostScenarioRun(w http.ResponseWriter, r *http.Request) {
10621081
},
10631082
},
10641083
Spec: corev1.PodSpec{
1065-
RestartPolicy: corev1.RestartPolicyNever,
1066-
ImagePullSecrets: imagePullSecrets,
1084+
ServiceAccountName: "krkn-scenario-runner",
1085+
RestartPolicy: corev1.RestartPolicyNever,
1086+
ImagePullSecrets: imagePullSecrets,
1087+
SecurityContext: &corev1.PodSecurityContext{
1088+
RunAsUser: &runAsUser,
1089+
RunAsGroup: &runAsGroup,
1090+
FSGroup: &fsGroup,
1091+
},
10671092
Containers: []corev1.Container{
10681093
{
10691094
Name: "scenario",

0 commit comments

Comments
 (0)