Resiliency Feature Draft (krknctl) by abhinavs1920 · Pull Request #82 · krkn-chaos/krknctl

abhinavs1920 · 2025-11-05T16:08:01Z

Description

Add env variable (krkn-run-mode) in krknctl-input.json.
Implement resiliency-config file volume binding in the container.
Implement result calculation fetching container's long.

Documentation

Is documentation needed for this update?

If checked, a documentation PR must be created and merged in the website repository.

Related Documentation PR (if applicable)

Signed-off-by: Abhinav Sharma <abhinavs1920bpl@gmail.com>

abhinavs1920 · 2025-11-06T15:14:29Z

Hii @tsebastiani, I have added the env flag for the krkn-mode, let me know if its the right approach?

Signed-off-by: Abhinav Sharma <abhinavs1920bpl@gmail.com>

abhinavs1920 · 2025-11-06T16:05:44Z

In my last commit I have updated the Scenario struct, and added ResiliencyConfig into it which is essentially will be a String containing the path of that specific scenario resiliency config.

tsebastiani · 2025-11-06T16:21:35Z

tests/containerfiles/dummyscenario/krknctl-input.json

-  }
+  },
+  {
+  "name": "krkn-run-mode",


unfortunately that's the wrong place, this is the dummy scenario containerfile, you have to open a PR in the krkn source repo from where the image is built:

https://github.com/krkn-chaos/krkn/blob/main/containers/krknctl-input.json

…ge Scenario struct to recieve weight and config Signed-off-by: Abhinav Sharma <abhinavs1920bpl@gmail.com>

Signed-off-by: Abhinav Sharma <abhinavs1920bpl@gmail.com>

tsebastiani · 2025-11-18T10:16:33Z

pkg/scenarioorchestrator/common_functions.go

+	var resiliencyWeights []float64
+	runStartTime := time.Now().UTC()
+
+	reportRegex := regexp.MustCompile(`^KRKN_RESILIENCY_REPORT_JSON:(.*)$`)


please move the regex in the config.json as all the other regex and const strings.

tsebastiani · 2025-11-18T10:19:27Z

pkg/scenarioorchestrator/common_functions.go

+			// inject resiliency config as env variable if provided (Use Case: Resiliency Config is provided via env variable)
+			if scenario.ResiliencyConfigPath != "" {
+				if content, err := os.ReadFile(scenario.ResiliencyConfigPath); err == nil {
+					env["RESILIENCY_CONFIG"] = string(content)


are you sure that passing the file content as it is will work in any condition? don't you think that serializing the file content in base64 for example might ensure that the file structure (including indentation, spacing quotes etc.) is safely kept?

Yes, I should be serializing the file, and I’ll make that update.

For the custom resiliency config, we could mount the entire file via a volume, but if the environment-variable approach I used doesn’t introduce any edge cases, I think the current method should be fine. Let me know if you see any problems with this.

tsebastiani · 2025-11-18T10:24:02Z

pkg/scenarioorchestrator/common_functions.go

+		}
 	}
+	runEndTime := time.Now().UTC()
+	_ = runStartTime


What's the point of these void initializations?

Signed-off-by: Abhinav Sharma <abhinavs1920bpl@gmail.com>

…dd the env variable Signed-off-by: Abhinav Sharma <abhinavs1920bpl@gmail.com>

Signed-off-by: Abhinav Sharma <abhinavs1920bpl@gmail.com>

…_ALERTS_YAML_CONTENT, remove ResiliencyConfig from Scenario, serialize ResiliencyConfig YAML file to base64 and inject it as env variable Signed-off-by: Abhinav Sharma <abhinavs1920bpl@gmail.com>

Signed-off-by: Abhinav Sharma <abhinavs1920bpl@gmail.com>

abhinavs1920 · 2025-11-24T05:58:44Z

Hi everyone,

Following up on the krkn PR....

Summary of changes:

Controller Mode: It runs the krkn containers in "controller" mode so they emit the raw JSON reports to stdout.
Aggregation: It scrapes those reports via regex, matches them to the graph node, and applies the weights defined in the scenario file (defaults to 1 if nothing is set).
Final Score: It calculates the final, global weighted average and produces the consolidated resiliency-report.json.

In krknctl you have to provide the prometheus url in the env, it will enable the resiliency feature automatically.

Let me know what you think about its working!

tsebastiani · 2025-11-25T10:04:04Z

cmd/run.go

+					return cerr
+				}
+				defer file.Close()
+				mw := io.MultiWriter(os.Stdout, file)


@abhinavs1920 please add a comment here to explain how the thing works because might be confusing. Explain that you're multiplexing a file and stdout to with a multiwriter and that the container stdout and stderr will written in the machine stdout and in the file. Thanks!

tsebastiani · 2025-11-25T10:06:30Z

pkg/scenarioorchestrator/common_functions.go

+			if prom, ok := env["PROMETHEUS_URL"]; ok {
+				promURL = prom
+			}
+			env["RESILIENCY_ENABLED_MODE"] = resiliency.ComputeResiliencyMode(promURL, config)


Please add these string constraints like RESILIENCY_ENABLED_MODE KRKN_ALERTS_YAML_CONTENT etc. to the config.

tsebastiani · 2025-11-25T10:08:41Z

pkg/resiliency/resiliency.go

+
+func PrintHumanSummary(report FinalReport) {
+	if data, err := json.MarshalIndent(report, "", "  "); err == nil {
+		fmt.Printf("Overall Resiliency Report Summary:\n%s\n", string(data))


logs must be lowercased (except for acronyms)

tsebastiani · 2025-11-25T10:09:15Z

pkg/resiliency/resiliency.go

+	}
+
+	// 2. Nested under telemetry.overall_resiliency_report.
+	type root2 struct {


do not use numbers in types and variable names

tsebastiani · 2025-11-25T10:09:59Z

cmd/utils.go

+			promURL = prom.value
+		}
+		mode := resiliency.ComputeResiliencyMode(promURL, cfg)
+		environment["RESILIENCY_ENABLED_MODE"] = ParsedField{value: mode, secret: false}


RESILIENCY_ENABLED_MODE must be in the config file

tsebastiani · 2025-11-25T10:17:28Z

cmd/run.go


 			if !runDetached {
+				// capture stdout/stderr to both terminal and a log file for parsing
+				filename := fmt.Sprintf("%s.log", containerName)


⚠️ The requirement was that in attached mode the krkn output should have been written on a bytes.Buffer, not in a file.

tsebastiani · 2025-11-25T10:18:26Z

pkg/resiliency/resiliency.go

+			Overall OverallResiliencyReport `json:"overall_resiliency_report"`
+		} `json:"telemetry"`
+	}
+	var r2 root2


please do not use numbers in variables and package aliases

tsebastiani · 2025-11-25T10:18:43Z

pkg/resiliency/resiliency.go

+	}
+
+	// 3. As a map of scenario scores at root with optional aggregate values.
+	type root3 struct {


please do not use numbers in variables and package aliases

tsebastiani · 2025-11-25T10:18:55Z

pkg/resiliency/resiliency.go

+		PassedSlos      int                `json:"passed_slos"`
+		TotalSlos       int                `json:"total_slos"`
+	}
+	var r3 root3


please do not use numbers in variables and package aliases

tsebastiani · 2025-11-25T10:19:01Z

pkg/resiliency/resiliency.go

+			Failed int `json:"failed"`
+		} `json:"breakdown"`
+	}
+	type root4 struct {


please do not use numbers in variables and package aliases

Signed-off-by: Abhinav Sharma <abhinavs1920bpl@gmail.com>

…ruct names Signed-off-by: Abhinav Sharma <abhinavs1920bpl@gmail.com>

feat: add krkn-run-mode env variable in krknctl-input.json

d48aa6f

Signed-off-by: Abhinav Sharma <abhinavs1920bpl@gmail.com>

chore: add resiliency config to scenario

3f0b9e7

Signed-off-by: Abhinav Sharma <abhinavs1920bpl@gmail.com>

tsebastiani reviewed Nov 6, 2025

View reviewed changes

abhinavs1920 added 2 commits November 8, 2025 13:27

feat: receive resiliency report from scenario container logs and chan…

ac5c0f3

…ge Scenario struct to recieve weight and config Signed-off-by: Abhinav Sharma <abhinavs1920bpl@gmail.com>

feat: Add resiliency data and functions in resiliency package

98be58e

Signed-off-by: Abhinav Sharma <abhinavs1920bpl@gmail.com>

tsebastiani reviewed Nov 18, 2025

View reviewed changes

abhinavs1920 force-pushed the resiliency branch from 214d70a to 98be58e Compare November 18, 2025 10:27

abhinavs1920 added 9 commits November 18, 2025 15:58

fix: remove wrong placed env variable

ed25086

Signed-off-by: Abhinav Sharma <abhinavs1920bpl@gmail.com>

chore: Update common_functions.go, implement use of resiliency package

241b439

Signed-off-by: Abhinav Sharma <abhinavs1920bpl@gmail.com>

chore: add resiliencyMode variable in config and update utils.go to a…

9c1f408

…dd the env variable Signed-off-by: Abhinav Sharma <abhinavs1920bpl@gmail.com>

chore: create GenerateAndWriteReport function for reusability

39bf6ef

Signed-off-by: Abhinav Sharma <abhinavs1920bpl@gmail.com>

feat: implement resiliency report generation for attach mode

90cb547

Signed-off-by: Abhinav Sharma <abhinavs1920bpl@gmail.com>

chore: remove RESILIENCY_CONFIG env variable and replace it with KRKN…

17eaf37

…_ALERTS_YAML_CONTENT, remove ResiliencyConfig from Scenario, serialize ResiliencyConfig YAML file to base64 and inject it as env variable Signed-off-by: Abhinav Sharma <abhinavs1920bpl@gmail.com>

fix: correct regex for resiliency yaml content

c0b2e2e

Signed-off-by: Abhinav Sharma <abhinavs1920bpl@gmail.com>

fix: compute resiliency mode based on prometheus url presence

5a08d45

Signed-off-by: Abhinav Sharma <abhinavs1920bpl@gmail.com>

feat: add weighted calculation of resiliency score

c38bf1d

Signed-off-by: Abhinav Sharma <abhinavs1920bpl@gmail.com>

tsebastiani reviewed Nov 25, 2025

View reviewed changes

abhinavs1920 added 2 commits December 2, 2025 18:54

fix: use bytes.buffer for writing logs in attached mode

4125210

Signed-off-by: Abhinav Sharma <abhinavs1920bpl@gmail.com>

fix: remove minor issues like logging format, add proper comments, st…

f636430

…ruct names Signed-off-by: Abhinav Sharma <abhinavs1920bpl@gmail.com>

tsebastiani force-pushed the main branch from 64261f5 to d8e33e6 Compare February 9, 2026 15:19

Conversation

abhinavs1920 commented Nov 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Documentation

Related Documentation PR (if applicable)

Uh oh!

abhinavs1920 commented Nov 6, 2025

Uh oh!

abhinavs1920 commented Nov 6, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

abhinavs1920 commented Nov 24, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tsebastiani Nov 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

abhinavs1920 commented Nov 5, 2025 •

edited

Loading

tsebastiani Nov 25, 2025 •

edited

Loading