Skip to content

Commit 792ee1b

Browse files
committed
skills: ask user for output directory before running commands
Without this, the agent picks an output directory on its own (e.g. odf/namespace-application) and the user has no way to change it. Add a "Pick an output directory" step to the validate-application, gather-application, test-run, and test-clean skills. The validate-clusters skill already had this step but used a vague <env> placeholder. All skills now suggest out/ as the base directory and ask the agent to use a new directory for each run. If the directory already exists the agent suggests a name with a datetime suffix (e.g. out/clusters-2026-07-20-19-46). Reusing a directory causes output files with numeric suffixes (-2, -3) that confuse the agent when inspecting results. Also rename <output-dir> to <output-directory> to match the step heading and reinforce that the agent should use the directory chosen in the previous step. Assisted-by: Cursor/Claude Opus 4.6 Signed-off-by: Nir Soffer <nsoffer@redhat.com>
1 parent 168597b commit 792ee1b

5 files changed

Lines changed: 63 additions & 41 deletions

File tree

pkg/skills/templates/skills/gather-application.tmpl

Lines changed: 13 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -35,30 +35,36 @@ argocd appset-deploy-rbd 69m dr1 dr2 Re
3535
Present the list and let the user choose which application to gather
3636
data for.
3737

38-
### Step 2: Run gather application
38+
### Step 2: Pick an output directory
3939

40-
Use `<env>/<namespace>-<app-name>` for the output directory, e.g., `myenv/myns-myapp`.
40+
Ask the user where to store the output. Suggest
41+
`out/<namespace>-<app-name>`, e.g., `out/myns-myapp`.
42+
43+
If the directory already exists, suggest another name such as
44+
`out/<namespace>-<app-name>-<YYYY-MM-DD-HH-MM>` (e.g., `out/myns-myapp-2026-07-20-19-46`).
45+
46+
### Step 3: Run gather application
4147

4248
```console
43-
$ {{.Command.Name}} gather application --name <drpc-name> --namespace <namespace> -o <output-dir>
49+
$ {{.Command.Name}} gather application --name <drpc-name> --namespace <namespace> -o <output-directory>
4450
```
4551

4652
Use `--config <file>` if the config file is not the default `config.yaml`.
4753

4854
The command takes a few seconds on local clusters and about a minute on
4955
remote clusters.
5056

51-
### Step 3: Check the result
57+
### Step 4: Check the result
5258

5359
On success the command ends with:
5460

5561
```console
5662
✅ Gather completed
5763
```
5864

59-
On failure, check `<output-dir>/gather-application.log` for details.
65+
On failure, check `<output-directory>/gather-application.log` for details.
6066

61-
### Step 4: Inspect gathered data
67+
### Step 5: Inspect gathered data
6268

6369
The output directory contains:
6470

@@ -76,7 +82,7 @@ resources. Ramen operator logs are gathered under
7682
`namespaces/ramen-system/pods/`.
7783

7884
```console
79-
$ tree -L3 <output-dir>/gather-application.data
85+
$ tree -L3 <output-directory>/gather-application.data
8086
gather-application.data/
8187
├── dr1/
8288
│ ├── cluster/

pkg/skills/templates/skills/test-clean.tmpl

Lines changed: 7 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -23,20 +23,23 @@ Delete resources created by `{{.Command.Name}} test run`.
2323

2424
## Workflow
2525

26-
### Step 1: Run test clean
26+
### Step 1: Pick an output directory
2727

28-
Use the same output directory and config file as the test run:
28+
Ask the user where to store the report. Suggest using the same output
29+
directory as the test run, but a different directory also works.
30+
31+
### Step 2: Run test clean
2932

3033
```console
31-
$ {{.Command.Name}} test clean -o <output-dir>
34+
$ {{.Command.Name}} test clean -o <output-directory>
3235
```
3336

3437
Use `--config <file>` if the config file is not the default `config.yaml`.
3538

3639
The command unprotects and undeploys test applications, then cleans the
3740
test environment (channels, namespaces).
3841

39-
### Step 2: Check the result
42+
### Step 3: Check the result
4043

4144
**On success:**
4245

pkg/skills/templates/skills/test-run.tmpl

Lines changed: 15 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -22,12 +22,17 @@ in the configuration file.
2222

2323
## Workflow
2424

25-
### Step 1: Run the test
25+
### Step 1: Pick an output directory
2626

27-
Use `<env>/test` for the output directory, e.g., `myenv/test`.
27+
Ask the user where to store the report. Suggest `out/test`.
28+
29+
If the directory already exists, suggest another name such as
30+
`out/test-<YYYY-MM-DD-HH-MM>` (e.g., `out/test-2026-07-20-19-46`).
31+
32+
### Step 2: Run the test
2833

2934
```console
30-
$ {{.Command.Name}} test run -o <output-dir>
35+
$ {{.Command.Name}} test run -o <output-directory>
3136
```
3237

3338
Use `--config <file>` if the config file is not the default `config.yaml`.
@@ -38,7 +43,7 @@ deploy, protect, failover, relocate, unprotect, undeploy.
3843
**This typically takes 15-20 minutes per test case.** Multiple test cases
3944
run in parallel.
4045

41-
### Step 2: Check the result
46+
### Step 3: Check the result
4247

4348
**On success:**
4449

@@ -53,7 +58,7 @@ clusters and S3 into `test-run.data/`:
5358
❌ failed (N passed, M failed, 0 skipped)
5459
```
5560

56-
### Step 3: Inspect the report
61+
### Step 4: Inspect the report
5762

5863
The output directory contains:
5964

@@ -68,25 +73,25 @@ Useful commands:
6873
Overall status:
6974

7075
```console
71-
$ yq '.status' < <output-dir>/test-run.yaml
76+
$ yq '.status' < <output-directory>/test-run.yaml
7277
```
7378

7479
Individual step status and durations:
7580

7681
```console
77-
$ yq '.steps[-1].items' < <output-dir>/test-run.yaml
82+
$ yq '.steps[-1].items' < <output-directory>/test-run.yaml
7883
```
7984

8085
Test events from the log:
8186

8287
```console
83-
$ grep -E '(INFO|ERROR).+<app-name>' <output-dir>/test-run.log
88+
$ grep -E '(INFO|ERROR).+<app-name>' <output-directory>/test-run.log
8489
```
8590

8691
The `test-run.yaml` report contains the config used, each test step with
8792
its status and duration, and a summary with pass/fail/skip counts.
8893

89-
### Step 4: Handle failures
94+
### Step 5: Handle failures
9095

9196
If the test failed:
9297

@@ -98,7 +103,7 @@ If the test failed:
98103
4. Fix the underlying issue
99104
5. **Always clean up** before re-running (see {{.Command.Slug}}-test-clean skill)
100105

101-
### Step 5: Clean up
106+
### Step 6: Clean up
102107

103108
Ask the user if they want to clean up. Do not clean up without an
104109
explicit request — failed test resources are evidence for debugging.

pkg/skills/templates/skills/validate-application.tmpl

Lines changed: 20 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -38,12 +38,18 @@ Present the list and let the user choose which application to validate.
3838
The `NAME` and `NAMESPACE` columns map to the `--name` and `--namespace`
3939
flags.
4040

41-
### Step 2: Run validate application
41+
### Step 2: Pick an output directory
4242

43-
Use `<env>/<namespace>-<app-name>` for the output directory, e.g., `myenv/myns-myapp`.
43+
Ask the user where to store the report. Suggest
44+
`out/<namespace>-<app-name>`, e.g., `out/myns-myapp`.
45+
46+
If the directory already exists, suggest another name such as
47+
`out/<namespace>-<app-name>-<YYYY-MM-DD-HH-MM>` (e.g., `out/myns-myapp-2026-07-20-19-46`).
48+
49+
### Step 3: Run validate application
4450

4551
```console
46-
$ {{.Command.Name}} validate application --name <drpc-name> --namespace <namespace> -o <output-dir> --interactive=false
52+
$ {{.Command.Name}} validate application --name <drpc-name> --namespace <namespace> -o <output-directory> --interactive=false
4753
```
4854

4955
`--interactive=false` prevents {{.Command.Name}} from opening a browser—you open the
@@ -54,7 +60,7 @@ Use `--config <file>` if the config file is not the default `config.yaml`.
5460
The command takes a few seconds on local clusters and about a minute on
5561
remote clusters.
5662

57-
### Step 3: Check the result
63+
### Step 4: Check the result
5864

5965
**On success:**
6066

@@ -73,16 +79,16 @@ there are problems **in** that report. If the command **fails before** a
7379
report is written (early error), there may be no HTML—use the log and console
7480
output instead.
7581

76-
### Step 4: Open the HTML report
82+
### Step 5: Open the HTML report
7783

78-
If `<output-dir>/validate-application.html` exists, open it in the
84+
If `<output-directory>/validate-application.html` exists, open it in the
7985
browser using `open <path>` (macOS), `xdg-open <path>` (Linux), or
8086
`cmd /c start "" <path>` (Windows). Do not skip unless the user asks
8187
to skip.
8288

8389
The report may not exist if the command failed before writing it.
8490

85-
### Step 5: Inspect the report
91+
### Step 6: Inspect the report
8692

8793
The output directory contains:
8894

@@ -96,13 +102,13 @@ The output directory contains:
96102
Read the application status:
97103

98104
```console
99-
$ yq '.applicationStatus' < <output-dir>/validate-application.yaml
105+
$ yq '.applicationStatus' < <output-directory>/validate-application.yaml
100106
```
101107

102108
#### Gathered data structure
103109

104110
```console
105-
$ tree -L3 <output-dir>/validate-application.data
111+
$ tree -L3 <output-directory>/validate-application.data
106112
validate-application.data/
107113
├── <cluster>/
108114
│ ├── cluster/
@@ -117,7 +123,7 @@ validate-application.data/
117123
└── <app-namespace>/
118124
```
119125

120-
### Step 6: Troubleshoot problems
126+
### Step 7: Troubleshoot problems
121127

122128
This section is an **initial draft** — DR troubleshooting is hard and
123129
needs more playbooks over time (for example a future analyze skill).
@@ -134,20 +140,20 @@ Useful commands for inspecting gathered data:
134140
Check VRG status:
135141

136142
```console
137-
$ yq '.status' < <output-dir>/validate-application.data/<cluster>/namespaces/<ns>/ramendr.openshift.io/volumereplicationgroups/<name>.yaml
143+
$ yq '.status' < <output-directory>/validate-application.data/<cluster>/namespaces/<ns>/ramendr.openshift.io/volumereplicationgroups/<name>.yaml
138144
```
139145

140146
Search ramen operator logs for errors related to the app:
141147

142148
```console
143-
$ grep -E 'ERROR.+<app-name>' <output-dir>/validate-application.data/<cluster>/namespaces/ramen-system/pods/*/manager/current.log
149+
$ grep -E 'ERROR.+<app-name>' <output-directory>/validate-application.data/<cluster>/namespaces/ramen-system/pods/*/manager/current.log
144150
```
145151

146-
### Step 7: Suggest validate clusters
152+
### Step 8: Suggest validate clusters
147153

148154
After completing the application validation, suggest also running
149155
`{{.Command.Name}} validate clusters` into a sibling output directory such as
150-
`<env>/clusters`. When they agree (or you run it for them), follow the
156+
`out/clusters`. When they agree (or you run it for them), follow the
151157
**{{.Command.Slug}}-validate-clusters** skill end to end.
152158

153159
Why this helps: the application report covers one DRPC, its namespaces, and

pkg/skills/templates/skills/validate-clusters.tmpl

Lines changed: 8 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -22,13 +22,15 @@ resources from all clusters, and checking that S3 endpoints are accessible.
2222

2323
### Step 1: Pick an output directory
2424

25-
Ask the user where to store the report. Suggest `<env>/clusters` or
26-
`<env>-<date>/clusters`, e.g., `myenv/clusters`.
25+
Ask the user where to store the report. Suggest `out/clusters`.
26+
27+
If the directory already exists, suggest another name such as
28+
`out/clusters-<YYYY-MM-DD-HH-MM>` (e.g., `out/clusters-2026-07-20-19-46`).
2729

2830
### Step 2: Run validate clusters
2931

3032
```console
31-
$ {{.Command.Name}} validate clusters -o <output-dir> --interactive=false
33+
$ {{.Command.Name}} validate clusters -o <output-directory> --interactive=false
3234
```
3335

3436
`--interactive=false` prevents {{.Command.Name}} from opening a browser—you open the
@@ -61,7 +63,7 @@ output instead.
6163

6264
### Step 4: Open the HTML report
6365

64-
If `<output-dir>/validate-clusters.html` exists, open it in the
66+
If `<output-directory>/validate-clusters.html` exists, open it in the
6567
browser using `open <path>` (macOS), `xdg-open <path>` (Linux), or
6668
`cmd /c start "" <path>` (Windows). Do not skip unless the user asks
6769
to skip.
@@ -82,13 +84,13 @@ The output directory contains:
8284
Read the YAML report to understand the status:
8385

8486
```console
85-
$ yq '.clustersStatus' < <output-dir>/validate-clusters.yaml
87+
$ yq '.clustersStatus' < <output-directory>/validate-clusters.yaml
8688
```
8789

8890
#### Gathered data structure
8991

9092
```console
91-
$ tree -L3 <output-dir>/validate-clusters.data
93+
$ tree -L3 <output-directory>/validate-clusters.data
9294
validate-clusters.data/
9395
├── <cluster>/
9496
│ ├── cluster/

0 commit comments

Comments
 (0)