Skip to content

Commit 0cb40c9

Browse files
chore(skills): consolidate updates across k6 skill packs
1 parent d048e15 commit 0cb40c9

10 files changed

Lines changed: 205 additions & 23 deletions

File tree

skills/k6-config/SKILL.md

Lines changed: 45 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -21,11 +21,27 @@ At the beginning of the workflow, detect and use interaction tools in this order
2121
4. Else emit the exact fallback and end the turn:
2222

2323
```md
24-
> [?] MISSING REQUIREMENT: [Missing required configuration detail]
24+
> [?] MISSING REQUIREMENT: Missing required configuration detail
25+
required: environment set, load profile inputs, and SLA target
26+
why: deterministic multi-environment config cannot be generated safely
27+
next_question: Which environments should be generated (dev/staging/prod) and what SLA should apply?
2528
```
2629

2730
Do not continue generation after fallback.
2831

32+
## Interoperability Fallback Contract
33+
34+
When fallback is required, always use this portable payload shape:
35+
36+
```md
37+
> [?] MISSING REQUIREMENT: <short missing requirement summary>
38+
required: <comma-separated missing fields>
39+
why: <why generation cannot continue deterministically>
40+
next_question: <single question that unblocks next step>
41+
```
42+
43+
Do not emit final configuration output after this fallback.
44+
2945
## Language Policy
3046

3147
1. If user language is explicit, answer in that language.
@@ -42,6 +58,12 @@ Always enforce these validations before returning configuration output:
4258
2. **Load profile is required**
4359
- Every generated environment config must include explicit `vus` and `duration`.
4460
- If missing, derive defaults per environment and state derivation logic.
61+
3. **Parameter coherence is required**
62+
- If arrival-rate style parameters are included, enforce `preAllocatedVUs <= maxVUs`.
63+
- If `stages` are used, ensure stage durations are explicit and non-empty.
64+
4. **Secrets and runnable safety are required**
65+
- Never hard-code credentials or tokens in runnable examples.
66+
- Require environment-variable placeholders (`__ENV` or `.env.example`) for secrets.
4567

4668
## Configuration Patterns
4769

@@ -73,6 +95,25 @@ Generate separate configs for dev/staging/prod with environment-specific:
7395
- `k6-config` applies environment-level policy (dev/staging/prod) and operational defaults.
7496
- If both are used, `k6-config` must preserve executor recommendation intent while enforcing environment safety defaults.
7597

98+
Dashboard precedence order shared with `k6-executor`:
99+
100+
1. CI/non-interactive contexts default to `K6_WEB_DASHBOARD=false` unless user explicitly opts in.
101+
2. Local interactive browser troubleshooting defaults to `K6_WEB_DASHBOARD=true`.
102+
3. Local non-browser runs default to `K6_WEB_DASHBOARD=false` unless user explicitly opts in.
103+
4. All other contexts default to `K6_WEB_DASHBOARD=false`.
104+
105+
## Output Contract
106+
107+
Every response must include these sections in order:
108+
109+
1. Environment Matrix
110+
2. Configuration Snippets (by environment)
111+
3. Thresholds (provided or derived)
112+
4. Load Profile (explicit `vus`/`duration` or staged equivalent)
113+
5. Guardrail Validation
114+
6. Web Dashboard Policy
115+
7. Assumptions and Next Step
116+
76117
## Progressive Disclosure
77118

78119
Keep this file focused on execution workflow. Place deep guidance in:
@@ -85,6 +126,7 @@ Keep this file focused on execution workflow. Place deep guidance in:
85126
2. Run Tool Discovery Protocol if critical input is missing.
86127
3. Validate or derive thresholds for each environment.
87128
4. Validate or derive `vus` and `duration` for each environment.
88-
5. Determine per-environment dashboard policy (`enable` only when explicitly requested for local analysis, or when the scenario requires browser UX troubleshooting differential analysis), aligned with `k6-executor` gate semantics.
129+
5. Determine per-environment dashboard policy using shared precedence with `k6-executor` (CI/headless first, local-browser second, default false otherwise).
89130
6. Generate deterministic config output and `.env.example` guidance, explicitly warning that real env files and generated reports must remain uncommitted.
90-
7. Include a short summary of derived assumptions.
131+
7. Validate guardrails and return output using the Output Contract section order.
132+
8. Include a short summary of derived assumptions.

skills/k6-config/references/sla-defaults.md

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -84,7 +84,9 @@ Output:
8484
}
8585
```
8686

87-
> **Note on p99 inference:** `p99 = p95 * 1.8` is an empirical heuristic, not an official k6 rule. Use measured p99 values from representative environments whenever possible.
87+
> **IMPORTANT: Note on p99 Inference:** The `p99 = p95 * 1.8` formula is an empirical heuristic based on common latency distributions. This is not an official k6 standard and must not be treated as universally valid.
88+
>
89+
> Use this approximation only as an initial baseline when real p99 measurements are unavailable. Replace it with measured p99 values from representative environments (staging or production-like) before final SLA sign-off.
8890
8991
## Protocol-Specific Metrics
9092

skills/k6-executor/SKILL.md

Lines changed: 48 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -21,11 +21,27 @@ At the beginning of the workflow, detect and use interaction tools in this order
2121
4. Else emit the exact fallback and end the turn:
2222

2323
```md
24-
> [?] MISSING REQUIREMENT: [Missing goal or execution constraints]
24+
> [?] MISSING REQUIREMENT: Missing goal or execution constraints
25+
required: goal, load-shape constraint, and execution context
26+
why: deterministic executor recommendation requires all three
27+
next_question: What is your primary goal and is it rate-controlled or VU-controlled?
2528
```
2629

2730
Do not continue recommendation after fallback.
2831

32+
## Interoperability Fallback Contract
33+
34+
When fallback is required, always use this portable payload shape:
35+
36+
```md
37+
> [?] MISSING REQUIREMENT: <short missing requirement summary>
38+
required: <comma-separated missing fields>
39+
why: <why recommendation cannot continue deterministically>
40+
next_question: <single question that unblocks next step>
41+
```
42+
43+
Do not emit final recommendation content after this fallback.
44+
2945
## Language Policy
3046

3147
1. If user language is explicit, answer in that language.
@@ -47,12 +63,24 @@ Always enforce these validations before final recommendation:
4763
3. **Parameter coherence is required**
4864
- Arrival-rate executors must satisfy `preAllocatedVUs <= maxVUs`.
4965
- `duration` values must be explicit and valid for time-based executors.
66+
- `constant-vus` must include explicit `vus` and `duration`.
67+
- `ramping-vus` must include explicit `startVUs` and non-empty `stages`.
68+
- `ramping-arrival-rate` must include `startRate`, `timeUnit`, non-empty `stages`, and valid capacity controls.
69+
- `per-vu-iterations` and `shared-iterations` must include explicit `vus` and `iterations`.
5070
- `externally-controlled` recommendations must include execution-context assumptions.
71+
4. **Secrets and runnable safety are required**
72+
- Never hard-code credentials or tokens in runnable snippets.
73+
- Require environment variables (`__ENV`) when auth or secrets are needed.
5174

5275
## Decision Tree
5376

5477
<decision-tree>
55-
Ask user three clarifying questions if `goal` parameter is incomplete:
78+
Round definition:
79+
80+
- **Round 1 (baseline block)**: ask the three baseline questions as one consolidated block.
81+
- **Round 2 (tie-break only)**: ask at most one tie-break question if conflict remains.
82+
83+
Ask user the baseline questions if `goal` parameter is incomplete:
5684

5785
1. **Do you need to control VU count or request rate?**
5886
- VU count → constant-vus or ramping-vus
@@ -66,7 +94,7 @@ Ask user three clarifying questions if `goal` parameter is incomplete:
6694
- Time-based → use duration parameter
6795
- Iteration-based → per-vu-iterations or shared-iterations
6896

69-
If user requirements conflict (for example strict RPS target and strict VU cap), ask one tie-break question:
97+
If user requirements conflict (for example strict RPS target and strict VU cap), ask one tie-break question in Round 2:
7098

7199
- "Which is more critical for this run: exact request-rate target or strict virtual-user ceiling?"
72100
</decision-tree>
@@ -195,10 +223,11 @@ Do not silently replace explicit user SLAs with defaults.
195223

196224
Apply this gate before final recommendation:
197225

198-
1. If scenario involves browser UX troubleshooting or local interactive analysis, recommend `K6_WEB_DASHBOARD=true`.
199-
2. If scenario is CI/non-interactive, keep dashboard disabled by default and prefer exported summaries.
200-
3. For all other cases, default to disabled unless user explicitly asks for interactive local monitoring.
201-
4. State one deterministic dashboard recommendation: `K6_WEB_DASHBOARD=true` or `K6_WEB_DASHBOARD=false` with rationale.
226+
1. If scenario is CI/non-interactive, keep dashboard disabled by default (`K6_WEB_DASHBOARD=false`) and prefer exported summaries.
227+
2. If scenario involves local interactive browser troubleshooting, recommend `K6_WEB_DASHBOARD=true`.
228+
3. If scenario is local non-browser, default to disabled (`K6_WEB_DASHBOARD=false`) unless user explicitly asks for interactive monitoring.
229+
4. For all other cases, default to disabled unless user explicitly asks for interactive local monitoring.
230+
5. State one deterministic dashboard recommendation: `K6_WEB_DASHBOARD=true` or `K6_WEB_DASHBOARD=false` with rationale.
202231

203232
Always emit a visible section in output:
204233

@@ -220,7 +249,17 @@ Every recommendation response must include these sections in order:
220249
5. Web Dashboard Recommendation
221250
6. Next Step
222251

223-
For arrival-rate executors, Guardrail Validation must include:
252+
Guardrail Validation must include executor-specific checks:
253+
254+
- `constant-vus`: explicit `vus` and `duration`
255+
- `ramping-vus`: explicit `startVUs` and non-empty `stages`
256+
- `constant-arrival-rate`: explicit `rate`, `timeUnit`, `duration`, and `preAllocatedVUs <= maxVUs`
257+
- `ramping-arrival-rate`: explicit `startRate`, `timeUnit`, non-empty `stages`, and `preAllocatedVUs <= maxVUs`
258+
- `per-vu-iterations`: explicit `vus` and `iterations`
259+
- `shared-iterations`: explicit `vus` and `iterations`
260+
- `externally-controlled`: explicit `vus`, `maxVUs`, `duration`, execution-context assumption, and documented control workflow
261+
262+
For arrival-rate executors, Guardrail Validation must also include:
224263

225264
- `preAllocatedVUs <= maxVUs` check
226265
- explicit capacity assumption note
@@ -267,7 +306,7 @@ Keep this file focused on decision workflow. Place deep guidance in:
267306
2. Select response mode (brief or detailed).
268307
3. Run Tool Discovery Protocol if required inputs are missing.
269308
4. If explicit SLAs are present, apply SLA Reconfirmation Rule.
270-
5. If ambiguous or conflicting, ask decision-tree questions (max two rounds).
309+
5. If ambiguous or conflicting, run decision-tree rounds with the explicit contract (Round 1 baseline, Round 2 tie-break only).
271310
6. Map answers to the most appropriate executor.
272311
7. Validate thresholds, load-profile invariants, and parameter coherence.
273312
8. Apply Web Dashboard Recommendation Gate and emit it visibly.

skills/k6-plan/SKILL.md

Lines changed: 45 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -21,11 +21,27 @@ At the beginning of the workflow, detect and use interaction tools in this order
2121
4. Else emit the exact fallback and end the turn:
2222

2323
```md
24-
> [?] MISSING REQUIREMENT: [Missing target, scenario, or SLA detail]
24+
> [?] MISSING REQUIREMENT: Missing target, scenario, or SLA detail
25+
required: target, scenario type, and SLA requirements
26+
why: deterministic planning cannot proceed without baseline planning inputs
27+
next_question: What target endpoint, scenario type, and SLA should this plan use?
2528
```
2629

2730
Do not continue plan generation after fallback.
2831

32+
## Interoperability Fallback Contract
33+
34+
When fallback is required, always use this portable payload shape:
35+
36+
```md
37+
> [?] MISSING REQUIREMENT: <short missing requirement summary>
38+
required: <comma-separated missing fields>
39+
why: <why plan generation cannot continue deterministically>
40+
next_question: <single question that unblocks next step>
41+
```
42+
43+
Do not emit final plan content after this fallback.
44+
2945
## Core Rules
3046

3147
<rules>
@@ -59,6 +75,12 @@ Do not continue plan generation after fallback.
5975
5. **Determinism**: Same inputs produce identical outputs every time.
6076
</rules>
6177

78+
## Terminology Contract
79+
80+
- **Scenario type** means the test objective shape (`load`, `stress`, `spike`, `soak`, `smoke`).
81+
- **Profile** means default intensity presets (`minimal`, `standard`, `aggressive`) used when explicit `vus`/`duration` are missing.
82+
- Scenario type selects the executor strategy; profile sets default intensity values.
83+
6284
## Language Policy
6385

6486
1. If user language is explicit, answer in that language.
@@ -96,6 +118,24 @@ Always enforce these validations before returning the plan:
96118
- Do not generate runnable scripts with fixed live target URLs.
97119
- Require `__ENV.BASE_URL` (or equivalent) for executable output.
98120
- If target is missing, ask for it instead of using a default live URL.
121+
4. **Parameter coherence is required**
122+
- Derived or explicit profile values must map to explicit `vus` and `duration`, or explicit staged equivalents.
123+
- If write methods are planned (`POST`/`PUT`/`PATCH`), payload assumptions and expected status must be explicit.
124+
5. **Secrets and runnable safety are required**
125+
- Never hard-code credentials or tokens in runnable snippets.
126+
- Require environment variables (`__ENV`) for auth inputs.
127+
128+
## Output Contract
129+
130+
Every response must include these sections in order:
131+
132+
1. Planning Inputs Summary
133+
2. Executor Recommendation
134+
3. Load Profile (explicit or derived)
135+
4. Thresholds (SLA-derived or defaults)
136+
5. Protocol-Specific Notes
137+
6. Assumptions
138+
7. Next recommended step
99139

100140
## Scenario to Executor Mapping
101141

@@ -183,6 +223,7 @@ When user invokes this skill:
183223
9. Parse SLA thresholds or apply deterministic defaults.
184224
10. Validate explicit or derived VUs and duration.
185225
11. Generate textual plan with recommendations.
186-
12. Add exactly one deterministic `Next recommended step` based on first unresolved dependency.
187-
13. If `output=script` or user explicitly requests code, generate complete k6 JavaScript.
188-
14. Return the plan and assumptions summary.
226+
12. Validate output structure using the Output Contract section order.
227+
13. Add exactly one deterministic `Next recommended step` based on first unresolved dependency.
228+
14. If `output=script` or user explicitly requests code, generate complete k6 JavaScript.
229+
15. Return the plan and assumptions summary.

skills/k6-plan/references/README.md

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,13 @@ Complete reference materials for comprehensive k6 performance test planning.
1919
- HTTP method confirmation and auth discovery for executable plans
2020
- Deterministic next-step selection from unresolved dependencies
2121

22+
## Terminology Contract
23+
24+
- **Scenario type**: objective shape (`load`, `stress`, `spike`, `soak`, `smoke`) used to select executor strategy.
25+
- **Profile**: intensity preset (`minimal`, `standard`, `aggressive`) used to derive default `vus` and `duration` when missing.
26+
- **Round**: one consolidated question block in the same adaptive question system; baseline questions are Round 1 and optional tie-break is Round 2.
27+
- **Explicit** means user-provided; **derived** means deterministic defaults declared in assumptions.
28+
2229
## Implementation Rules
2330

2431
1. **Plans must include explicit thresholds** — provided by user or derived from defaults

skills/k6-plan/references/protocol-guide.md

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,13 @@
22

33
## HTTP
44

5+
### Timeout Coherence Rule
6+
7+
1. Include explicit request timeout for executable HTTP examples (`timeout: '30s'` baseline).
8+
2. Use stricter values only when SLA requires them and document the rationale.
9+
3. Missing explicit timeout should be treated as a validation warning in `k6-validate`, not an automatic blocker.
10+
4. Plan and validation guidance must not contradict: if plan emits explicit timeout, validation should not warn on timeout absence.
11+
512
### Method Confirmation Checklist
613

714
Before generating executable HTTP scripts:

skills/k6-validate/SKILL.md

Lines changed: 35 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -21,11 +21,27 @@ At the beginning of the workflow, detect and use interaction tools in this order
2121
4. Else emit the exact fallback and end the turn:
2222

2323
```md
24-
> [?] MISSING REQUIREMENT: [Missing script path or validation scope]
24+
> [?] MISSING REQUIREMENT: Missing script path or validation scope
25+
required: script path and validation scope (protocol or expected profile)
26+
why: deterministic validation report cannot run without target and scope
27+
next_question: Which script should be validated and what protocol/profile context applies?
2528
```
2629

2730
Do not continue validation after fallback.
2831

32+
## Interoperability Fallback Contract
33+
34+
When fallback is required, always use this portable payload shape:
35+
36+
```md
37+
> [?] MISSING REQUIREMENT: <short missing requirement summary>
38+
required: <comma-separated missing fields>
39+
why: <why validation cannot continue deterministically>
40+
next_question: <single question that unblocks next step>
41+
```
42+
43+
Do not emit final validation findings after this fallback.
44+
2945
## Language Policy
3046

3147
1. If user language is explicit, answer in that language.
@@ -72,6 +88,23 @@ Always enforce these validations as mandatory checks:
7288
2. **Load profile is required**
7389
- Flag as error when no explicit load profile exists.
7490
- Require explicit `vus` and `duration` for time-based cases, or clear equivalent (`stages`, `iterations` + `vus`) for scenario-based definitions.
91+
3. **Parameter coherence is required**
92+
- If arrival-rate parameters exist, validate `preAllocatedVUs <= maxVUs`.
93+
- If staged scenarios exist, validate non-empty stages with explicit duration per stage.
94+
4. **Secrets and runnable safety are required**
95+
- Flag hard-coded credentials/tokens as error.
96+
- Flag insecure runnable defaults for secrets as error or warning based on impact.
97+
98+
## Output Contract
99+
100+
Every validation response must include these sections in order:
101+
102+
1. Validation Summary (`pass`/`warn`/`fail`)
103+
2. Scope and Assumptions
104+
3. Mandatory Invariant Results
105+
4. Detailed Findings
106+
5. Suggested Fixes
107+
6. Next Step
75108

76109
## Progressive Disclosure
77110

@@ -87,4 +120,4 @@ Keep this file focused on validation workflow. Place deep guidance in:
87120
4. Validate performance best practices and protocol-specific rules.
88121
5. Enforce required threshold and load-profile invariants.
89122
6. Run quality-hardening checks (silent catch, unsafe parse, static-analysis signals).
90-
7. Return deterministic report with pass/warn/fail, findings, and fixes.
123+
7. Return deterministic report using the Output Contract section order.

skills/k6-validate/references/load-profile-validation-rules.md

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -190,8 +190,9 @@ stages: [
190190
- **Behavior:** Iterations split across VUs
191191

192192
### externally-controlled
193-
- **Required:** None (controlled via k6 REST API)
194-
- **Typical:** REST calls to set VUs and runState
193+
- **Required:** Explicit `vus`, `maxVUs`, `duration`, and execution-context assumption (control workflow documented externally)
194+
- **Typical:** REST calls to set VUs and runState post-initialization
195+
- **Note:** Aligns with k6-executor guardrail requiring explicit baseline parameters and documented external control context
195196

196197
## Edge Cases
197198

skills/k6-validate/references/severity-and-antipatterns.md

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,7 @@
2424
| Scenario | Rule | Severity | Fix |
2525
|----------|------|----------|-----|
2626
| No thresholds defined | ERROR | MUST have at least one threshold | Add threshold (e.g., `p95<500`) |
27-
| Only success checks, no metrics | WARNING | Thresholds define success criteria (MOS - Measure of Success), not just checks | Add latency/availability thresholds |
27+
| Only checks pass-rate threshold, no latency/error metrics | WARNING | Checks alone are insufficient for SLA validation | Add latency and error-rate thresholds |
2828
| Inconsistent across environments | WARNING | Same test, different thresholds per env | Document environment-specific SLAs |
2929
| p95 > p99 | ERROR | Invalid percentile relationship (p95 must be ≤ p99) | Swap values or fix percentile order |
3030
| Error threshold > 5% | WARNING | Too lenient for most SLAs | Tighten error rate threshold |
@@ -49,6 +49,10 @@
4949
| Batch size > 10 | WARNING | Batch requests might mask individual failures | Reduce batch size or use separate calls |
5050
| No timeout defined | WARNING | No per-request timeout is defined; behavior falls back to k6 defaults that may not match SLA expectations | Set `timeout: '30s'` or similar |
5151

52+
Timeout coherence note:
53+
54+
- When a script follows approved plan/config guidance and already defines explicit timeouts, timeout warnings must not be emitted.
55+
5256
#### gRPC
5357
| Scenario | Rule | Severity | Fix |
5458
|----------|------|----------|-----|

0 commit comments

Comments
 (0)