Skip to content

Latest commit

 

History

History
102 lines (81 loc) · 4.91 KB

traffic.md

File metadata and controls

102 lines (81 loc) · 4.91 KB

Traffic

labctl traffic runs a k6 load generator in-cluster so scenarios, incidents and autoscaling play out under realistic load. The generator lives in its own traffic namespace; the scripts are in src/services/traffic/.

Never use a curl loop in scenario or incident content - use this.

start, stop and status are recorded operations. Each prints its run ID, streams its output and appears in labctl runs list. Read an operation's saved output with labctl runs logs <id> or cancel it with labctl runs cancel <id>. Ctrl-C cancels the operation being followed.

Start and stop use the run engine's traffic lock; status takes no lock. The operation timeout is two minutes for start and stop, and 30 seconds for status. These limits apply to the command, not to the k6 Job's duration.

Each CLI engine preserves other live owners. A status command in a second terminal leaves an active traffic operation running; another start or stop reports the run holding the traffic key. See the recovery and cancellation limits before cancelling or explicitly releasing interrupted work.

labctl traffic start

Starts the generator, replacing any active run.

After start returns, the k6 Job keeps running in the cluster. Use labctl traffic stop to remove it. Cancelling a start during setup or the readiness wait removes the Job it creates and any namespace or ConfigMap it creates. A namespace or ConfigMap that already exists is preserved. Cleanup is bounded; if the cluster does not answer, the transcript names what could not be removed so you can inspect it before starting again.

labctl traffic start                                # steady 10 rps for 10m against go-api
labctl traffic start --app echo-server --rps 25     # the same against echo-server, in-cluster
labctl traffic start --profile spike --rps 20       # 20 rps baseline, 200 rps spike
labctl traffic start --profile soak --duration 4h
labctl traffic start --target http://echo-server.snowops.localhost/ --rps 50
Flag Default Meaning
--profile steady one of the profiles below
--rps 10 requests per second; the baseline for spike
--duration profile-specific run length (30s, 10m, 1h30m). steady 10m, soak 2h, spike fixed
--app go-api the workload to load, at its in-cluster root. Scenario commands pass the app they were activated for
--target the --app workload's / URL to load, overriding --app. Multi-endpoint profiles treat it as a base origin
--method profile-specific HTTP method for the write and errors profiles

Profiles

Discovered from src/services/traffic/profiles/. Every profile sends exactly one request per iteration: Application Request Metrics counts the iterations k6 drops (k6_dropped_iterations_total) as offered requests, so a new profile has to keep that one-to-one.

Profile Shape
steady constant rate, single endpoint
spike 10× burst on a fixed ~6-minute shape, single endpoint
soak long sustained run, single endpoint
browse weighted read mix across /, /version, /health
write write-heavy mix; the target is a base origin
errors deliberately error-producing mix; the target is a base origin
labctl traffic profiles
labctl traffic status      # active run, profile, pods and recent k6 output
labctl traffic stop        # stop and remove the job, configmap and namespace

Why the default target is /

The app root is access-logged and metered, so a run is visible in kubectl logs and in Grafana. Probe endpoints (/health, /ready, /metrics) are deliberately excluded from the access log and from tracing - they run on a timer forever and would bury real traffic.

k6's own metrics

start.sh looks for the Prometheus service and, when it finds one, remote-writes k6's client-side metrics (k6_http_reqs_total, k6_http_req_duration_p95, k6_http_req_failed_rate) into it. These are the other half of evaluating load: the app's /metrics only counts requests that arrived, so a gap between "k6 sent" and "app handled" on the Application Request Metrics dashboard means requests are being lost before the app sees them.

Override the endpoint with K6_PROMETHEUS_RW_SERVER_URL, or set TRAFFIC_METRICS=off to skip the probe.

k6 opens a new connection for every request (K6_NO_CONNECTION_REUSE), as a stream of users does. A kept-alive connection would keep reaching its pod after the Service stops selecting it or a NetworkPolicy starts denying new traffic, so faults like service-selector-broken and network-blackhole would never show on the k6 panels.

k6's remote-write metrics are in seconds, not milliseconds, even though k6's own CLI output is in ms. A panel labelled ms is wrong by 1000×. See R13.