Skip to content

Commit 428a050

Browse files
Miguel Medinillamamedin
authored andcommitted
feat: integrate Cookie Guard BotD telemetry
Persist BotD verdict/kind/confidence/request_id in the public session table and propagate the cached verdict plus age back to HAProxy. Document that match.botd.* rules run against the cached session snapshot so detections survive beyond Cookie Guard's 5-minute cache.
1 parent 5d76d89 commit 428a050

File tree

12 files changed

+322
-25
lines changed

12 files changed

+322
-25
lines changed

README.md

Lines changed: 26 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,7 @@ Decision maintains an in‑memory "public" session entry keyed by a durable iden
2020
- `session.public.rate` (logged as `rate`): per‑second rate over the rolling window, computed as `recent_hits / rate_window_seconds`. For per‑minute, multiply by 60.
2121
- `session.public.idle_seconds`: time since this key last appeared.
2222
- `session.public.first_path` + `first_path_deep`: first path seen for the key and a boolean indicating whether it looks like a "deep" path.
23+
- `session.public.botd_*`: the most recent FingerprintJS BotD verdict (`verdict`, `kind`, `confidence`, `request_id`) plus `botd_age_seconds`, captured whenever Cookie Guard shares telemetry. Decision keeps this alongside the session so verdicts persist beyond Cookie Guard’s 5‑minute cache, and any `match.botd.*` condition reads this cached snapshot rather than requiring the live `cookieguard.*` vars to be present on the current request.
2324

2425
Configuration knobs:
2526

@@ -278,6 +279,10 @@ The tables below list the request/response variables exchanged between HAProxy a
278279
| `cookieguard_age` | Optional | `var(txn.cookieguard.age_seconds)` | Age of the verified token |
279280
| `cookieguard_level` | Optional | `var(txn.cookieguard.challenge_level)` | Challenge tier just issued (only set during challenge) |
280281
| `cookieguard_session` | Optional | `var(txn.cookieguard.session_hmac)` | Durable hb_v3 session hash (requires issue-token) |
282+
| `botd_verdict` | Optional | `var(txn.cookieguard.botd_verdict)` | Latest Fingerprint BotD verdict (`good`, `bad`, `suspect`). |
283+
| `botd_kind` | Optional | `var(txn.cookieguard.botd_kind)` (alias `var(txn.cookieguard.botd_tool)`) | BotD classification string describing the detected tool/kind of automation. |
284+
| `botd_confidence` | Optional | `var(txn.cookieguard.botd_confidence)` | BotD confidence score (`0``1`). |
285+
| `botd_request_id` | Optional | `var(txn.cookieguard.botd_request_id)` | Fingerprint BotD `requestId` for correlating detections. |
281286
| `session.public.key` *(response → request handoff)* | Optional | Resent by HAProxy if you capture it between calls | Keeps session continuity through restarts |
282287
| `res.hdrs` *(response message)* | Optional | Full response headers on `decide_response` | context.yml allowlist (special/public tables) |
283288

@@ -298,8 +303,10 @@ Each output becomes `var(txn.decision.<name>)` because the SPOE agent uses `opti
298303
| `session.public.key`, `.req_count`, `.rate`, `.idle_seconds`, `.first_path`, `.first_path_deep`, `.recent_hits`, `.rate_window_seconds` | Inspect/limit session behaviour directly in HAProxy (e.g., `http-request deny if { var(txn.decision.session.public.req_count) gt 1000 }`). |
299304
| `session.public.suspicious_score` | Decision-managed score that accumulates via `session.suspicious.increment/reset`. Use it to gate ALTCHA re-challenges or hard denies once a threshold is reached. |
300305
| `session.public.suspicious_ignored` | `true` when the session is marked via `session.suspicious.ignore: true` (no further increments). Helpful for gating rules so trusted bots/networks are exempt. |
306+
| `session.public.botd_verdict`, `.botd_kind`, `.botd_confidence`, `.botd_request_id`, `.botd_age_seconds` | Last BotD verdict stored in Decision’s session tracker (age expresses seconds since telemetry was seen). Lets HAProxy/policy rules reuse detections even after Cookie Guard evicts its 5‑minute cache, and `match.botd.*` rules evaluate against this cached state. |
301307
| `session.special.role`, `.groups`, `.idle_seconds` | Trusted user hints (from context.yml). Skip challenges or grant bypass for known roles. |
302308
| `cookieguard.valid`, `.age_seconds`, `.challenge_level` | Mirror of Cookie Guard telemetry (validity/age/challenge mode). |
309+
| `cookieguard.botd_verdict`, `.botd_kind`, `.botd_confidence`, `.botd_request_id` | Mirror of Cookie Guard’s BotD verdict cache so HAProxy ACLs can reuse Decision’s normalized view (`cookieguard.botd_tool` aliases `.botd_kind`). |
303310

304311
All outputs are optional except the ones your HAProxy logic relies on; use `-m bool`/`-m found` guards before acting on them. Defaults + fallback in `policy.yml` ensure the variables you care about are always present.
305312

@@ -409,6 +416,11 @@ This section documents every supported field in `policy.yml` and how matches are
409416
- `valid`: boolean
410417
- `age_seconds`: numeric comparator map
411418
- `challenge_level`: array of levels (strings)
419+
- `botd`: BotD verdict cache from Cookie Guard
420+
- `verdict`: array of verdicts (`good`, `bad`, `suspect`, etc.)
421+
- `kind`: array of automation tool labels (case-insensitive)
422+
- `confidence`: numeric comparator map (matches `botd_confidence`)
423+
- `request_id`: array of exact Fingerprint request IDs
412424

413425
- Return map
414426
- `reason` (string): optional human‑readable reason. If omitted, the fallback reason applies (default `"default-policy"`).
@@ -429,7 +441,7 @@ Notes on session‑driven inputs
429441
- Suspicion score: `session.public.suspicious_score` reflects the accumulated score managed by Decision; use `session.suspicious.increment/reset` to mutate it.
430442
- Suspicion ignore flag: `session.public.suspicious_ignored` is `true` after a rule sets `session.suspicious.ignore: true`, preventing future increments for that session.
431443
- Special session (trusted profile): `session.special.role`, `session.special.groups`, `session.special.idle_seconds`.
432-
- Cookie‑Guard: `cookieguard.valid` (bool), `cookieguard.age_seconds` (float), `cookieguard.challenge_level` (string).
444+
- Cookie‑Guard: `cookieguard.valid` (bool), `cookieguard.age_seconds` (float), `cookieguard.challenge_level` (string), BotD verdict metadata (`cookieguard.botd_verdict`, `.botd_kind`/`.botd_tool`, `.botd_confidence`, `.botd_request_id`), and the session-cached copies under `session.public.botd_*`.
433445
Examples using session and Cookie‑Guard matchers
434446
```yaml
435447
- name: bursty-client
@@ -457,6 +469,15 @@ Examples using session and Cookie‑Guard matchers
457469
return:
458470
reason: cg-fresh
459471
472+
- name: botd-bad-high-confidence
473+
match:
474+
botd:
475+
verdict: ["bad"]
476+
confidence: { ge: 0.9 }
477+
return:
478+
deny: true
479+
reason: botd-bad
480+
460481
- name: allow-search-bots
461482
match:
462483
asn: [15169, 8075]
@@ -538,6 +559,10 @@ The policy engine consumes the following inputs (see `internal/policy/engine.go:
538559
- `CookieGuardValid` → `match.cookie_guard.valid`.
539560
- `CookieAgeSeconds` → `match.cookie_guard.age_seconds`.
540561
- `ChallengeLevel` → `match.cookie_guard.challenge_level`.
562+
- `BotdVerdict` → `match.botd.verdict` (case-insensitive).
563+
- `BotdKind` → `match.botd.kind` (case-insensitive; `botd_tool` alias is folded here).
564+
- `BotdConfidence` → `match.botd.confidence`.
565+
- `BotdRequestID` → `match.botd.request_id`.
541566

542567
- Geo dependencies
543568
- `Country` and `ASN` require GeoIP DBs to be present (`--city-db`, `--asn-db`). When missing, matches on these fields never fire.

cmd/decision-spoa/main.go

Lines changed: 51 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -461,6 +461,16 @@ func handleRequestMessage(args map[string]string, raw map[string]string, cfg pol
461461
}
462462
challengeLevel := strings.ToLower(strings.TrimSpace(args["cookieguard_level"]))
463463
cookieSession := args["cookieguard_session"]
464+
botdVerdict := strings.TrimSpace(args["botd_verdict"])
465+
botdKind := strings.TrimSpace(args["botd_kind"])
466+
if botdKind == "" {
467+
botdKind = strings.TrimSpace(args["botd_tool"])
468+
}
469+
rawBotdConfidence := strings.TrimSpace(args["botd_confidence"])
470+
botdConfidence := parseFloat(rawBotdConfidence)
471+
botdConfidenceProvided := rawBotdConfidence != ""
472+
botdRequestID := strings.TrimSpace(args["botd_request_id"])
473+
hasBotdInput := botdVerdict != "" || botdKind != "" || botdRequestID != "" || botdConfidenceProvided
464474

465475
var publicSnapshot session.PublicSnapshot
466476
var publicRate float64
@@ -484,14 +494,36 @@ func handleRequestMessage(args map[string]string, raw map[string]string, cfg pol
484494
publicKey = trust.publicSessionKey(baseToken, ip, ua)
485495
if publicKey != "" {
486496
decisionSessionKeySourceTotal.WithLabelValues(keySource).Inc()
487-
publicSnapshot = trust.public.Record(publicKey, path, now)
497+
var botdTelemetry *session.BotdTelemetry
498+
if hasBotdInput {
499+
botdTelemetry = &session.BotdTelemetry{
500+
Verdict: botdVerdict,
501+
Kind: botdKind,
502+
Confidence: botdConfidence,
503+
RequestID: botdRequestID,
504+
SeenAt: now,
505+
}
506+
}
507+
publicSnapshot = trust.public.RecordWithBotd(publicKey, path, now, botdTelemetry)
488508
if publicSnapshot.RecentWindowSec > 0 {
489509
publicRate = float64(publicSnapshot.RecentHits) / publicSnapshot.RecentWindowSec
490510
}
491511
if !publicSnapshot.LastSeen.IsZero() {
492512
publicIdle = now.Sub(publicSnapshot.LastSeen).Seconds()
493513
}
494514
firstPathDeep = pathLooksDeep(publicSnapshot.FirstPath)
515+
if !hasBotdInput && !publicSnapshot.Botd.SeenAt.IsZero() {
516+
botdVerdict = publicSnapshot.Botd.Verdict
517+
if botdKind == "" {
518+
botdKind = publicSnapshot.Botd.Kind
519+
}
520+
if !botdConfidenceProvided {
521+
botdConfidence = publicSnapshot.Botd.Confidence
522+
}
523+
if botdRequestID == "" {
524+
botdRequestID = publicSnapshot.Botd.RequestID
525+
}
526+
}
495527
}
496528
}
497529

@@ -550,6 +582,10 @@ func handleRequestMessage(args map[string]string, raw map[string]string, cfg pol
550582
CookieAgeSeconds: cookieAgeSeconds,
551583
ChallengeLevel: challengeLevel,
552584
CookieGuardValid: cookieGuardValid,
585+
BotdVerdict: botdVerdict,
586+
BotdKind: botdKind,
587+
BotdConfidence: botdConfidence,
588+
BotdRequestID: botdRequestID,
553589
}
554590

555591
out := cfg.Evaluate(input, promRuleCounter{CounterVec: decisionRulesHitTotal}, metricsHostLabel)
@@ -610,6 +646,13 @@ func handleRequestMessage(args map[string]string, raw map[string]string, cfg pol
610646
setResp(resp, "session.public.rate_window_seconds", publicSnapshot.RecentWindowSec)
611647
setResp(resp, "session.public.suspicious_score", publicSnapshot.SuspiciousScore)
612648
setResp(resp, "session.public.suspicious_ignored", publicSnapshot.SuspiciousIgnored)
649+
if !publicSnapshot.Botd.SeenAt.IsZero() {
650+
setResp(resp, "session.public.botd_verdict", publicSnapshot.Botd.Verdict)
651+
setResp(resp, "session.public.botd_kind", publicSnapshot.Botd.Kind)
652+
setResp(resp, "session.public.botd_confidence", publicSnapshot.Botd.Confidence)
653+
setResp(resp, "session.public.botd_request_id", publicSnapshot.Botd.RequestID)
654+
setResp(resp, "session.public.botd_age_seconds", now.Sub(publicSnapshot.Botd.SeenAt).Seconds())
655+
}
613656
}
614657
if specialSnapshot.Key != "" {
615658
setResp(resp, "session.special.role", specialSnapshot.Role)
@@ -619,6 +662,11 @@ func handleRequestMessage(args map[string]string, raw map[string]string, cfg pol
619662
setResp(resp, "cookieguard.valid", cookieGuardValid)
620663
setResp(resp, "cookieguard.age_seconds", cookieAgeSeconds)
621664
setResp(resp, "cookieguard.challenge_level", challengeLevel)
665+
setResp(resp, "cookieguard.botd_verdict", botdVerdict)
666+
setResp(resp, "cookieguard.botd_kind", botdKind)
667+
setResp(resp, "cookieguard.botd_tool", botdKind)
668+
setResp(resp, "cookieguard.botd_confidence", botdConfidence)
669+
setResp(resp, "cookieguard.botd_request_id", botdRequestID)
622670

623671
if cfg.Debug {
624672
// Optional verbose line with raw inputs and snapshots
@@ -627,8 +675,9 @@ func handleRequestMessage(args map[string]string, raw map[string]string, cfg pol
627675
entries := trusted.Entries()
628676
hb2 := cookies["hb_v2"] != ""
629677
hb3 := cookies["hb_v3"] != ""
630-
log.Printf("policy-verbose: raw_input=%v fe=%s be=%s src=%s xff=%s ip=%v xff_used=%t xff_stripped=%d trusted_peer=%t trusted_entries=%v asn=%d c=%s m=%s host=%s path=%s query=%q sni=%s ja3=%s hb2=%t hb3=%t key_src=%s public={key=%s req=%d hits=%d rate=%.6f idle=%.3f first_path=%s deep=%t sus=%d ignore=%t} special={role=%s idle=%.3f groups=%s} reason=%s bucket=%s elapsed=%.6f ua=%q vars=%v",
678+
log.Printf("policy-verbose: raw_input=%v fe=%s be=%s src=%s xff=%s ip=%v xff_used=%t xff_stripped=%d trusted_peer=%t trusted_entries=%v asn=%d c=%s m=%s host=%s path=%s query=%q sni=%s ja3=%s hb2=%t hb3=%t key_src=%s botd={verdict=%s kind=%s confidence=%.3f request_id=%s} public={key=%s req=%d hits=%d rate=%.6f idle=%.3f first_path=%s deep=%t sus=%d ignore=%t} special={role=%s idle=%.3f groups=%s} reason=%s bucket=%s elapsed=%.6f ua=%q vars=%v",
631679
sortedRaw(raw), frontend, backend, src, xff, ip, xffUsed, strippedHops, pt, entries, asn, country, strings.ToUpper(method), host, truncatePath(path, 200), query, sni, ja3, hb2, hb3, keySource,
680+
botdVerdict, botdKind, botdConfidence, botdRequestID,
632681
publicSnapshot.Key, publicSnapshot.RequestCount, publicSnapshot.RecentHits, publicRate, publicIdle, publicSnapshot.FirstPath, firstPathDeep, publicSnapshot.SuspiciousScore, publicSnapshot.SuspiciousIgnored,
633682
specialSnapshot.Role, specialIdle, strings.Join(specialSnapshot.Groups, ","), out.Reason, bucketLabel, elapsed, ua, resp)
634683
}

0 commit comments

Comments
 (0)