|
| 1 | +--- |
| 2 | +sut_path: /home/ssm-user/src/saluki |
| 3 | +commit: 042f41db3bd97118c38981765fd49696fce9d318 |
| 4 | +updated: 2026-05-29 |
| 5 | +external_references: |
| 6 | + - path: https://datadoghq.atlassian.net/wiki/spaces/DADP/ |
| 7 | + why: ADP Confluence space — headline guarantees that frame these defects as bugs. |
| 8 | +--- |
| 9 | + |
| 10 | +# Bug Ledger |
| 11 | + |
| 12 | +Accounting of every defect discovered during `antithesis-research`, with the vehicle used to |
| 13 | +demonstrate it. Goal: burn each discovered bug into a local repro case or an Antithesis triage shot. |
| 14 | +**No fixes were applied.** Following TDD, each local repro asserts the *desired* invariant, so it |
| 15 | +**currently FAILS** against the buggy code — the failing test *is* the demonstration. It turns green |
| 16 | +once the defect is fixed (a ready-made regression guard). Each test's comment describes why/how the |
| 17 | +bug happens. |
| 18 | + |
| 19 | +## Burned into local repro cases (failing unit tests) |
| 20 | + |
| 21 | +| # | Property | Test | Defect | |
| 22 | +|---|----------|------|--------| |
| 23 | +| 1 | `aggregate-no-panic-any-window` | `lib/saluki-components/.../transforms/aggregate/mod.rs::tests::bug_sub_second_aggregate_window_panics_on_insert` | Sub-second `aggregate_window_duration` → `bucket_width_secs=0` → `align_to_bucket_start` does `% 0` → panic on first insert (crash + restart loop). | |
| 24 | +| 2 | `ddsketch-no-nan-poison` | `lib/ddsketch/src/agent/sketch.rs::tests::bug_nan_sample_poisons_sum_and_avg` | One NaN sample permanently poisons sketch `sum`/`avg` (sticky), `count`/`min`/`max` stay valid → silent corruption; no finiteness guard at the sketch boundary. | |
| 25 | +| 3 | `replay-corruption-not-silent-eof` | `lib/saluki-components/.../sources/dogstatsd/replay/reader.rs::tests::bug_corrupt_length_prefix_silently_drops_following_records` | A corrupt/oversized length prefix is read as clean EOF → all following well-formed records silently dropped (false replay fidelity / data loss). | |
| 26 | +| 4 | `aggregate-clock-skew-stable` (forward-jump facet) | `lib/saluki-components/.../transforms/aggregate/mod.rs::tests::bug_forward_clock_jump_floods_zero_value_points` | No `current_time >= last_flush` guard; a forward wall-clock jump builds `zero_value_buckets` over the whole interval — O(jump) work/alloc — and floods one idle counter with points proportional to the jump. | |
| 27 | +| 5 | `rss-bounded-under-cardinality` / `interner-full-bounded` (root cause) | `lib/saluki-context/src/resolver.rs::tests::bug_default_heap_fallback_makes_context_resolution_unbounded` | `allow_heap_allocations` defaults true → a full fixed-size interner silently spills to the heap and `resolve` never refuses → unbounded memory under a high-cardinality flood; the only bounded mode (heap disallowed) silently drops. | |
| 28 | +| 6 | `config-stall-no-deadlock` | `lib/saluki-config/src/lib.rs::tests::bug_config_ready_hangs_forever_without_snapshot` | `GenericConfiguration::ready()` awaits the first dynamic snapshot with no internal timeout; with the sender held open and silent, it never resolves → ADP startup hangs forever. | |
| 29 | + |
| 30 | +Run all six (expect six FAILURES — the failing tests are the demonstrations): |
| 31 | +`cargo nextest run --no-fail-fast -E 'test(/bug_sub_second_aggregate_window_panics_on_insert|bug_nan_sample_poisons_sum_and_avg|bug_corrupt_length_prefix_silently_drops_following_records|bug_forward_clock_jump_floods_zero_value_points|bug_default_heap_fallback_makes_context_resolution_unbounded|bug_config_ready_hangs_forever_without_snapshot/)'` |
| 32 | + |
| 33 | +## Burned into an Antithesis triage shot (submitted run) |
| 34 | + |
| 35 | +- **`rss-bounded-under-cardinality` (behavioral)** and **`forwarder-eventual-delivery` (baseline liveness)** — run id (redacted; tracked internally) (test-name `saluki-adp-bug-hunt`, 30 min, submitted 2026-05-29). The `parallel_driver_send_dogstatsd` high-cardinality regime drives memory growth; `finally_verify_delivery` checks delivery. Triage with the `antithesis-triage` skill once it completes. |
| 36 | + |
| 37 | +## Antithesis-shot-only — blocked on harness infrastructure (not locally reproducible) |
| 38 | + |
| 39 | +These discovered defects cannot be reproduced as local unit tests and require infrastructure not yet |
| 40 | +built. Each is a follow-up Antithesis shot, not a current repro. |
| 41 | + |
| 42 | +- **`memory-limiter-survives-rss-read-failure`** — `check_memory_usage` does `querier.resident_set_size().expect(...)`; an RSS-read failure panics the checker thread, silently disabling memory protection. **Not locally reproducible:** the `Querier` is constructed internally with no injection seam. **Shot blocker:** a custom `/proc` fault (enabled for the tenant) + a memory-limiter-enabled ADP config variant + a SUT-side `assert_unreachable!` at the `.expect` site. |
| 43 | +- **`config-runtime-update-not-revalidated`** — the incompatibility gate runs only at startup; a runtime config-stream update can introduce a high-severity-incompatible key with no re-gate. **Shot blocker:** config-stream add-on **and** human confirmation of intent (intentional trust of the authoritative Agent vs. oversight) — flagged `(needs human input)` in the catalog. |
| 44 | +- **`shutdown-drains-no-loss` / `graceful-shutdown-within-30s` (forceful-stop data loss)** — behavioral; needs the running harness shut down under a slow/blocked intake to exceed the 30s grace window. **Shot blocker:** an intake failure-mode toggle + a shutdown driver. |
| 45 | + |
| 46 | +## Recorded as covered / not a distinct defect |
| 47 | + |
| 48 | +- **`aggregate-clock-skew-stable` (backward-jump gap)** — dual of bug #4; same root cause (no monotonicity guard on `current_time` vs `last_flush`). Covered by #4's triage. |
| 49 | +- **aggregate context-limit counts contexts, not bytes** — a single context with many timestamped values has unbounded value memory; a facet of `rss-bounded-under-cardinality`, partially covered by #5 and the catalog's open question. |
| 50 | +- **`ddsketch-relative-error-bound` merge non-associativity** — f64 ordering sensitivity; a library-level numeric property, not a clear ADP runtime defect (the catalog demoted it to a harness/proptest invariant). |
| 51 | +- **`source-dispatch-no-misroute`** — misroute is structurally improbable with the current `extract`-then-`send_all` ordering; the live facet (silent uncounted loss on a downstream dispatch error) would need a failing-dispatcher harness. Candidate future local repro; not a confirmed defect today. |
| 52 | + |
| 53 | +## Status |
| 54 | + |
| 55 | +All cleanly-reproducible discovered bugs are burned in: **6 local repros + 1 submitted run.** The |
| 56 | +remaining items are explicitly blocked on harness infrastructure (custom `/proc` fault, config-stream |
| 57 | +add-on, intake failure toggle) or human input, and are recorded above as follow-up Antithesis shots. |
| 58 | +No further bug is reproducible without building that infrastructure. |
0 commit comments