Skip to content

Commit b30c433

Browse files
fix(release): Phase 3.1 F-C2 — purge lookups/ test artifacts from .spl payload
Pre-public audit follow-up. User raised during public-flip readiness review: "lookups/ has many test csv files — are they going public?" Targeted recheck found significant pollution that the initial Phase 3.1 sweep missed. Why this matters more than ordinary repo cruft: files under `lookups/` are packaged into the .spl and installed onto every customer's Splunk instance. Names like `DR_BROWSER_TEST.csv`, `DR_E2E_ADMIN.csv`, `DR_STRESS_2000x100.csv` would have shown up as live whitelist dropdown options in the customer's UI. That's not "weird artifacts in source"; that's user-facing product pollution. Changes: Deleted (11 tracked test CSVs at lookups/ root): DR_TEST_2.csv, DR_TEST_3.csv, DR_E2E_ADMIN.csv, DR_BROWSER_TEST.csv, DR_VERSION_TEST.csv, DR_TRASH_TEST.csv, DR_APPROVAL_TEST_1_v2.csv, DR_LONG_NOTIFICATION_TEST_2.csv, DR_STRESS_2000x100.csv, AL_test_1775974555656.csv, AL_super_1775974555731.csv Deleted (lookups/_trash/ entire subtree, 18 files): Runtime soft-delete state captured during past test sessions. The trash directory is the running app's recycle-bin storage, not source-of-truth. Trimmed (lookups/rule_csv_map.csv, 33 rows → 19 rows): Removed 10 mappings that pointed at the now-deleted test CSVs. Also removed 4 pre-existing "ghost map" entries that pointed at CSVs which never existed in the tree: DR_TEST_4.csv, test_gated_csv.csv, AL13_test_limits.csv, AL15_super_exempt.csv The map was internally inconsistent before this change; trimming is a net improvement regardless of public-release context. Updated (.gitignore): Added `lookups/_trash/` alongside existing entries for `lookups/_versions/`, `lookups/_detection_rules.json`, and the runtime `lookups/_*.json` state files. Existing `lookups/DR777_*.csv` / `DR778_*.csv` / `DR998_*.csv` patterns already covered the working-tree-only cruft cleaned during this pass. Updated (docs/PRE_PUBLIC_AUDIT.md): New section "CRITICAL — fixed in-turn (added post-audit)" documenting F-C2. Revision-log entry added. Methodology lesson recorded: future pre-public sweeps must explicitly ask "does every file in each .spl-payload directory belong in the product." Test-impact verification before deletion: every test referencing the deleted CSV names creates the CSV at runtime via the REST API as part of test setup. None depend on the tracked file pre-existing. Confirmed across tests/test_e2e_api.py, tests/e2e/test_admin_limits.cjs, tests/test_e2e_advanced.py, tests/unit/test_approval_queue_state_machine.py, tests/integration/test_chaos_save_csv_chain.py, tests/test_e2e_manual_browser.py. Doc-drift: passes (31 docs / build 660). Actions-budget impact: zero. GitHub Actions is at 100% for the month; this lands as a docs/data commit so no path-filtered workflows would run anyway. Public-flip vs wait-for-June-1 decision deferred per user.
1 parent d5faf1c commit b30c433

32 files changed

Lines changed: 79 additions & 2707 deletions

.gitignore

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -35,6 +35,9 @@ lookups/DR778_*.csv
3535
lookups/DR998_*.csv
3636
lookups/_detection_rules.json
3737
lookups/_versions/
38+
# Soft-delete state — populated when users (or tests) trash CSVs.
39+
# Phase 3.1 audit F-C2 (2026-05-18) found this previously tracked.
40+
lookups/_trash/
3841

3942
# Runtime state files (generated by app at runtime)
4043
lookups/_daily_limits.json

docs/PRE_PUBLIC_AUDIT.md

Lines changed: 76 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -101,6 +101,80 @@ pre-rewrite state until the explicit force-push.
101101

102102
---
103103

104+
### CRITICAL — fixed in-turn (added post-audit, 2026-05-18)
105+
106+
**F-C2: `lookups/` polluted with E2E test artifacts that would ship to customers.**
107+
108+
The initial Phase 3.1 sweep checked `lookups/DR*.csv` for "real IPs vs
109+
synthetic" (the RFC1918 lens) and confirmed all hostnames/IPs were
110+
synthetic. It did NOT check "is every CSV here legitimate seed data
111+
vs leftover test fixture." User raised the concern during the
112+
public-flip readiness discussion, and a targeted recheck found
113+
significant pollution.
114+
115+
**Why this would have been actively user-facing**: files in `lookups/`
116+
are packaged into the `.spl` and installed onto every customer's
117+
Splunk instance. `DR_BROWSER_TEST.csv` / `DR_TEST_2.csv` /
118+
`DR_E2E_ADMIN.csv` would have shown up as real whitelist dropdown
119+
options in the customer's UI. That's not "weird stuff in source"; that's
120+
visible product pollution.
121+
122+
**Removed (tracked, 29 files)**:
123+
124+
- 11 root-level test CSVs that match the
125+
`DR_(TEST|STRESS|VERSION|TRASH|APPROVAL|E2E|BROWSER|LONG)_*.csv`
126+
or `AL_(test|super)_*` patterns:
127+
`DR_TEST_2.csv`, `DR_TEST_3.csv`, `DR_E2E_ADMIN.csv`,
128+
`DR_BROWSER_TEST.csv`, `DR_VERSION_TEST.csv`, `DR_TRASH_TEST.csv`,
129+
`DR_APPROVAL_TEST_1_v2.csv`, `DR_LONG_NOTIFICATION_TEST_2.csv`,
130+
`DR_STRESS_2000x100.csv`, `AL_test_1775974555656.csv`,
131+
`AL_super_1775974555731.csv`.
132+
- 18 files in `lookups/_trash/` — runtime soft-delete state captured
133+
from past test sessions. The `_trash/` directory is the running
134+
app's recycle-bin storage, not source-of-truth.
135+
136+
**Removed (working-tree only, 6 files, never tracked)**: the
137+
DR777 / DR778 / DR998 test artifacts that the existing `.gitignore`
138+
patterns at `lookups/DR777_*.csv` / `DR778_*.csv` / `DR998_*.csv`
139+
already covered. These were untracked but visible in `ls`; cleaned
140+
up for tidiness.
141+
142+
**`rule_csv_map.csv` trimmed from 33 rows → 19 rows**. The 14 removed
143+
rows split into two groups:
144+
145+
- 10 mappings that pointed at the test CSVs above (now orphaned).
146+
- 4 mappings that pointed at CSVs which never existed in the working
147+
tree at all: `DR_TEST_4.csv`, `test_gated_csv.csv`,
148+
`AL13_test_limits.csv`, `AL15_super_exempt.csv`. These were a
149+
pre-existing "ghost map entry" bug — the map was already
150+
internally inconsistent. Removing them is a net improvement
151+
regardless of public release.
152+
153+
**`.gitignore` updated**: `lookups/_trash/` is now ignored alongside
154+
the already-ignored `lookups/_versions/`, `lookups/_detection_rules.json`,
155+
and the runtime `_*.json` state files. Future test runs that exercise
156+
the trash feature will not re-track its state.
157+
158+
**Test-impact verification before deletion**: every test referencing
159+
the deleted CSV names (`tests/test_e2e_api.py`,
160+
`tests/e2e/test_admin_limits.cjs`, `tests/test_e2e_advanced.py`,
161+
`tests/unit/test_approval_queue_state_machine.py`,
162+
`tests/integration/test_chaos_save_csv_chain.py`,
163+
`tests/test_e2e_manual_browser.py`) creates the CSV at runtime via the
164+
REST API as part of test setup. None depend on the tracked file
165+
pre-existing on disk. Deletion is safe.
166+
167+
**Why the initial audit missed it**: the "outsider would find this
168+
embarrassing" lens has multiple orthogonal sub-lenses (real-data
169+
leaks, branding inconsistency, legal/license issues, product
170+
pollution, dev cruft, ...). The initial pass exercised four of them
171+
strongly but skipped product-pollution-via-tracked-data. F-C2
172+
extends the audit lens; future pre-public sweeps should explicitly
173+
ask "does every file in a directory that ships to customers belong
174+
in the product" for each directory under the .spl payload.
175+
176+
---
177+
104178
### HIGH — fixed in-turn
105179

106180
**F-H1: License inconsistency — LICENSE/NOTICE say Apache 2.0; README, mkdocs.yml, docs/index.md, app.manifest, sbom.cdx.json claimed MIT.**
@@ -289,3 +363,5 @@ All three items have been decided. Phase 3.1 is closed.
289363
| Date | Auditor | Notes |
290364
|------|---------|-------|
291365
| 2026-05-18 | claude-opus-4-7 (Phase 3.1) | Initial audit. 1 CRITICAL + 1 HIGH fixed in-turn; 1 LOW fixed in-turn; 2 LOW + 1 git-history question surfaced for user decision. |
366+
| 2026-05-18 | claude-opus-4-7 + user | Follow-up: F-C1 history rewritten + force-pushed per user authorization; F-L2 superpowers moved to .planning/; F-L3 deferred to v1.1. |
367+
| 2026-05-18 | claude-opus-4-7 + user | Post-audit catch — F-C2 (`lookups/` test-artifact pollution) raised by user during public-flip readiness review. 29 tracked test CSVs + 18 `_trash/` items removed; `rule_csv_map.csv` trimmed 33→19 rows; `.gitignore` extended. Lesson recorded: future pre-public sweeps must explicitly ask "does every file in each .spl-payload directory belong in the product." |

lookups/AL_super_1775974555731.csv

Lines changed: 0 additions & 1 deletion
This file was deleted.

lookups/AL_test_1775974555656.csv

Lines changed: 0 additions & 1 deletion
This file was deleted.

lookups/DR_APPROVAL_TEST_1_v2.csv

Lines changed: 0 additions & 6 deletions
This file was deleted.

0 commit comments

Comments
 (0)