Skip to content

Commit b43915e

Browse files
docs(planning): Phase 1.6 first API run results — R1.1/D7 NOT triggered
Replaces the §5 "Open items for Phase 1.5/1.6" placeholder in docs/APPINSPECT_FINDINGS.md with the actual hosted-API findings from run ID 26000914082 against HEAD 027014a (Phase 1.5 path-doubling hotfix). Result summary: - Cloud Vetting (cloud tag): 0 errors, 1 failure, 5 warnings, 161 pass - Self-Service Cloud (private_app tag): 0 errors, 0 failures, 5 warnings, 159 pass The single failure is check_that_app_passes_slim_validation_for_cloud, decomposing into: F1 (HARD ERROR): app.manifest platformRequirements.splunk = ">=9.0.0" is rejected by SLIM — no concrete supported version. One-line manifest edit for Phase 1.7. F2-F11: SLIM spec-drift on python.version + python.required across 5 stanzas (inputs.conf x3, restmap.conf, commands.conf). Source settings ARE present at default/inputs.conf:16-17, :43-44, :61-62; default/restmap.conf:22-23; default/commands.conf:8-9 — required by static AppInspect's python-version check. SLIM's spec catalog is older and flags them as undefined. Phase 1.7 will suppress via .appinspect_api.expect.yaml; settings stay. F12: default/app.conf [id] missing check_for_updates. One-line fix for Phase 1.7. R1.1 / D7 escalation assessment: NOT triggered. The pre-flagged "persistent scripted input rejection" surface (wl_fim_watch.py interval = 0, called out as "single biggest remaining unknown" in the previous §5) did NOT materialize — Cloud Vetting accepted the stanza; the only flags on wl_fim_watch.py were the same spec-drift noise (F6/F7) every other script stanza got. All 12 sub-findings are config edits, total estimated effort <=2 hours vs the >2 week budget the escalation clause assumes. Phase 1.7 ("Fix all error-severity findings") proceeds within original scope and schedule. The §7 revision log records both the Phase 1.5 workflow drift fix (commit 027014a) and the Phase 1.6 result. Existing §3 warning triage applies unchanged — the dynamic API run reproduced the local CLI's 5 cloud warnings byte-for-byte; no new warning classes emerged.
1 parent 027014a commit b43915e

1 file changed

Lines changed: 145 additions & 24 deletions

File tree

docs/APPINSPECT_FINDINGS.md

Lines changed: 145 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -196,30 +196,140 @@ before the migration).
196196

197197
---
198198

199-
## 5. Open items for Phase 1.5 / 1.6 (Cloud API + dynamic checks)
200-
201-
Static AppInspect (this run) is the input to Phase 1.5
202-
(`splunk/appinspect-api-action` wiring) and Phase 1.6 (first API run).
203-
The API stage performs container boot + runtime checks the local CLI
204-
cannot do. Expected surfaces of concern, in order of likelihood:
205-
206-
1. **Persistent scripted inputs**. `bin/wl_fim_watch.py` runs with
207-
`interval = 0` (long-running daemon). Cloud Vetting historically
208-
has restrictions on persistent processes. This is the single biggest
209-
remaining unknown — call it out explicitly in the Phase 1.6 triage
210-
pass.
211-
2. **Outbound network calls**. The handler talks to `localhost:8089`
212-
for the audit-emission `simpleRequest` call. Loopback is normally
213-
permitted, but Cloud Vetting may treat `splunk.rest.simpleRequest`
214-
differently than direct socket usage. Confirm in 1.6.
215-
3. **CycloneDX SBOM file (`*.cdx.json`)** sits next to the .spl, not
216-
inside it. Splunkbase upload accepts it as a sibling. Phase 1.5
217-
workflow may need an explicit upload step.
218-
219-
None of these are *expected* to fail; they are the surfaces where
220-
expected and actual could plausibly diverge. The plan-doc D7 escalation
221-
clause (Phase 1 week 4) governs if any of these turn out to be hard
222-
blockers.
199+
## 5. Cloud API + dynamic checks (Phase 1.6 first API run)
200+
201+
> **Status**: Phase 1.6 first API run executed 2026-05-17 against
202+
> `wl_manager-1.0.0-rc1.spl` via `.github/workflows/appinspect-api.yml`.
203+
> Run ID: `26000914082` (GHA `appinspect-api.yml`, HEAD `027014a`).
204+
> The dynamic stage reproduced the Phase 1.3 local-CLI warning set
205+
> exactly and surfaced ONE additional **failure** that the local CLI
206+
> does not run — the SLIM packager validator. See §5.4 below for the
207+
> escalation assessment.
208+
209+
### 5.1 Headline numbers (API stage)
210+
211+
| Result class | Cloud Vetting (`cloud`) | Self-Service Cloud (`private_app`) |
212+
|-----------------|-------------------------|-----------------------------------|
213+
| **error** | **0** | **0** |
214+
| **failure** | **1** | **0** |
215+
| **future_failure** | **0** | **0** |
216+
| skipped | 0 | 0 |
217+
| not_applicable | 80 | 78 |
218+
| warning | 5 | 5 |
219+
| manual_check | 0 | 0 |
220+
| success | 161 | 159 |
221+
222+
The hosted API runs 4 more checks than the local CLI on the cloud
223+
profile (161 success vs 157 in §1) — those additions are SLIM-related
224+
dynamic checks the local CLI does not invoke. The 5 warnings on each
225+
profile are byte-identical to the §3 triage above (`check_for_splunk_js`,
226+
`check_for_python_script_existence`, `check_for_scripted_inputs`,
227+
`check_for_gratuitous_cron_scheduling`, `check_collections_conf`); no
228+
new warnings were introduced by the dynamic stage.
229+
230+
The `private_app` profile passed cleanly — Self-Service Cloud is
231+
already a valid distribution path for the current build.
232+
233+
### 5.2 The one failure — `check_that_app_passes_slim_validation_for_cloud`
234+
235+
The hosted API embeds the Splunk Packaging Toolkit (SLIM) and runs
236+
its `slim validate` step against the unpacked .spl. SLIM rejected the
237+
package with a single hard error plus eleven secondary "undefined
238+
setting" observations:
239+
240+
| # | Class | File | Stanza | Setting / message |
241+
|---|-------|------|--------|------------------|
242+
| **F1** | **HARD ERROR** | `app.manifest` | `platformRequirements.splunk` | "Version requirement includes no supported version of Splunk Enterprise: `>=9.0.0`" |
243+
| F2 | Undefined setting | `default/inputs.conf` | `[script://...wl_expiration_cleanup.py]` | `python.version` |
244+
| F3 | Undefined setting | `default/inputs.conf` | `[script://...wl_expiration_cleanup.py]` | `python.required` |
245+
| F4 | Undefined setting | `default/inputs.conf` | `[script://...wl_fim.py]` | `python.version` |
246+
| F5 | Undefined setting | `default/inputs.conf` | `[script://...wl_fim.py]` | `python.required` |
247+
| F6 | Undefined setting | `default/inputs.conf` | `[script://...wl_fim_watch.py]` | `python.version` |
248+
| F7 | Undefined setting | `default/inputs.conf` | `[script://...wl_fim_watch.py]` | `python.required` |
249+
| F8 | Undefined setting | `default/restmap.conf` | `[script:wl_manager_handler]` | `python.version` |
250+
| F9 | Undefined setting | `default/restmap.conf` | `[script:wl_manager_handler]` | `python.required` |
251+
| F10 | Undefined setting | `default/commands.conf` | `[wlexpiringsoon]` | `python.version` |
252+
| F11 | Undefined setting | `default/commands.conf` | `[wlexpiringsoon]` | `python.required` |
253+
| F12 | Undefined setting | `default/app.conf` | `[id]` | `check_for_updates` |
254+
255+
**F1 (HARD ERROR) — root cause analysis**: SLIM expects a closed-range
256+
or specific-version constraint, not the open lower bound `>=9.0.0`.
257+
The current manifest declares only a floor, which SLIM reads as "no
258+
upper bound = no concrete Splunk version is in range". Fix in Phase 1.7
259+
is a one-line manifest edit — e.g., `">=9.0.0,<11.0.0"` or the
260+
Splunkbase-recommended form. Documented for Phase 1.7.
261+
262+
**F2–F11 — root cause analysis**: SLIM is using an older `.conf.spec`
263+
catalog than current AppInspect. The settings `python.version` and
264+
`python.required` ARE declared at the source (`default/inputs.conf:16-17`,
265+
`:43-44`, `:61-62`; `default/restmap.conf:22-23`; `default/commands.conf:8-9`)
266+
because static AppInspect's `check_python_version_correctness_for_splunk_enterprise`
267+
requires them. SLIM does not recognize them in its spec and flags
268+
both as undefined. This is the spec-drift between AppInspect and SLIM
269+
that the existing source-comments (e.g., `inputs.conf:12-15`)
270+
predicted. Two valid remediations for Phase 1.7:
271+
272+
1. Live with the SLIM noise; document it in `.appinspect_api.expect.yaml`
273+
so the workflow stops failing on these. The settings stay (static
274+
AppInspect needs them).
275+
2. Engage Splunk on the SLIM/AppInspect spec divergence. Likely too
276+
slow to be a Phase 1.7 fix.
277+
278+
Option (1) is the intended Phase 1.7 path; option (2) is for the
279+
roadmap.
280+
281+
**F12 — root cause analysis**: `default/app.conf [id]` is missing
282+
`check_for_updates`. The setting is optional in static AppInspect but
283+
SLIM treats its absence as undefined (vs. the explicit
284+
`check_for_updates = false` Splunk recommends for Cloud apps that
285+
should not auto-update via the in-product update mechanism). One-line
286+
fix in Phase 1.7.
287+
288+
### 5.3 Pre-flagged surfaces — outcome
289+
290+
The pre-Phase-1.6 `§5` (this section in its previous form) listed
291+
three surfaces of concern. Outcome from the actual run:
292+
293+
| # | Pre-flagged surface | Outcome |
294+
|---|---------------------|---------|
295+
| 1 | **Persistent scripted inputs** (`bin/wl_fim_watch.py` `interval = 0`, "single biggest remaining unknown") | **NOT rejected.** The Cloud profile accepted the stanza's presence; the only flag on `wl_fim_watch.py` was the spec-drift `python.version`/`python.required` noise (F6/F7) shared with every other script stanza. The R1.1 / D7 escalation surface (refactor `wl_fim_watch.py` to non-persistent) did **NOT** materialize. |
296+
| 2 | **Outbound network calls** (handler → `localhost:8089` via `splunk.rest.simpleRequest`) | **Not flagged.** Loopback to splunkd is implicit-allow on both profiles. |
297+
| 3 | **CycloneDX SBOM** as a `.spl` sibling | **Not flagged by the validator.** Splunkbase upload step (Phase 4) will exercise this separately. |
298+
299+
### 5.4 R1.1 / D7 escalation assessment — **NOT triggered**
300+
301+
The Phase 1 plan (`docs/PUBLIC_RELEASE_PLAN.md` §1) escalates to D7
302+
(architectural refactor) if Cloud Vetting categorically rejects the
303+
persistent scripted input. That did not happen — see §5.3 row 1.
304+
305+
All twelve sub-findings (F1–F12) are config / manifest edits, not
306+
architectural changes. Total estimated Phase 1.7 effort: **≤2 hours**
307+
for the manifest + `.appinspect_api.expect.yaml` + `[id]` edits,
308+
versus the **>2 week** budget the escalation clause assumes for a
309+
`wl_fim_watch.py` refactor.
310+
311+
Phase 1.7 ("Fix all error-severity findings") can proceed within its
312+
original scope on the original schedule.
313+
314+
### 5.5 Phase 1.5 workflow drift discovered and fixed during this run
315+
316+
The first Phase 1.5 push (run ID `25998624960`, commit `ee0d449`)
317+
failed before any AppInspect check ran, due to a path-doubling bug in
318+
the action wrapper's entrypoint (`splunk/appinspect-api-action@v3.0.5`
319+
entrypoint.sh runs `ls $INPUT_APP_PATH` and joins it onto the input;
320+
the workflow passed the .spl file directly instead of a single-file
321+
directory). Fix landed in the same Phase 1.6 session, commit `027014a`:
322+
stage the .spl into a clean `dist/appinspect/` directory and point the
323+
action at that dir. Re-run (`26000914082`) progressed past the entry
324+
step and produced the numbers in §5.1.
325+
326+
### 5.6 Raw outputs
327+
328+
The hosted API's HTML and JSON reports are downloaded inside the
329+
action's container but no `actions/upload-artifact` step exists yet,
330+
so they are lost when the runner is torn down. **Phase 1.7 follow-up**:
331+
add an artifact upload step so future runs preserve the JSON for
332+
diffing.
223333

224334
---
225335

@@ -259,3 +369,14 @@ The CI variant of this command lives in
259369
- 2026-05-17 — initial Phase 1.3 baseline. App.manifest version drift
260370
caught + fixed in same run; §3.5 pre-flight extended to cover it.
261371
All warnings re-triaged. Zero delta vs Phase 0.0 build-660 baseline.
372+
- 2026-05-17 — Phase 1.6 first hosted-API run (run ID `26000914082`,
373+
HEAD `027014a`). Cloud profile surfaced 1 failure
374+
(`check_that_app_passes_slim_validation_for_cloud`); Self-Service
375+
Cloud profile clean. R1.1 / D7 escalation assessed and **NOT
376+
triggered** — persistent scripted input (`wl_fim_watch.py`) was
377+
accepted by Cloud Vetting; the failure is a SLIM-spec issue
378+
(manifest version range + spec-drift undefined-setting noise on
379+
`python.version` / `python.required`) that decomposes into 12
380+
config edits (F1–F12 in §5.2). Phase 1.5 workflow path-doubling
381+
drift fixed in commit `027014a` during the same session. §5
382+
replaced with the actual Phase 1.6 results (was placeholder).

0 commit comments

Comments
 (0)