Skip to content

fix(security): remediate findings from security assessment#33

Open
rpolysec wants to merge 10 commits into
mainfrom
fix/security-assessment-pen-2026-005
Open

fix(security): remediate findings from security assessment#33
rpolysec wants to merge 10 commits into
mainfrom
fix/security-assessment-pen-2026-005

Conversation

@rpolysec

@rpolysec rpolysec commented May 6, 2026

Copy link
Copy Markdown

Summary

Remediate findings from an internal security assessment. The core issue is that LLM output was treated as trusted input across multiple code paths — command strings, file paths, pickle deserialization, and YAML construction.

  • Fix critical command injection in script scorer and pickle deserialization RCE in eval cache
  • Harden LLM trust boundary: restrict skill generator tools, validate skill names against path traversal
  • Add shlex.quote() to all shell interpolation in Daytona and Docker backends
  • Replace manual YAML construction with yaml.safe_dump()
  • Harden Dockerfile: non-root user, GPG-verified Node.js, pinned Goose version
  • Add upper-bound dependency pins, API key config warning, scoped process killing
  • Add Security Considerations section to README and fix vulnerable scorer example
  • 40 security regression tests

Breaking changes

Script scorer interface: The command field no longer supports {predicted} and {expected} placeholders. Predictions and expected values are now passed as PREDICTED and EXPECTED environment variables. Existing scorer scripts need to read from os.environ instead of command-line arguments.

Before:

[scorer]
type = "script"
command = "python score.py --predicted {predicted} --expected {expected}"

After:

[scorer]
type = "script"
command = "python score.py"
# score.py reads os.environ["PREDICTED"] and os.environ["EXPECTED"]

Eval result cache format: Results are now serialized as JSON instead of pickle. Existing .pkl result files from prior runs will not load. Re-run evaluations to regenerate results. The default output extension changed from .pkl to .json.

Test plan

  • Existing test suite: 594 passed, 1 pre-existing failure (unrelated)
  • 40 security regression tests covering injection, serialization, path traversal, YAML safety, tool restriction, and shell quoting
  • Docker image build and runtime validated (non-root user, volume mount, pip install, CLI)
  • Daytona integration: 14 shell quoting tests passed against live sandbox
  • Full evolution loop end-to-end (requires LLM API keys + compute)
  • linux/amd64 cross-compile build for Daytona production target

rpolysec added 10 commits May 6, 2026 12:34
Script scorer no longer interpolates LLM output into command strings.
Predicted/expected values are passed via environment variables instead
of .replace() + shlex.split(), preventing argument injection from
adversarial LLM responses.

Replace pickle serialization with JSON in eval cache. Pickle
deserialization executes arbitrary code; a crafted .pkl file from a
compromised sandbox achieves RCE on the host. JSON serialization
eliminates this attack vector.
…ools and validate skill names

Remove dangerous tools (Bash, BashOutput, WebFetch, WebSearch) from the
skill generator agent to prevent cross-agent prompt injection. Add
field_validator on target_skill to enforce safe naming pattern and block
path traversal via LLM-controlled skill names.
…ends

Add shlex.quote() to all interpolated values in daytona.py shell
commands, including git tag-derived branch names, container paths, and
CLI args. Replace manual YAML string construction in launcher.py with
yaml.safe_dump() to prevent injection via paths with special characters.
Add non-root 'evoskill' user to prevent container-as-root. Replace
pipe-to-bash Node.js install with GPG-verified apt repo setup and pin
Goose to a specific version via build arg instead of fetching
/releases/latest at build time.
…safe YAML

Add upper-bound version pins to all runtime dependencies. Warn when
Daytona API key is loaded from config file instead of env var. Replace
system-wide pkill with tracked-PID-only killing. Use yaml.safe_dump()
instead of yaml.dump().
…mple

Add Security Considerations section covering the trust model, safe
configuration patterns, and vulnerability reporting. Replace the
vulnerable {predicted} interpolation example with the env-var-based
pattern that matches the actual implementation.
Update script scorer test to use env-var pattern. Remove Skill tool
from skill_generator required_tools assertions to match restricted
tool access.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant