Skip to content

Add ToolGuard security middleware for validating agent tool execution #6066

@karthikeyavelivela

Description

@karthikeyavelivela

Problem

@karthikeyavelivela
AI agents interacting with external tools are vulnerable to prompt injection and unintended destructive actions.

Currently, tool execution is dispatched directly through ToolRegistry without a centralized validation layer. This means that if an LLM is manipulated through prompt injection, it could potentially invoke sensitive tools (e.g. destructive GitHub operations, filesystem actions, etc.).

Proposed Solution

Introduce a lightweight security middleware called ToolGuard that validates tool calls before execution.

The middleware intercepts tool calls inside ToolRegistry.get_executor() and enforces configurable policies such as:

  • optional allowlist for permitted tools
  • destructive tool blocking
  • keyword filtering in tool arguments

Design

Implementation points:

  • Add a ToolGuard module under core/framework/runner/
  • Intercept tool execution in ToolRegistry.get_executor()
  • Return structured ToolResult errors when a call is blocked
  • Make the policy configurable through AgentRunner

Benefits

  • mitigates prompt injection risks
  • prevents destructive tool usage
  • improves safety for production agents
  • keeps enforcement centralized and configurable

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions