Skip to content

Latest commit

 

History

History
37 lines (37 loc) · 131 KB

File metadata and controls

37 lines (37 loc) · 131 KB
id title description category personas components risks mappings lifecycleStage impactType actorAccess
controlAdversarialTrainingAndTesting Adversarial Training and Testing Use techniques to make AI models robust to adversarial inputs (i.e. prompts) in the context of their use in applications.
controlsModel personaModelProvider
personaApplicationDeveloper
componentTheModel riskModelEvasion
riskPromptInjection
riskSensitiveDataDisclosure
riskInferredSensitiveData
riskInsecureModelOutput
mitre-atlas: AML.M0003, AML.M0006, AML.M0008 model-training
evaluation
runtime
integrity
reliability
safety
external
api
user
controlAgentCredentialIsolation Agent Credential Isolation Ensure that credentials issued to agents are scoped to individual tenants and cannot propagate across tenant boundaries at any point in an orchestration or delegation chain. This control addresses the credential issuance and propagation properties of agents — answering "are this agent's credentials structurally isolated to its tenant?" — as distinct from agent integrity (which attests that an agent matches its registered identity) and agent permissions (which governs the action scope of tool calls).

Implementation requires: per-tenant credential issuance with no shared credential pools across tenant boundaries; short-lived, tenant-scoped tokens for agent-to-tool authentication, with lifetimes bounded to the duration of a single task or session; tenant boundary validation at every MCP protocol hop and delegation step, so that a credential issued in tenant A cannot be forwarded to or reused in tenant B's context; and credential rotation and revocation scoped to tenant context, ensuring that decommissioning an agent in one tenant does not leave credentials valid in another.
controlsApplication personaAgenticProvider
personaPlatformProvider
personaApplicationDeveloper
personaGovernance
componentReasoningCore
componentOrchestrationInputHandling
componentOrchestrationOutputHandling
componentTools
componentMemory
riskCrossTenantCredentialPropagation
riskShadowAndUnknownAgents
riskStaleAgentIdentityBinding
owasp-top10-llm: LLM06
nist-ai-rmf: MS-2.10, GV-6.2
deployment
runtime
confidentiality
integrity
compliance
privacy
agent
privileged
infrastructure-provider
controlAgentExecutionBounds Agent Execution Bounds Enforce per-agent, per-session, and per-tool-call execution limits at the orchestration layer to prevent runaway tool-calling cycles. This control addresses the enforcement of loop bounds, context-window budgets, and circuit breakers within agent reasoning and tool invocation sequences — as distinct from API-layer rate limiting (which operates at the external request boundary) and agent observability (which detects loops but does not halt them).

Implementation requires: mandatory max_turns defaults for all agent-to-tool execution sequences, with no framework permitting unbounded execution by default; per-agent and per-session tool-call budgets that are enforced by the orchestration layer before each tool invocation; context-window budget tracking that halts agent execution before context saturation degrades reasoning quality; and circuit breakers that terminate agent execution when cumulative resource thresholds (compute, tokens, wall-clock time) are exceeded. Loop detection at the orchestration layer should identify repeated tool-call patterns and trigger early termination.
controlsApplication personaAgenticProvider
personaApplicationDeveloper
personaPlatformProvider
componentReasoningCore
componentOrchestrationInputHandling
componentOrchestrationOutputHandling
riskRunawayAgentToolLoops
riskEconomicDenialOfWallet
owasp-top10-llm: LLM10
mitre-atlas: AML.M0004
nist-ai-rmf: MS-2.6, MG-2.1
deployment
runtime
availability
safety
agent
controlAgentIntegrityManagement Agent Integrity Management Ensure that each running agent's identity and configuration are cryptographically bound to its registered inventory entry and verifiable at runtime. This includes credential binding to individual agent identities (not shared pools), attestation of agent configuration against a registered baseline, and automated revocation propagation on decommission or underlying model swap.

This control extends the integrity management concept from Model and Data Integrity Management to agent identity. Where model/data integrity verifies that artifact content (weights, code) has not been tampered with, agent integrity verifies that a running agent's identity credentials match its registered entry. A model can pass artifact integrity checks while the agent wrapping it has drifted from its registered configuration or had its credential bindings compromised.
controlsApplication personaAgenticProvider
personaPlatformProvider
personaGovernance
componentReasoningCore
componentTools
componentOrchestrationInputHandling
componentModelServing
riskShadowAndUnknownAgents
riskStaleAgentIdentityBinding
riskAgentDelegationChainOpacity
mitre-atlas: AML.M0013
nist-ai-rmf: MP-3.4, MS-2.9
deployment
runtime
maintenance
accountability
integrity
confidentiality
agent
privileged
infrastructure-provider
controlAgentInventoryManagement Agent Inventory Management Ensure that all agents operating in the system are registered in a central inventory before credential issuance, with lifecycle state tracking (active, suspended, decommissioned) and automated detection of unregistered agents. This control addresses agent identity governance — knowing what agents are authorized to exist — as distinct from agent observability, which addresses what agents do.

This control extends the inventory management concept from Model and Data Inventory Management to dynamic, running agent entities. Where model/data inventory tracks static artifacts through build and deployment pipelines, agent inventory tracks entities with operational lifecycle states in a live system. Enforcement includes credential revocation on lifecycle state change and blocking of unregistered agent actions. The same inventory discipline extends to MCP tool server endpoints, providing the lifecycle authority on which runtime provenance and observability controls depend.
controlsApplication personaAgenticProvider
personaPlatformProvider
personaGovernance
componentReasoningCore
componentTools
componentOrchestrationInputHandling
componentOrchestrationOutputHandling
riskShadowAndUnknownAgents
riskZombieShadowMCPServers
riskAgentDelegationChainOpacity
nist-ai-rmf: GV-1.6, MP-3.4 deployment
runtime
maintenance
accountability
compliance
confidentiality
integrity
agent
privileged
infrastructure-provider
controlAgentObservability Agent Observability Ensure an agent's actions, tool use, and reasoning are transparent and auditable through logging, allowing for debugging, security oversight, and user insights into agent activity.
controlsApplication personaAgenticProvider
personaApplicationDeveloper
componentOrchestrationInputHandling
componentOrchestrationOutputHandling
componentAgentInputHandling
componentAgentOutputHandling
componentReasoningCore
riskSensitiveDataDisclosure
riskRogueActions
riskToolRegistryTampering
riskAgentDelegationChainOpacity
riskShadowAndUnknownAgents
riskRunawayAgentToolLoops
riskZombieShadowMCPServers
riskAgenticDelegationConfusedDeputy
controlAgentPluginPermissions Agent Permissions Use least-privilege principle as the upper bound on agentic system permissions to minimize the number of tools that an agent is permitted to interact with and the actions it is allowed to take. An agentic system's use of privileges should be contextual and dynamic, adapting to the specific user query and trusted contextual information. This design also applies to agents that have access to user information. For example, an agent asked to fill out a form or answer questions should share only contextually appropriate information and can be designed to dynamically minimize exposed data using reference monitors.

At delegation boundaries, an agent acting on behalf of a caller must verify the caller's authority for the requested action — not rely on the agent's own (often broader) permissions. Each delegation hop should narrow the agent's effective authority to the intersection of its own permissions and the caller's authorization, preventing the agent from being used as a confused deputy to execute actions the caller could not perform directly.
controlsApplication personaAgenticProvider
personaApplicationDeveloper
componentReasoningCore
componentTools
componentRAGContent
componentMemory
riskInsecureIntegratedComponent
riskSensitiveDataDisclosure
riskRogueActions
riskToolSourceProvenance
riskAgenticDelegationConfusedDeputy
riskCrossTenantCredentialPropagation
mitre-atlas: AML.M0028
nist-ai-rmf: MS-2.11, GV-6.2
owasp-top10-llm: LLM06
development
runtime
confidentiality
integrity
safety
agent
user
controlAgentPluginUserControl Agent User Control Ensure user approval for any actions performed by agents/plugins that alter user data or act on the user's behalf.
controlsApplication personaAgenticProvider
personaApplicationDeveloper
personaEndUser
componentReasoningCore riskSensitiveDataDisclosure
riskRogueActions
controlApplicationAccessManagement Application Access and Resource Management Ensure comprehensive access governance through:

Identity and authorization controls that restrict resources to authorized users and endpoints for authorized actions.

Resource governance controls including usage quotas, rate limiting, cost monitoring, and anomaly detection to prevent resource exhaustion and economic denial of wallet attacks.
controlsApplication personaPlatformProvider
personaModelServing
personaApplicationDeveloper
componentApplication riskDenialOfMLService
riskEconomicDenialOfWallet
riskModelReverseEngineering
mitre-atlas: AML.M0019
owasp-top10-llm: LLM10
runtime availability
confidentiality
external
api
user
controlComponentIdentityProvenance Component Identity Provenance Ensure that when one component invokes another, the calling component can verify the callee is the currently registered, non-revoked principal for that role — not merely the holder of a key for a domain or network address. This control closes the gap that transport security leaves open: TLS and mTLS prove key possession for an endpoint, but do not attest that the endpoint is the legitimate principal a registry system of record says it should be.

Implementation requires: a binding between a component's runtime identity and an authoritative registry entry, such that revocation in the registry is observable to callers within a bounded time window; verification of that binding at invocation time, not solely at channel establishment; and audit telemetry that records the provenance assertion used so post-hoc detection of calls against revoked or expired principals is possible.

This control is mechanism-agnostic. Implementers may use SPIFFE/SVIDs, capability tokens, signed registration receipts, mTLS with a private CA whose issuance is gated on registry state, short-lived JWTs from a registration authority, hardware-attested identities, or other approaches that satisfy the intent. The control specifies the question that must be answered at invocation time — "is this the registered principal?" — not the cryptographic protocol used to answer it.

This control is distinct from Inter-Component Transport Security, which secures the channel, and from Agent Inventory Management, which maintains the registry of authorized principals. Inventory is the lifecycle authority; provenance is the runtime attestation that surfaces inventory state to callers; transport is the channel over which both ride.
controlsInfrastructure personaPlatformProvider
personaAgenticProvider
personaApplicationDeveloper
personaModelServing
personaGovernance
componentOrchestrationInputHandling
componentOrchestrationOutputHandling
componentTools
componentModelServing
componentApplication
riskZombieShadowMCPServers
riskStaleAgentIdentityBinding
riskMCPTransportHijacking
nist-ai-rmf: MP-3.4, MS-2.7 deployment
runtime
maintenance
integrity
confidentiality
accountability
external
supply-chain
agent
controlIncidentResponseManagement Incident Response Management Manage response to AI security and privacy incidents controlsAssurance personaModelProvider
personaDataProvider
personaPlatformProvider
personaModelServing
personaAgenticProvider
personaApplicationDeveloper
personaGovernance
personaEndUser
all all
controlInputValidationAndSanitization Input Validation and Sanitization Block or restrict adversarial queries to AI models. controlsModel personaPlatformProvider
personaModelServing
personaAgenticProvider
personaApplicationDeveloper
componentApplicationInputHandling
componentAgentInputHandling
componentOrchestrationInputHandling
riskPromptInjection
riskMaliciousLoaderDeserialization
riskPromptResponseCachePoisoning
riskRetrievalVectorStorePoisoning
riskInsecureIntegratedComponent
mitre-atlas: AML.M0010, AML.M0015, AML.M0020, AML.M0024
owasp-top10-llm: LLM01
runtime integrity
confidentiality
safety
external
api
user
controlInterComponentTransportSecurity Inter-Component Transport Security Enforce authenticated, encrypted, and integrity-protected communication channels for all inter-component connections in AI systems, including agent-to-tool, model serving, orchestration, and application API channels.

Implement the following transport-layer controls across all component communication paths:

TLS/mTLS enforcement: Require TLS 1.2 minimum (TLS 1.3 preferred) for all inter-component connections. Require mutual TLS (mTLS) for machine-to-machine paths — agent-to-tool server, orchestrator-to-model serving endpoint, and application-to-serving API — so that both parties authenticate before any payload is exchanged. Locally-exposed tool server endpoints (e.g., MCP servers on localhost) are in scope and must not be treated as implicitly trusted.

Mutual authentication: Each component must present a valid certificate or credential verified against an expected identity before the connection is accepted. This applies to machine-to-machine paths, not user-facing authentication (covered by application access controls).

Session token binding: Bind session tokens and API credentials to transport-layer identifiers (e.g., TLS channel binding per RFC 5929) so that a token captured from one TLS session cannot be replayed on a different channel.

Replay protection: Apply nonce, timestamp, or sequence number mechanisms to tool invocation requests and inter-component messages carrying side-effecting instructions, preventing captured messages from being reinjected.

Origin validation: Validate the origin of requests to locally-exposed endpoints against an allowlist of expected callers, preventing DNS rebinding attacks.

Certificate lifecycle management: Issue individual certificates per component identity rather than shared wildcards. Automate certificate rotation before expiry. Revoke certificates immediately on component decommission or credential compromise.

This control addresses the transport layer — how component messages are protected in transit. It is distinct from Orchestrator and Route Integrity (which governs where traffic is routed), Secure-by-Default ML Tooling (which governs tooling selection and configuration), and Isolated and Confidential Computing (which governs hardware-level compute isolation). Those controls assume secure channels; this control establishes them.
controlsInfrastructure personaPlatformProvider
personaModelServing
personaAgenticProvider
personaApplicationDeveloper
componentModelServing
componentTools
componentOrchestrationInputHandling
componentOrchestrationOutputHandling
componentApplication
riskMCPTransportHijacking
riskOrchestratorRouteHijacking
riskInsecureIntegratedComponent
riskToolRegistryTampering
riskModelDeploymentTampering
riskZombieShadowMCPServers
nist-ai-rmf: MS-2.7, MP-3.4
owasp-top10-llm: LLM03
deployment
runtime
confidentiality
integrity
external
agent
privileged
infrastructure-provider
controlInternalPoliciesAndEducation Internal Policies and Education Publish comprehensive AI security and privacy policies and education for your employees.
controlsGovernance personaGovernance none all
controlIsolatedConfidentialComputing Isolated and Confidential Computing Use technologies that minimize the risk of 3rd-party access to critical resources via hardware isolation invariants and/or physically isolated computing systems.

Examples include including CPU and TPU-based confidential computing technologies (TEEs, secure enclaves), system isolation, secure scheduling, and side-channel monitoring to prevent cross-tenant information leakage.
controlsInfrastructure personaPlatformProvider
personaModelServing
personaAgenticProvider
personaApplicationDeveloper
componentModelServing
componentModelTrainingTuning
componentMemory
componentReasoningCore
riskAcceleratorAndSystemSideChannels
riskModelSourceTampering
riskModelExfiltration
riskModelDeploymentTampering
riskOrchestratorRouteHijacking
riskCrossTenantCredentialPropagation
controlModelAndDataAccessControls Model and Data Access Controls Minimize internal access to models, weights, datasets, etc. in storage and in production use.
controlsInfrastructure personaModelProvider
personaDataProvider
personaModelServing
componentModelServing
componentModelEvaluation
componentModelTrainingTuning
componentModelStorage
riskDataPoisoning
riskModelSourceTampering
riskModelExfiltration
riskAcceleratorAndSystemSideChannels
riskAdapterPEFTInjection
riskOrchestratorRouteHijacking
riskMaliciousLoaderDeserialization
riskPromptResponseCachePoisoning
controlModelAndDataExecutionIntegrity Model and Data Execution Integrity Ensure that the code and model results from training and development are of known provenance and lineage and not modified from known-good sources in unexpected ways at inference time.

Examples include:

Validating expected code and model signatures / hashes at inference-time.

Limiting and immutably recording all modifications to runtime AI system components via oversight processes.

Other integrity controls appropriate to the deployment context.
controlsInfrastructure personaPlatformProvider
personaModelServing
componentModelServing
componentModelStorage
componentTheModel
riskAdapterPEFTInjection
riskInsecureIntegratedComponent
riskModelDeploymentTampering
riskMaliciousLoaderDeserialization
riskModelSourceTampering
controlModelAndDataIntegrityManagement Model and Data Integrity Management Ensure that all data, models, and code used to produce AI models are verifiably integrity-protected during development and deployment.

Implement repository trust policies, cryptographic signatures, attestation frameworks, and offline verification capabilities to ensure integrity and authenticity of models and artifacts.
controlsInfrastructure personaModelProvider
personaDataProvider
personaModelServing
componentModelServing
componentModelEvaluation
componentModelTrainingTuning
componentModelStorage
riskAdapterPEFTInjection
riskCovertChannelsInModelOutputs
riskDataPoisoning
riskEvaluationBenchmarkManipulation
riskFederatedDistributedTrainingPrivacy
riskMaliciousLoaderDeserialization
riskModelSourceTampering
riskModelExfiltration
riskOrchestratorRouteHijacking
riskPromptResponseCachePoisoning
riskRetrievalVectorStorePoisoning
riskToolSourceProvenance
riskStaleAgentIdentityBinding
mitre-atlas: AML.M0013, AML.M0014
nist-ai-rmf: MS-2.3
data-preparation
model-training
deployment
runtime
integrity
accountability
supply-chain
privileged
infrastructure-provider
controlModelAndDataInventoryManagement Model and Data Inventory Management Ensure that all data, code, models, and transformation tools used in AI applications are inventoried and tracked.
controlsInfrastructure personaModelProvider
personaDataProvider
componentModelServing
componentModelEvaluation
componentModelTrainingTuning
componentModelStorage
riskDataPoisoning
riskModelSourceTampering
riskModelExfiltration
controlModelPrivacyEnhancingTechnologies Privacy Enhancing Technologies for Model Training Use technologies that minimize, de-identify, or restrict use of PII data in training or evaluating models, including differential privacy, secure aggregation protocols, federated learning, data anonymization, and synthetic data generation.
controlsData personaModelProvider
personaDataProvider
componentModelTrainingTuning
componentModelEvaluation
riskSensitiveDataDisclosure
riskFederatedDistributedTrainingPrivacy
controlOrchestratorAndRouteIntegrity Orchestrator and Route Integrity Implement signed route manifests, configuration integrity verification, and response provenance tracking to prevent routing manipulation and ensure request routing to authorized models.
controlsInfrastructure personaPlatformProvider
personaModelServing
personaAgenticProvider
personaApplicationDeveloper
componentModelServing
componentApplication
riskOrchestratorRouteHijacking
riskModelDeploymentTampering
riskToolRegistryTampering
controlOutputValidationAndSanitization Output Validation and Sanitization Block, nullify, or sanitize insecure output from AI models before passing it to applications, extensions or users.
controlsModel personaModelServing
personaApplicationDeveloper
componentApplicationOutputHandling
componentAgentOutputHandling
componentOrchestrationOutputHandling
riskPromptInjection
riskRogueActions
riskSensitiveDataDisclosure
riskInferredSensitiveData
riskCovertChannelsInModelOutputs
riskPromptResponseCachePoisoning
riskInsecureModelOutput
riskRetrievalVectorStorePoisoning
mitre-atlas: AML.M0020, AML.M0024
owasp-top10-llm: LLM02, LLM05
runtime confidentiality
integrity
privacy
safety
external
api
user
agent
controlProductGovernance Product Governance Validate that all AI models and products meet the established security and privacy requirements.
controlsGovernance personaGovernance none all
controlRedTeaming Red Teaming Drive security and privacy improvements through self-driven adversarial attacks on AI infrastructure and products
controlsAssurance personaModelProvider
personaPlatformProvider
personaModelServing
personaAgenticProvider
personaApplicationDeveloper
all all
controlRetrievalAndVectorSystemIntegrity Retrieval and Vector System Integrity Management Implement provenance tracking, cryptographic signing, deduplication, embedding anomaly detection, and index integrity checks to protect retrieval systems and vector databases from poisoning attacks.
controlsData personaDataProvider
personaPlatformProvider
personaApplicationDeveloper
componentDataSources
componentDataStorage
componentDataFilteringAndProcessing
riskRetrievalVectorStorePoisoning
controlRiskGovernance Risk Governance Inventory, measure, and monitor residual risk to AI in your organization.
controlsGovernance personaGovernance none all
controlRuntimePrivacyEnhancingTechnologies Privacy Enhancing Technologies for Inference Use technologies that minimize, de-identify, or restrict use of PII data during model deployment and inference, including secure multi-party computation, homomorphic encryption, inference log de-identification, on-device processing, and privacy-preserving query mechanisms.
controlsData personaPlatformProvider
personaModelServing
personaAgenticProvider
personaApplicationDeveloper
componentApplicationOutputHandling
componentModelServing
riskSensitiveDataDisclosure
riskFederatedDistributedTrainingPrivacy
controlSecureByDefaultMLTooling Secure-by-Default ML Tooling Use secure-by-default frameworks, libraries, software systems, and hardware components for AI development or deployment to protect confidentiality and integrity of AI assets and outputs.
controlsInfrastructure personaPlatformProvider
personaModelServing
personaAgenticProvider
personaApplicationDeveloper
componentModelServing
componentModelTrainingTuning
componentModelEvaluation
componentModelStorage
riskDataPoisoning
riskModelSourceTampering
riskModelExfiltration
riskModelDeploymentTampering
riskAcceleratorAndSystemSideChannels
riskFederatedDistributedTrainingPrivacy
riskAdapterPEFTInjection
riskOrchestratorRouteHijacking
riskMaliciousLoaderDeserialization
riskToolSourceProvenance
riskMCPTransportHijacking
riskShadowAndUnknownAgents
riskAgenticDelegationConfusedDeputy
riskCrossTenantCredentialPropagation
mitre-atlas: AML.M0011 development
model-training
deployment
runtime
confidentiality
integrity
availability
supply-chain
infrastructure-provider
controlThreatDetection Threat Detection Detect and alert on internal or external attacks on AI assets, infrastructure, and products
controlsAssurance personaGovernance all all
controlTrainingDataManagement Training Data Management Ensure that all data used to train and evaluate models is authorized for the intended purposes.
controlsData personaModelProvider
personaDataProvider
componentDataSources
componentTrainingData
componentModelTrainingTuning
componentModelEvaluation
riskInferredSensitiveData
riskUnauthorizedTrainingData
riskExcessiveDataHandling
controlTrainingDataSanitization Training Data Sanitization Detect and remove or remediate poisoned or sensitive data in training and evaluation.
controlsData personaModelProvider
personaDataProvider
componentDataFilteringAndProcessing riskDataPoisoning
riskUnauthorizedTrainingData
riskRetrievalVectorStorePoisoning
mitre-atlas: AML.M0007 data-preparation
model-training
evaluation
integrity
reliability
safety
privacy
fairness
supply-chain
privileged
controlUserDataManagement User Data Management Store, process, and use all user data (e.g. prompts and logs) from AI applications in compliance with user consent.
controlsData personaDataProvider
personaApplicationDeveloper
personaEndUser
componentDataStorage riskSensitiveDataDisclosure
riskPromptResponseCachePoisoning
riskExcessiveDataHandlingDuringInference
controlUserPoliciesAndEducation User Policies and Education Publish easy to understand AI security and privacy policies and education for users.
controlsGovernance personaGovernance
personaEndUser
none riskSensitiveDataDisclosure
riskInsecureIntegratedComponent
controlUserTransparencyAndControls User Transparency and Controls Inform users of relevant AI risks with disclosures, and provide transparency and control experiences for use of their data in AI applications.
controlsApplication personaModelServing
personaAgenticProvider
personaApplicationDeveloper
componentApplication riskSensitiveDataDisclosure
riskExcessiveDataHandling
riskExcessiveDataHandlingDuringInference
controlVulnerabilityManagement Vulnerability Management Proactively and continually test and monitor production infrastructure and products for security and privacy regressions
controlsAssurance personaGovernance
personaEndUser
all all