-
Notifications
You must be signed in to change notification settings - Fork 30
Expand file tree
/
Copy pathcontrols.yaml
More file actions
1109 lines (1108 loc) · 37.5 KB
/
controls.yaml
File metadata and controls
1109 lines (1108 loc) · 37.5 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
# Copyright 2025 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
title: Controls
description:
- >
The following sections describe controls available to organizations for specific
AI risks.
- >
Each control is mapped onto the corresponding risks it can address, with the exception
of Governance and Assurance controls, which apply universally to all risks and
every stage of the AI development process.
categories:
- id: controlsData
title: Data Controls
- id: controlsInfrastructure
title: Infrastructure Controls
- id: controlsModel
title: Model Controls
- id: controlsApplication
title: Application Controls
- id: controlsAssurance
title: Assurance Controls
- id: controlsGovernance
title: Governance Controls
controls:
- id: controlModelPrivacyEnhancingTechnologies
title: Privacy Enhancing Technologies for Model Training
description:
- >
Use technologies that minimize, de-identify, or restrict use of PII data in
training or evaluating models, including differential privacy, secure aggregation
protocols, federated learning, data anonymization, and synthetic data generation.
category: controlsData
personas:
- personaModelProvider
- personaDataProvider
components:
- componentModelTrainingTuning
- componentModelEvaluation
risks:
- riskSensitiveDataDisclosure
- riskFederatedDistributedTrainingPrivacy
- id: controlRuntimePrivacyEnhancingTechnologies
title: Privacy Enhancing Technologies for Inference
description:
- >
Use technologies that minimize, de-identify, or restrict use of PII data during
model deployment and inference, including secure multi-party computation,
homomorphic encryption, inference log de-identification, on-device processing, and
privacy-preserving query mechanisms.
category: controlsData
personas:
- personaPlatformProvider
- personaModelServing
- personaAgenticProvider
- personaApplicationDeveloper
components:
- componentApplicationOutputHandling
- componentModelServing
risks:
- riskSensitiveDataDisclosure
- riskFederatedDistributedTrainingPrivacy
- id: controlTrainingDataManagement
title: Training Data Management
description:
- >
Ensure that all data used to train and evaluate models is authorized for the
intended purposes.
category: controlsData
personas:
- personaModelProvider
- personaDataProvider
components:
- componentDataSources
- componentTrainingData
- componentModelTrainingTuning
- componentModelEvaluation
risks:
- riskInferredSensitiveData
- riskUnauthorizedTrainingData
- riskExcessiveDataHandling
- id: controlTrainingDataSanitization
title: Training Data Sanitization
description:
- >
Detect and remove or remediate poisoned or sensitive data in training and evaluation.
category: controlsData
personas:
- personaModelProvider
- personaDataProvider
components:
- componentDataFilteringAndProcessing
risks:
- riskDataPoisoning
- riskUnauthorizedTrainingData
- riskRetrievalVectorStorePoisoning
mappings:
mitre-atlas:
- AML.M0007
lifecycleStage:
- data-preparation
- model-training
- evaluation
impactType:
- integrity
- reliability
- safety
- privacy
- fairness
actorAccess:
- supply-chain
- privileged
- id: controlUserDataManagement
title: User Data Management
description:
- >
Store, process, and use all user data (e.g. prompts and logs) from AI applications
in compliance with user consent.
category: controlsData
personas:
- personaDataProvider
- personaApplicationDeveloper
- personaEndUser
components:
- componentDataStorage
risks:
- riskSensitiveDataDisclosure
- riskPromptResponseCachePoisoning
- riskExcessiveDataHandlingDuringInference
- id: controlModelAndDataInventoryManagement
title: Model and Data Inventory Management
description:
- >
Ensure that all data, code, models, and transformation tools used in AI applications
are inventoried and tracked.
category: controlsInfrastructure
personas:
- personaModelProvider
- personaDataProvider
components:
- componentModelServing
- componentModelEvaluation
- componentModelTrainingTuning
- componentModelStorage
risks:
- riskDataPoisoning
- riskModelSourceTampering
- riskModelExfiltration
- id: controlModelAndDataAccessControls
title: Model and Data Access Controls
description:
- >
Minimize internal access to models, weights, datasets, etc. in storage and
in production use.
category: controlsInfrastructure
personas:
- personaModelProvider
- personaDataProvider
- personaModelServing
components:
- componentModelServing
- componentModelEvaluation
- componentModelTrainingTuning
- componentModelStorage
risks:
- riskDataPoisoning
- riskModelSourceTampering
- riskModelExfiltration
- riskAcceleratorAndSystemSideChannels
- riskAdapterPEFTInjection
- riskOrchestratorRouteHijacking
- riskMaliciousLoaderDeserialization
- riskPromptResponseCachePoisoning
- id: controlModelAndDataIntegrityManagement
title: Model and Data Integrity Management
description:
- >
Ensure that all data, models, and code used to produce AI models are verifiably
integrity-protected during development and deployment.
- >
Implement repository trust policies, cryptographic signatures, attestation
frameworks, and offline verification capabilities to ensure integrity
and authenticity of models and artifacts.
category: controlsInfrastructure
personas:
- personaModelProvider
- personaDataProvider
- personaModelServing
components:
- componentModelServing
- componentModelEvaluation
- componentModelTrainingTuning
- componentModelStorage
risks:
- riskAdapterPEFTInjection
- riskCovertChannelsInModelOutputs
- riskDataPoisoning
- riskEvaluationBenchmarkManipulation
- riskFederatedDistributedTrainingPrivacy
- riskMaliciousLoaderDeserialization
- riskModelSourceTampering
- riskModelExfiltration
- riskOrchestratorRouteHijacking
- riskPromptResponseCachePoisoning
- riskRetrievalVectorStorePoisoning
- riskToolSourceProvenance
- riskStaleAgentIdentityBinding
mappings:
mitre-atlas:
- AML.M0013
- AML.M0014
nist-ai-rmf:
- MS-2.3
lifecycleStage:
- data-preparation
- model-training
- deployment
- runtime
impactType:
- integrity
- accountability
actorAccess:
- supply-chain
- privileged
- infrastructure-provider
- id: controlModelAndDataExecutionIntegrity
title: Model and Data Execution Integrity
description:
- >
Ensure that the code and model results from training and development are of known
provenance and lineage and not modified from known-good sources in unexpected ways
at inference time.
- >
Examples include:
- - >
Validating expected code and model signatures / hashes at inference-time.
- >
Limiting and immutably recording all modifications to runtime AI system components
via oversight processes.
- >
Other integrity controls appropriate to the deployment context.
category: controlsInfrastructure
personas:
- personaPlatformProvider
- personaModelServing
components:
- componentModelServing
- componentModelStorage
- componentTheModel
risks:
- riskAdapterPEFTInjection
- riskInsecureIntegratedComponent
- riskModelDeploymentTampering
- riskMaliciousLoaderDeserialization
- riskModelSourceTampering
- id: controlSecureByDefaultMLTooling
title: Secure-by-Default ML Tooling
description:
- >
Use secure-by-default frameworks, libraries, software systems, and hardware
components for AI development or deployment to protect confidentiality and
integrity of AI assets and outputs.
category: controlsInfrastructure
personas:
- personaPlatformProvider
- personaModelServing
- personaAgenticProvider
- personaApplicationDeveloper
components:
- componentModelServing
- componentModelTrainingTuning
- componentModelEvaluation
- componentModelStorage
risks:
- riskDataPoisoning
- riskModelSourceTampering
- riskModelExfiltration
- riskModelDeploymentTampering
- riskAcceleratorAndSystemSideChannels
- riskFederatedDistributedTrainingPrivacy
- riskAdapterPEFTInjection
- riskOrchestratorRouteHijacking
- riskMaliciousLoaderDeserialization
- riskToolSourceProvenance
- riskMCPTransportHijacking
- riskShadowAndUnknownAgents
- riskAgenticDelegationConfusedDeputy
- riskCrossTenantCredentialPropagation
mappings:
mitre-atlas:
- AML.M0011
lifecycleStage:
- development
- model-training
- deployment
- runtime
impactType:
- confidentiality
- integrity
- availability
actorAccess:
- supply-chain
- infrastructure-provider
- id: controlInputValidationAndSanitization
title: Input Validation and Sanitization
description:
- Block or restrict adversarial queries to AI models.
category: controlsModel
personas:
- personaPlatformProvider
- personaModelServing
- personaAgenticProvider
- personaApplicationDeveloper
components:
- componentApplicationInputHandling
- componentAgentInputHandling
- componentOrchestrationInputHandling
risks:
- riskPromptInjection
- riskMaliciousLoaderDeserialization
- riskPromptResponseCachePoisoning
- riskRetrievalVectorStorePoisoning
- riskInsecureIntegratedComponent
mappings:
mitre-atlas:
- AML.M0010
- AML.M0015
- AML.M0020
- AML.M0024
owasp-top10-llm:
- LLM01
lifecycleStage:
- runtime
impactType:
- integrity
- confidentiality
- safety
actorAccess:
- external
- api
- user
- id: controlOutputValidationAndSanitization
title: Output Validation and Sanitization
description:
- >
Block, nullify, or sanitize insecure output from AI models before passing it
to applications, extensions or users.
category: controlsModel
personas:
- personaModelServing
- personaApplicationDeveloper
components:
- componentApplicationOutputHandling
- componentAgentOutputHandling
- componentOrchestrationOutputHandling
risks:
- riskPromptInjection
- riskRogueActions
- riskSensitiveDataDisclosure
- riskInferredSensitiveData
- riskCovertChannelsInModelOutputs
- riskPromptResponseCachePoisoning
- riskInsecureModelOutput
- riskRetrievalVectorStorePoisoning
mappings:
mitre-atlas:
- AML.M0020
- AML.M0024
owasp-top10-llm:
- LLM02
- LLM05
lifecycleStage:
- runtime
impactType:
- confidentiality
- integrity
- privacy
- safety
actorAccess:
- external
- api
- user
- agent
- id: controlAdversarialTrainingAndTesting
title: Adversarial Training and Testing
description:
- >
Use techniques to make AI models robust to adversarial inputs (i.e. prompts)
in the context of their use in applications.
category: controlsModel
personas:
- personaModelProvider
- personaApplicationDeveloper
components:
- componentTheModel
risks:
- riskModelEvasion
- riskPromptInjection
- riskSensitiveDataDisclosure
- riskInferredSensitiveData
- riskInsecureModelOutput
mappings:
mitre-atlas:
- AML.M0003
- AML.M0006
- AML.M0008
lifecycleStage:
- model-training
- evaluation
- runtime
impactType:
- integrity
- reliability
- safety
actorAccess:
- external
- api
- user
- id: controlApplicationAccessManagement
title: Application Access and Resource Management
description:
- >
Ensure comprehensive access governance through:
- >
Identity and authorization controls that restrict resources to authorized
users and endpoints for authorized actions.
- >
Resource governance controls including usage quotas, rate limiting, cost
monitoring, and anomaly detection to prevent resource exhaustion and economic
denial of wallet attacks.
category: controlsApplication
personas:
- personaPlatformProvider
- personaModelServing
- personaApplicationDeveloper
components:
- componentApplication
risks:
- riskDenialOfMLService
- riskEconomicDenialOfWallet
- riskModelReverseEngineering
mappings:
mitre-atlas:
- AML.M0019
owasp-top10-llm:
- LLM10
lifecycleStage:
- runtime
impactType:
- availability
- confidentiality
actorAccess:
- external
- api
- user
- id: controlUserTransparencyAndControls
title: User Transparency and Controls
description:
- >
Inform users of relevant AI risks with disclosures, and provide transparency
and control experiences for use of their data in AI applications.
category: controlsApplication
personas:
- personaModelServing
- personaAgenticProvider
- personaApplicationDeveloper
components:
- componentApplication
risks:
- riskSensitiveDataDisclosure
- riskExcessiveDataHandling
- riskExcessiveDataHandlingDuringInference
- id: controlAgentPluginUserControl
title: Agent User Control
description:
- >
Ensure user approval for any actions performed by agents/plugins that alter user
data or act on the user's behalf.
category: controlsApplication
personas:
- personaAgenticProvider
- personaApplicationDeveloper
- personaEndUser
components:
- componentReasoningCore
risks:
- riskSensitiveDataDisclosure
- riskRogueActions
- id: controlAgentPluginPermissions
title: Agent Permissions
description:
- >
Use least-privilege principle as the upper bound on agentic system permissions to
minimize the number of tools that an agent is permitted to interact with and the
actions it is allowed to take. An agentic system's use of privileges should be
contextual and dynamic, adapting to the specific user query and trusted contextual
information. This design also applies to agents that have access to user
information. For example, an agent asked to fill out a form or answer questions
should share only contextually appropriate information and can be designed to
dynamically minimize exposed data using reference monitors.
- >
At delegation boundaries, an agent acting on behalf of a caller must verify
the caller's authority for the requested action — not rely on the agent's
own (often broader) permissions. Each delegation hop should narrow the
agent's effective authority to the intersection of its own permissions and
the caller's authorization, preventing the agent from being used as a
confused deputy to execute actions the caller could not perform directly.
category: controlsApplication
personas:
- personaAgenticProvider
- personaApplicationDeveloper
components:
- componentReasoningCore
- componentTools
- componentRAGContent
- componentMemory
risks:
- riskInsecureIntegratedComponent
- riskSensitiveDataDisclosure
- riskRogueActions
- riskToolSourceProvenance
- riskAgenticDelegationConfusedDeputy
- riskCrossTenantCredentialPropagation
mappings:
mitre-atlas:
- AML.M0028
nist-ai-rmf:
- MS-2.11
- GV-6.2
owasp-top10-llm:
- LLM06
lifecycleStage:
- development
- runtime
impactType:
- confidentiality
- integrity
- safety
actorAccess:
- agent
- user
- id: controlRedTeaming
title: Red Teaming
description:
- >
Drive security and privacy improvements through self-driven adversarial attacks
on AI infrastructure and products
category: controlsAssurance
personas:
- personaModelProvider
- personaPlatformProvider
- personaModelServing
- personaAgenticProvider
- personaApplicationDeveloper
components: all
risks: all
- id: controlVulnerabilityManagement
title: Vulnerability Management
description:
- >
Proactively and continually test and monitor production infrastructure and
products for security and privacy regressions
category: controlsAssurance
personas:
- personaGovernance
- personaEndUser
components: all
risks: all
- id: controlThreatDetection
title: Threat Detection
description:
- >
Detect and alert on internal or external attacks on AI assets, infrastructure,
and products
category: controlsAssurance
personas:
- personaGovernance
components: all
risks: all
- id: controlIncidentResponseManagement
title: Incident Response Management
description:
- Manage response to AI security and privacy incidents
category: controlsAssurance
personas:
- personaModelProvider
- personaDataProvider
- personaPlatformProvider
- personaModelServing
- personaAgenticProvider
- personaApplicationDeveloper
- personaGovernance
- personaEndUser
components: all
risks: all
- id: controlUserPoliciesAndEducation
title: User Policies and Education
description:
- >
Publish easy to understand AI security and privacy policies and education for
users.
category: controlsGovernance
personas:
- personaGovernance
- personaEndUser
components: none
risks:
- riskSensitiveDataDisclosure
- riskInsecureIntegratedComponent
- id: controlInternalPoliciesAndEducation
title: Internal Policies and Education
description:
- >
Publish comprehensive AI security and privacy policies and education for your
employees.
category: controlsGovernance
personas:
- personaGovernance
components: none
risks: all
- id: controlProductGovernance
title: Product Governance
description:
- >
Validate that all AI models and products meet the established security and privacy
requirements.
category: controlsGovernance
personas:
- personaGovernance
components: none
risks: all
- id: controlRiskGovernance
title: Risk Governance
description:
- >
Inventory, measure, and monitor residual risk to AI in your organization.
category: controlsGovernance
personas:
- personaGovernance
components: none
risks: all
- id: controlAgentObservability
title: Agent Observability
description:
- >
Ensure an agent's actions, tool use, and reasoning are transparent and auditable
through logging, allowing for debugging, security oversight, and user insights
into agent activity.
category: controlsApplication
personas:
- personaAgenticProvider
- personaApplicationDeveloper
components:
- componentOrchestrationInputHandling
- componentOrchestrationOutputHandling
- componentAgentInputHandling
- componentAgentOutputHandling
- componentReasoningCore
risks:
- riskSensitiveDataDisclosure
- riskRogueActions
- riskToolRegistryTampering
- riskAgentDelegationChainOpacity
- riskShadowAndUnknownAgents
- riskRunawayAgentToolLoops
- riskZombieShadowMCPServers
- riskAgenticDelegationConfusedDeputy
- id: controlIsolatedConfidentialComputing
title: Isolated and Confidential Computing
description:
- >
Use technologies that minimize the risk of 3rd-party access to critical resources
via hardware isolation invariants and/or physically isolated computing systems.
- >
Examples include including CPU and TPU-based confidential computing technologies
(TEEs, secure enclaves), system isolation, secure scheduling, and side-channel
monitoring to prevent cross-tenant information leakage.
category: controlsInfrastructure
personas:
- personaPlatformProvider
- personaModelServing
- personaAgenticProvider
- personaApplicationDeveloper
components:
- componentModelServing
- componentModelTrainingTuning
- componentMemory
- componentReasoningCore
risks:
- riskAcceleratorAndSystemSideChannels
- riskModelSourceTampering
- riskModelExfiltration
- riskModelDeploymentTampering
- riskOrchestratorRouteHijacking
- riskCrossTenantCredentialPropagation
- id: controlRetrievalAndVectorSystemIntegrity
title: Retrieval and Vector System Integrity Management
description:
- >
Implement provenance tracking, cryptographic signing, deduplication,
embedding anomaly detection, and index integrity checks to protect
retrieval systems and vector databases from poisoning attacks.
category: controlsData
personas:
- personaDataProvider
- personaPlatformProvider
- personaApplicationDeveloper
components:
- componentDataSources
- componentDataStorage
- componentDataFilteringAndProcessing
risks:
- riskRetrievalVectorStorePoisoning
- id: controlOrchestratorAndRouteIntegrity
title: Orchestrator and Route Integrity
description:
- >
Implement signed route manifests, configuration integrity verification,
and response provenance tracking to prevent routing manipulation and
ensure request routing to authorized models.
category: controlsInfrastructure
personas:
- personaPlatformProvider
- personaModelServing
- personaAgenticProvider
- personaApplicationDeveloper
components:
- componentModelServing
- componentApplication
risks:
- riskOrchestratorRouteHijacking
- riskModelDeploymentTampering
- riskToolRegistryTampering
- id: controlAgentInventoryManagement
title: Agent Inventory Management
description:
- >
Ensure that all agents operating in the system are registered in a
central inventory before credential issuance, with lifecycle state
tracking (active, suspended, decommissioned) and automated detection
of unregistered agents. This control addresses agent identity
governance — knowing what agents are authorized to exist — as distinct
from agent observability, which addresses what agents do.
- >
This control extends the inventory management concept from
{{controlModelAndDataInventoryManagement}} to dynamic, running agent
entities. Where model/data inventory tracks static artifacts through
build and deployment pipelines, agent inventory tracks entities with
operational lifecycle states in a live system. Enforcement includes
credential revocation on lifecycle state change and blocking of
unregistered agent actions. The same inventory discipline extends
to MCP tool server endpoints, providing the lifecycle authority on
which runtime provenance and observability controls depend.
category: controlsApplication
personas:
- personaAgenticProvider
- personaPlatformProvider
- personaGovernance
components:
- componentReasoningCore
- componentTools
- componentOrchestrationInputHandling
- componentOrchestrationOutputHandling
risks:
- riskShadowAndUnknownAgents
- riskZombieShadowMCPServers
- riskAgentDelegationChainOpacity
mappings:
nist-ai-rmf:
- GV-1.6
- MP-3.4
lifecycleStage:
- deployment
- runtime
- maintenance
impactType:
- accountability
- compliance
- confidentiality
- integrity
actorAccess:
- agent
- privileged
- infrastructure-provider
- id: controlAgentIntegrityManagement
title: Agent Integrity Management
description:
- >
Ensure that each running agent's identity and configuration are
cryptographically bound to its registered inventory entry and
verifiable at runtime. This includes credential binding to individual
agent identities (not shared pools), attestation of agent
configuration against a registered baseline, and automated
revocation propagation on decommission or underlying model swap.
- >
This control extends the integrity management concept from
{{controlModelAndDataIntegrityManagement}} to agent identity. Where
model/data integrity verifies that artifact content (weights, code)
has not been tampered with, agent integrity verifies that a running
agent's identity credentials match its registered entry. A model can
pass artifact integrity checks while the agent wrapping it has
drifted from its registered configuration or had its credential
bindings compromised.
category: controlsApplication
personas:
- personaAgenticProvider
- personaPlatformProvider
- personaGovernance
components:
- componentReasoningCore
- componentTools
- componentOrchestrationInputHandling
- componentModelServing
risks:
- riskShadowAndUnknownAgents
- riskStaleAgentIdentityBinding
- riskAgentDelegationChainOpacity
mappings:
mitre-atlas:
- AML.M0013
nist-ai-rmf:
- MP-3.4
- MS-2.9
lifecycleStage:
- deployment
- runtime
- maintenance
impactType:
- accountability
- integrity
- confidentiality
actorAccess:
- agent
- privileged
- infrastructure-provider
- id: controlAgentCredentialIsolation
title: Agent Credential Isolation
description:
- >
Ensure that credentials issued to agents are scoped to individual
tenants and cannot propagate across tenant boundaries at any point
in an orchestration or delegation chain. This control addresses the
credential issuance and propagation properties of agents — answering
"are this agent's credentials structurally isolated to its tenant?"
— as distinct from agent integrity (which attests that an agent
matches its registered identity) and agent permissions (which
governs the action scope of tool calls).
- >
Implementation requires: per-tenant credential issuance with no
shared credential pools across tenant boundaries; short-lived,
tenant-scoped tokens for agent-to-tool authentication, with
lifetimes bounded to the duration of a single task or session;
tenant boundary validation at every MCP protocol hop and delegation
step, so that a credential issued in tenant A cannot be forwarded
to or reused in tenant B's context; and credential rotation and
revocation scoped to tenant context, ensuring that decommissioning
an agent in one tenant does not leave credentials valid in another.
category: controlsApplication
personas:
- personaAgenticProvider
- personaPlatformProvider
- personaApplicationDeveloper
- personaGovernance
components:
- componentReasoningCore
- componentOrchestrationInputHandling
- componentOrchestrationOutputHandling
- componentTools
- componentMemory
risks:
- riskCrossTenantCredentialPropagation
- riskShadowAndUnknownAgents
- riskStaleAgentIdentityBinding
mappings:
owasp-top10-llm:
- LLM06
nist-ai-rmf:
- MS-2.10
- GV-6.2
lifecycleStage:
- deployment
- runtime
impactType:
- confidentiality
- integrity
- compliance
- privacy
actorAccess:
- agent
- privileged
- infrastructure-provider
- id: controlInterComponentTransportSecurity
title: Inter-Component Transport Security
description:
- >
Enforce authenticated, encrypted, and integrity-protected communication
channels for all inter-component connections in AI systems, including
agent-to-tool, model serving, orchestration, and application API
channels.
- >
Implement the following transport-layer controls across all component
communication paths:
- - >
TLS/mTLS enforcement: Require TLS 1.2 minimum (TLS 1.3 preferred)
for all inter-component connections. Require mutual TLS (mTLS) for
machine-to-machine paths — agent-to-tool server, orchestrator-to-model
serving endpoint, and application-to-serving API — so that both
parties authenticate before any payload is exchanged. Locally-exposed
tool server endpoints (e.g., MCP servers on localhost) are in scope
and must not be treated as implicitly trusted.
- >
Mutual authentication: Each component must present a valid certificate
or credential verified against an expected identity before the
connection is accepted. This applies to machine-to-machine paths,
not user-facing authentication (covered by application access
controls).
- >
Session token binding: Bind session tokens and API credentials to
transport-layer identifiers (e.g., TLS channel binding per RFC 5929)
so that a token captured from one TLS session cannot be replayed on
a different channel.
- >
Replay protection: Apply nonce, timestamp, or sequence number
mechanisms to tool invocation requests and inter-component messages
carrying side-effecting instructions, preventing captured messages
from being reinjected.
- >
Origin validation: Validate the origin of requests to locally-exposed
endpoints against an allowlist of expected callers, preventing DNS
rebinding attacks.
- >
Certificate lifecycle management: Issue individual certificates per
component identity rather than shared wildcards. Automate certificate
rotation before expiry. Revoke certificates immediately on component
decommission or credential compromise.
- >
This control addresses the transport layer — how component messages are
protected in transit. It is distinct from {{controlOrchestratorAndRouteIntegrity}}
(which governs where traffic is routed), {{controlSecureByDefaultMLTooling}}
(which governs tooling selection and configuration), and
{{controlIsolatedConfidentialComputing}} (which governs hardware-level
compute isolation). Those controls assume secure channels; this control
establishes them.
category: controlsInfrastructure
personas:
- personaPlatformProvider
- personaModelServing
- personaAgenticProvider
- personaApplicationDeveloper
components:
- componentModelServing
- componentTools
- componentOrchestrationInputHandling
- componentOrchestrationOutputHandling
- componentApplication
risks:
- riskMCPTransportHijacking
- riskOrchestratorRouteHijacking
- riskInsecureIntegratedComponent
- riskToolRegistryTampering
- riskModelDeploymentTampering
- riskZombieShadowMCPServers
mappings:
nist-ai-rmf:
- MS-2.7
- MP-3.4
owasp-top10-llm:
- LLM03
lifecycleStage:
- deployment
- runtime
impactType:
- confidentiality
- integrity
actorAccess:
- external
- agent
- privileged
- infrastructure-provider
- id: controlComponentIdentityProvenance
title: Component Identity Provenance
description:
- >
Ensure that when one component invokes another, the calling component
can verify the callee is the currently registered, non-revoked
principal for that role — not merely the holder of a key for a
domain or network address. This control closes the gap that transport
security leaves open: TLS and mTLS prove key possession for an