RFC: Shared Responsibility Model for AI

[imolloy]

Shared Responsibility Model for AI

Authors:

• @billstout-snc
• @wildelaw

Summary

We propose a structured AI Shared Responsibility Model that clearly delineates accountability across three key stakeholder groups: Customers, Service Providers (Deployers), and Model Providers. This model reflects the real-world operational layers in AI deployments and aligns with emerging governance frameworks such as the EU AI Act, NIST AI RMF, and CSA GenAI SRM.
Customers and Users are responsible for defining appropriate AI use, overseeing ethical deployment, and managing internal users, acceptable use, and organizational risks. Customers can be further grouped by regulated industry or domain safety.

Service Providers operate the stack—from infrastructure and platform software to application delivery, AI and agentic system red teaming, and model fine-tuning—ensuring secure, compliant, and reliable AI services. These can be organized in application, platform, and infrastructure layers of responsibility.

Model Providers supply the foundational models and datasets (disclosed or not), and bear responsibility for model provenance, model red teaming, and disclosure obligations.
This layered model supports traceability, clarifies risk ownership, and enables enterprise-wide compliance. This may also clarify who within a company is responsible for what, and what regulatory controls apply to what section of a provider, and what section of a customer.
In AI, there is more uncertainty and ambiguity in the layers of the stack, requiring more explicit scoping and communication for responsibilities for security. Further, the autonomy provided by AI (e.g., agents) crosses organizational boundaries, requiring a more explicit and renewed take on defining responsibilities.
With AI based systems performing more autonomous actions, the responsibility model needs to clearly cover who is responsible for the agency given to these systems. Including the moral and legal responsibilities within and outside the use cases of the AI agents.

Priority

• P1: This is important to include in the next release from this workstream.

Level of Effort

Medium: This will take a week or two to document.

Drawbacks

Ownership claims - the exact reason for defining who is responsible for what, may arise.
Scope creep - Including too many types of AI vendors or regulatory domains
Cross-org alignment - Legal, sales, security, research may have different perspectives or understanding of new technology

Please consider:

• is it too opinionated?
• is it too complex to implement?
• does the ecosystem exist to support this yet?

Alternatives

Adopting the Azure AI Shared Responsibility model. This does not include industry or regional regulatory domains at the customer level. This was also written prior to AI Agents / Agentic system development.

Collaborating with the Cloud Security Alliance on their 2023 proposed AI Shared Responsibility Model

Reference Material & Prior Art

The Cloud Shared Responsibility Model (CSRM) defining customer vs. service provider responsibilities for Software as a Service, Platform as a Service, and Infrastructure as as Service is well established framework. Cloud Service Provider websites have lengthy descriptions of the CSRM in their own words.

• Is there an existing framework or paper that discusses this?
  Unravelling Responsibility for AI. Zoe Porter, Philippa Ryan, Phillip Morgan, Joanna Al-Qaddoumi, Bernard Twomey, Paul Noordhof, John McDermid, Ibrahim Habli

• Was this discussed in a talk that was recorded?
  No

Unresolved questions

• What help from the group do you need to make this successful?
  Active participation

Need input from a diverse set of roles:

• Security
• AI Practitioners/Researchers
• Application Developers
• Governance & Compliance members
• Government Relations members
• End-users, Customers
• Business Analysts (if business process automation via agents is considered)
• Regulatory domain input - To identify domain specific AI responsibilities owned by end-users and Customers (Healthcare, Financial, Manufacturing, Public Sector, etc.).

Endorsement by the group - Do you feel the pain of determining who is responsible for what in your company, and between your company and your customers.
Alignment to other frameworks or working groups (e.g.; AI Agent vs. AI system vs. model red team scoping)

[davidlabianca]

Thanks @imolloy, @billstout-snc, @wildelaw...

Seems like a clear problem that can be addressed in parallel to other WS3 (and other WS) efforts and would require some collaboration between WS4 and WS3 to properly land the Agentic model. Some questions, below, to facilitate the discussion, prioritization, and review...

How would you prefer to see this progress:

1. initial scoping paper on the need, current exemplars, paths forward
2. proposal for a shared AI responsibility model with Agents in the foreground
3. Outreach and collab on a "shared endstate" after item 2 is completed?

Or more towards a Step 2. as a starting point?

Also, given the Alternatives, you highlighted, did we have any members highlighting pre-existing collab with CSA on their model? Did we poll if any members have anything they wanted to contrib in this space?

[billstout-snc]

Agreed. Similar situation exists with AIA fa2, safety domains and related regulations (healthcare, finance, manufacturing, etc.). Customers generate 50 or more questions compiled from various sources, which gets distributed to SMEs who are not responsible for all questions which results in months of delay. Layers in the Azure model can be expanded to identify who is responsible for what, which also helps route questions properly.

• Safety domain responsibilities
• User responsibilities
• Customer Company responsibilities
• Service Provider responsibilities
• Application team responsibilities
• Platform team responsibilities
• Research/Foundational model responsibilities

Within companies there is friction on who owns what as teams claim too much or too little ownership, so an AI responsibility model will help all of us with a standard.

Often missing when automating business processes with agents is identifying the responsibility of business analysts, who can apply workflow security with proper checks and balances, and prevent issues such as business process compromise and loss of transactional integrity (rollback, replay, undo, corrections, etc.).

[billstout-snc]

Example chart of safety domains (proposed, not official)
The-AI-Alliance/ranking-safety-priorities#7

[anton73456]

A most excellent, and long overdue too

Happy to contribute, we had some early work on this a few years ago https://cloud.google.com/blog/transform/from-turnkey-to-custom-tailor-your-ai-risk-governance-to-help-build-confidence but this definitely needs a consistent effort

Anton

[iamwillbar]

I'm very interested in this and it's a project I'm actively working on.

We've been trying to find the right granularity of actors in the AI value chain so that nuances in responsibility can be captured but without having too many actors defined.

We're currently using:

• Pre-Deployment Actors
  • Training Data Providers
  • AI Model Developers
  • AI System Developers
• Deployment/Post-Deployment Actors
  • AI System Deployer
  • Users
    • Active Users
    • Passive Users
• Cross-Cutting Actors
  • AI Infrastructure Providers
  • AI Platform Providers

We also have a set of definitions and synonyms for (most) of these and are developing a mapping to specific definitions in standards/legislation.

We're also starting to look at existing taxonomies of responsibilities in standards/legislation and elsewhere.

A couple of resources to consider (disclosure: we contributed to both of these):

• https://partnershiponai.org/resource/risk-mitigation-strategies-for-the-open-foundation-model-value-chain/
• https://www.weforum.org/publications/navigating-the-ai-frontier-a-primer-on-the-evolution-and-impact-of-ai-agents/

Let me know how we can help out.

[billstout-snc]

@anton73456 @iamwillbar thank you for your wisdom and input, and recognizing the problem and need. @imolloy is organizing this effort with alliances who also see the need.

We should keep this simple enough to evangelize verbally, and with wide enough input to be accepted broadly. AI is an advanced technology and we can build a responsibility framework on the familiar cloud shared responsibility model. This makes this Most Advanced Yet Acceptable (MAYA - what makes things cool).

Both top layer and bottom layer already could also be extended horizontally. Safety domains on top, and development lifecycle (such as MLOps or CRISP-ML) on the bottom. It is tempting to extend the model to where it can't be evangelized simply (thinking 'agents'). Maybe there should be the basic framework and a detailed model. People can make careers of it.

[vishwasmanral]

@billstout-snc, @anton73456 , @iamwillbar - Great initiative. I wrote the Shared Responsibility model in 2023 with the Cloud Security Alliance https://cloudsecurityalliance.org/blog/2023/07/28/generative-ai-proposed-shared-responsibility-model . The landscape has changed a lot since:

1. Models now have layers of software built in with reasoning models
2. With Agents models are now getting more complicated
3. Newer use cases like code are making the shared responsibility model more interesting
4. Multimodal aspects of generated content makes for some interesting exercise

Happy to partner, review and help where needed.

[davidlabianca]

• @MSscotch

[wildelaw]

The current shared responsibility models have attempted to merge the current operating models with AI.

• Microsoft’s Artificial intelligence (AI) shared responsibility model includes the shared responsibilities for AI usage, AI applications, and AI Platform into IaaS, PaaS, and SaaS.

• Vishwas Manral (@vishwasmanral) Generative AI: Proposed Shared Responsibility Model introduced additional layer to overlay on to a shared responsibility model with the operating models of IaaS, PaaS, and SaaS.

• IBM Institute for Business Value Securing generative AI introduced additional layers to overlay on to a shared responsibility model with the operating models of IaaS, PaaS, and SaaS.

These model start to highlight some of the initial details around new concerns about the gaps in Cloud Security Shared Responsibility Models when working with AI systems. What these do not cover yet is an additional operating model where activities are being executed based on some level of agency given to an AI service. Should we still consider an agentic expert systems which can make and execute on decisions based on an arbitrary input as a Software as a Service (SaaS) operating model? Would it make sense to separate out an operating model or activity layer categories that are associated with what could be considered as a digital worker with agency to make decisions? Digital workers could be considered a SaaS but how would we change the shared responsibility model when adding probabilistic behavior introduced into expert systems?

Adding an additional operating/deployment model with AI service that has agency adds another level but may simplify the shared responsibility model ease of comprehension. Although, looking foreword there may be fewer cloud services at the PaaS and SaaS level that does not include built in digital worker with agency to make decisions. While not adding an additional digital worker as a service operating and deployment model may make it more difficult to create a generalized model with clear responsibilities of three basic entities of customer, cloud provider, and shared responsibility.

From an activity point of view, the additional of AI has added additional high-level actives as well as added some nuances to existing layers of the model. At a high-level the layers of activities for a cloud shared responsibility model includes application, operating system, network, and infrastructure. These are expanded by the different providers in include items such as configuration, identity access controls, and data. Overlaying AI and agency would introduce new responsibility concerns around usage, data, and applications. These concerns are more than just security, they should include all protection concerns around security, safety, and privacy.

@davidlabianca, based on the complexities of creating a simple yet compressive framework and @billstout-snc comments about the ability to break it down into domains or roles across many different layers, we should start with a paper. This would help solidify a framework to support the variety of needs along with a generalized set of responsibility models from the top level and how it could be tailored and made specific for each vendor.

[borisgvishnevsky-ai]

Oh' Well - Can I take a liberty ( as a human!) to be a little opinionated?:

• I would vote for us to NARROW the "shared" responsibility model for AI for many reasons:

  • AI is NOT like any other technology, platform, application, or algorithm that humans have experienced and used thus far, and
  • AI uses the most powerful API for humans - a multi-language-human API' interface, and
  • unlike many other business applications and platforms, AI is constantly evolving - never static, and
  • AI Business Solutions got to be 'owned', TQM-Tested, monitored, and operated by a minimal set (1-2(MAX!)) of Business Owners. We can discuss, but 2d, 3d, etc... party risks would not be helpful in AI world in my view ... ( because of the uniqueness of AI-Assisted, AI-Augmented, AI-infused business solutions, and
  • AI can autonomously 'reproduce' by autonomously building AI agents and cloning itself, ...and...

• I can see and agree that there may be two business owners: the business owner of the frontier ( foundational ) LLM, and the business owner of the adapted AI business solution (business-specific adaptions of the frontier LLM) unless this is the single business owner who operates both the frontier (foundational) LLM and business(domain) specific adaptations.

• It is an interesting case and discussion of how to handle ChatGPT-like AI Adaptations marketplace (AI App Store)?
  I suggest that it falls under the two-business-owner model above.

What do you think??
Best,
Boris.

[victorjunlu]

I’m currently preparing for my KubeCon Atlanta talk, which will include an introduction to COsai. I’d like to briefly touch on both the governance model and the shared responsibility model. Since the AI governance model has already been under discussion within workstream 3 for many months, would it be possible to separate it out and get approval to publish the first version even if it is version 0.1 ?