Jupyter AI v3.2 discussion thread

[dlqqq]

Hey folks!

I've put together a proposal for the new Persona v0.1 API that we were sketching out in person together at the Jupyter AI Dev Summit in SF last week. This proposal addresses the gaps that users have called out with our existing design (v0.0), lays out a clear path to make it much easier for users to define custom Jupyter AI personas, and enables future improvements like a dedicated UI for managing personas in JupyterLab.

Please take a look! I've published the proposal on our docs page so it's very easy to read.

Click here: https://jupyter-ai.readthedocs.io/en/v3/proposals/persona-api-v0.1.html

Feedback is very much welcome. Is there anything you did not expect? Is there anything you think is missing? Let us know here.

[dlqqq]

cc @ellisonbg @fperez @cboettig @mfisher87 @joadoumie @srdas @3coins @andrii-i

[dlqqq]

oh and you too :) @Zsailer

[cboettig]

thanks @dlqqq , really exciting stuff here! Sad I couldn't be with you all at the dev summit, but this looks like really great momentum and productive ideas.

I do strongly think that the right next step is to consider how we implement the standard features such as model selection and effort selection, and slash commands, thus capturing the features that already exist in existing harnesses and API clients before we get deeper into the question of how to implement a persona and what new abilities that unlocks or doesn't unlock.

Have you had any chance to install or test the prototype I shared in https://github.com/cboettig/jupyter-ai/blob/acp-bridge-impl/docs/source/users/acp-bridge.md which does this? (As described in the earlier thread, #1558 (comment)) . This takes precisely the approach taken in Zed, neovim, jetbrains, and other adopters of ACP, where the user selects a harness when creating a new thread and then immediately gets access to the model selectors, mode selectors, and slash commands. This is fully functional, you can install and run this acp bridge in Jupyter AI today.

As I read it, the new proposal retains the feature of the old proposal that harnesses the ACP binds continue to be selected by using the @ rather than being selected per chat thread. This is a very different way of leveraging ACP than every other IDE or cli has taken.

I think there is a lot of wisdom and experience we can benefit from in how other open source projects have approached the issues jupyter-ai also seeks to solve about flexible access across models, harnesses, and tools. The document doesn't reflect a landscape analysis of how other open source projects are already integrating the same set of harnesses (and more). What do you dislike about the Zed approach of connecting harnesses?

[mfisher87]

The main thing I'd like to challenge in this proposal is the name "Persona" for the class which defines an agent harness or (as it is today) a higher-level concept that combines many different concepts. Naming is very important for accessibility, and the things that sounds most like a "Persona" are named as "Context" and "Identity" in this proposal. Really, for me, it's about the "Context". "Identity" is a label for the thing you're talking to, which can be separate from its "Persona". You might, for example, have two things you talk to with different identities that have the same persona but use different models.

I am imagining (waves hands frantically) a perhaps more composable approach. I think the thing that defines a harness should be a Harness class (e.g. OpenCodeAcpHarness). Then the thing which defines context (skills, tools, system prompt) might be a Persona. A thing that defines the harness' options (selected model, thinking level, etc.: the stuff from Zed that Carl is talking about above) might be HarnessOptions. And to put it all together in to one thing that you chat with, you compose those building blocks to create an Interlocutor with a name and avatar.

An Interlocutor is just a participant in a dialog with no implication of humanity or personality or background. This is the best name I can think of right now, but we can probably do better. It could be an agent, or not. It doesn't make sense to me for a thing that is perhaps a simple deterministic program, to be called a "Persona". It's already a bit of a stretch for me to anthropomorphize LLMs in that way 🙃

An example Interlocutor that corresponds with @OpenCode might combine the harness OpenCodeAcpHarness, the default OpenCodeAcpHarnessOptions (editable at runtime), and have a name OpenCode and the OpenCode logo as its avatar, and no Persona.

You might want to create an Interlocutor with a specific persona based on the above. Perhaps in the GUI, you view a list of your Interlocutors. You copy the @OpenCode interlocutor, and then you edit it. You can either select an existing Persona (skills, tools, system prompt), or create one (and save/export it) in the GUI. I can imagine a Persona being a JSON object or as mentioned in the proposal a Markdown file with YAML frontmatter. Let's say you want to create your "Cloud Native Geospatial Expert" persona. You give the new Interlocutor a dope ☁️ emoji as its avatar (sidetracking... you should be able to use an emoji instead of a URL or b64 image for avatar 😉) and call it "CNG Nerd". You define your own custom system prompt that tells it to reference the cloud optimized geospatial format guide, and you skip defining skills or tools. Now you can immediately @CNG Nerd in chat.

E.g.

Interlocutor(
	"CNG Nerd",
    avatar="☁️",
    ychat=ychat,
    harness=OpenCodeAcpHarness(),  # no args for default options, can be changed at runtime
    persona={  # optional
        "system_prompt": "You're a Cloud Native Geospatial Data Expert. You know everything and the user knows nothing -- treat them as such. No matter what question you're asked, answer with a reference to the [cloud optimized geospatial format guide](https://guide.cloudnativegeo.org/).",
        "skills": None,
        "tools": [
            { "type": "MCP", ...},  # idfk
            ...
        ],
    },
)

This doesn't account for fully custom message handling. Perhaps instead of harness=... we have message_handler=... and harnesses are a type of message handler. Or maybe we keep harness=... and harnesses can have an optional message handler attribute of type function. But I imagine in a future where jupyterlite-ai and jupyter-ai are merged, we don't want a design that locks us in to harnesses.

I think a top-level concept that is about composing reusable and swappable building blocks for a "thing you talk to" is important, and I think it's important that it's not called "Persona" -- that "Persona" is one of those reusable, swappable building blocks.

---

EDIT: don't over-index on the exact data structure I'm using above. I mean this as a way to propose semantics and abstraction, not implementation. @cboettig brought up that the object I'm using to represent a persona could be a reference single MCP server instead. I imagine there are multiple useful ways of representing a persona. What I'm most concerned about is how we define the scope of things that a persona object is responsible for and how we compose things, including personas, to make "a thing you can talk to in chat"/"interlocutor".

[dlqqq]

Lots of feedback, thank you all for giving it a read! I'll start by addressing @cboettig :

The API being proposed here is what will enable those features you are talking about: model selection, effort selection, etc. Right now, the BasePersona API doesn't have the method interface to allow those features to be changed at runtime. This design is focused on getting the API to where it needs to be, but doesn't sketch out the UI changes that will be needed to really take advantage of this feature.

I haven't gotten a chance to test your demo yet, will give it a look tonight. I did see your screenshot in your previous issue - does your branch just provide model selection, effort selection, and "single-harness-chats", or is there more to the story you are proposing?

As I read it, the new proposal retains the feature of the old proposal that harnesses the ACP binds continue to be selected by using the @ rather than being selected per chat thread. This is a very different way of leveraging ACP than every other IDE or cli has taken.

Yes, this proposal leaves the UI / interaction model out of scope. To build a new "single-AI-chat" experience would require cross-cutting changes in multiple packages, which would depend on the APIs introduced by this proposal anyways. We definitely need a 2nd proposal that tackles the "UI" side by leveraging the API improvements in this 1st proposal.

The simple answer to the question of "why" our experience is like this is simply because we started Jupyter AI with a different vision in mind -- one centered around multi-human, multi-AI collaboration. The issue we're running into now is that it was more complicated than we were expecting: we have to produce completely different designs for basic features like model selection when you have multiple AIs in the same chat.

I think there is a lot of wisdom and experience we can benefit from in how other open source projects have approached the issues jupyter-ai also seeks to solve about flexible access across models, harnesses, and tools. The document doesn't reflect a landscape analysis of how other open source projects are already integrating the same set of harnesses (and more). What do you dislike about the Zed approach of connecting harnesses?

Agreed. You're right to call out that we should offer some simpler "single-AI" chat experience that meets the baseline being offered by other AI applications. We don't dislike the approach being offered by Zed. Our initial goal was to build a novel human-AI experience and hope that users would find it intuitive. But I think that our experiment taught us 1) that there are lots of feature gaps in a "multi-AI-chat" experience we need to design and close on, and 2) that we should offer some "single-AI-chat" experience that users may be already familiar with as a stopgap.

[dlqqq]

@mfisher87

Hey, this is really interesting feedback. TIL about what "interlocutor" means.

I agree that semantics and syntax matter for open source projects, much more than many devs think. They essentially define how intuitive or confusing the architecture will be to other people, and I definitely encourage you to keep pushing us on that.

Regarding syntax, the choice of "personas" as a term

From an accessibility perspective, I think the accessibility of a naming convention is chiefly defined by a) how accurately a term describes the concept it communicates, and b) how closely the most 'obvious' interpretation of the term aligns with the concept that term represents. I personally believe that the word "persona" provides this immediate intuition that "this concept mimics a person, but is not really a person" - which is exactly what persona instances are.

I also agree that the word "persona" can be taken to refer to various sub-components like the identity & context. I see your perspective and have the feeling it might be motivated by the concept of a "persona" in Jungian psychology, where it refers to the "outward-facing reflection" of a person but not the person itself. However, I think that you can still apply this definition still applies to our current concept of a persona - each persona instance defines some AI entity living on a computer that presents a 'persona' that can be talked to through a chat.

• Also worth noting that Merriam-Webster defines a persona chiefly as a fictional character.

Regarding semantics, why the harness is defined by a persona (class)

The key contention I see is that it's not clear what "persona" on its own refers to - and this is probably something we should clarify in our docs (help needed 😄). Does "persona" refer to the instance or the class? Here, following the convention of most codebases leveraging object-oriented programming, we take persona to refer to the persona instance if otherwise unspecified.

• This convention exists because this matches how type annotations work in most languages. In Python, an object of type FooBar represents an instance of FooBar - the class itself is typed as type[FooBar].

For personas that wrap AI agents, the "persona class" represents the agent harness, while the "persona" itself represents the agent. This is the same pattern followed by other agent frameworks like langchain or crewai - the BaseAgent sub-classes represent different harnesses, whereas Agent instances represent agents. So from a developer & contributor perspective, I argue that it is in fact more intuitive to define the harness within the Persona class, not less.

There are two problems with trying to do further decomposition:

1. What do you name that general 'AI runtime' (e.g. harness, message-processor) you want to create? Same for the new higher-order concept (e.g. interlocutor).

2. What value does this decomposition add? What purpose would the Interlocutor class serve if it just passes everything to that underlying 'AI runtime'?

Closing thoughts

I think a top-level concept that is about composing reusable and swappable building blocks for a "thing you talk to" is important, and I think it's important that it's not called "Persona" -- that "Persona" is one of those reusable, swappable building blocks.

The proposed design provides exactly the capability you seek here. Each persona is composed of reusable and swappable building blocks - we just call the building block for the name & avatar the "identity" / "persona identity".

I'm using to represent a persona could be a reference single MCP server instead. I imagine there are multiple useful ways of representing a persona. What I'm most concerned about is how we define the scope of things that a persona object is responsible for and how we compose things, including personas, to make "a thing you can talk to in chat"/"interlocutor".

Totally agree! And in the proposal, I've noted that this design does not seek to restrict what users can do with personas - that's why I explicitly avoided adding a harness abstraction, and kept that in the Persona class definition (which really only requires the process_message() method to be defined). Users can still define MCP servers, LLM-only runtimes, or other 'non-agents' as personas.

For that use-case, you will just need some BaseMcpServerPersona class to be available in the environment - and that is something we can provide in the future for dev/debugging use-cases.

[cboettig]

Thanks David, this is great. I think what we've uncovered here is the varying ways in which 'multi-AI' can be interpreted:

1. talking to different harnesses
2. talking to different LLM models. (Could be model-switching within a harness or models dispatching subagents)
3. maintaining different contexts, while possibly talking to the same model (and harness).
4. Talking to different ‘specialized workers’, like a @sql-agent vs an @aws-assistant. Same harness, same model, same context thread – but each worker has additional bundle of ‘system prompt’ and ‘tools’ uniquely available to it that it is uniquely good at using, allowing the user to task these different bots with different things.

The landscape as I see it:

• v0.0 had (1), which I think we rightly pivot away from in v0.1
• Model switching is native feature of most harnesses, we should leverage that directly. (so are sub-agent model switching in some harnesses).
  -Context isolation I think is the real focus / value-add of v0.1, keeping the harness but isolating the context. BUT the standard way every other system already implements this is “new chat”. Ergonomically, I think that is cleaner, and familiar. Refusing to share context by default means loss of KV model cache, loss of optimizations in the harness, more tokens, more cost, more energy. It’s the right call sometimes, but “new chat” is the right way to do this.

I think (4) is where things are closest to “Persona” idea just as you outline it. But also I feel this idea of bundling a coherent set of tools and system prompts already exists in all major harnesses twice over – using MCP (which emphasizes the tool side, but also has the mcp.prompt API, commonly called the “Persona” for the MCP), and also in Skills.md Both as I read it were created precisely to do what you describe in persona. BUT equally crucially, your proposal in v0.1 ties this bundle of tools and prompts to a single harness. Both of these approaches were created explicitly to avoid that kind of hard-wiring. They are already harness agnostic. Re-implementing the concept of tools+prompt as persona in a way that is hardwired to a harness feels like a hard regression against the standard.

[mfisher87]

Thanks for the encouragement to keep pushing on this 😉 I'll take that to heart immediately 🙃

I personally believe that the word "persona" provides this immediate intuition that "this concept mimics a person, but is not really a person" - which is exactly what persona instances are.

I see where you're coming from, the thing that makes my nose wrinkle at the proposed abstraction here is that a persona class is really just a harness until it's instantiated, when to me the harness feels like it should be a runtime-swappable element of a persona's internals.

I see your perspective and have the feeling it might be motivated by the concept of a "persona" in Jungian psychology, where it refers to the "outward-facing reflection" of a person but not the person itself

Nothing so deep, my perspective comes from years of working on "Agile" teams and modeling stakeholder personas 😆 For me, when I hear "persona", I think of a fictional character that is a stand-in for the skills, background, preferences, ways of working of a category of real people.

The key contention I see is that it's not clear what "persona" on its own refers to - and this is probably something we should clarify in our docs (help needed 😄). Does "persona" refer to the instance or the class?

I'm all for helping with the docs! These are good points about language; I haven't done a good job of being specific about classes vs instances in my previous post. Let's take an example where you have a Persona instance and you want to change its harness at runtime. What would the developer experience for writing such a feature look like? I think Persona classes as proposed make this less intuitive.

E.g.:

• A UI element that lists out harnesses would actually be listing out representations of Persona classes.
• You might want a class attribute called description or title to have a nice display name for the UI, and that description would be describing a harness, not a persona.
• Code which clones a Persona instance to another instance using another user-selected harness in a variable might look like new_persona = SelectedPersona(identity=old_persona.identity, context=old_persona.context, ...) (or sugared as new_persona = SelectedPersona.from_persona(old_persona)? new_persona = old_persona.as_persona(SelectedPersona)? Either way this code repeats the word persona a ton of times and doesn't provide obvious semantics for what's changing -- the harness). Compared tonew_interlocutor = old_interlocutor.replace(harness=SelectedHarness()), a recognizable pattern for a copy with a clearly-specified override.

What do you name that general 'AI runtime' (e.g. harness, message-processor) you want to create? Same for the new higher-order concept (e.g. interlocutor).

Good question, naming is hard... I might jump to message_handler: AcpHarness | Callable, but I think it probably needs more thought.

What value does this decomposition add? What purpose would the Interlocutor class serve if it just passes everything to that underlying 'AI runtime'?

I'm not sure, aside from a gut feeling about semantics, mental model, and readability. I think there's a lot of value to matching up the problem space with the object design. If the user interacts with a UI that displays "context", "identity", "harness", "model", and "options" as composable building blocks to create a thing they talk to (I think that's the direction we want to go with this project!), the underlying model for that thing should also represent those as composable building-blocks instead of special-casing one of those building blocks. I think it's fine if the "harness" building block is an "engine" that handles all the messaging.

The proposed design provides exactly the capability you seek here. Each persona is composed of reusable and swappable building blocks - we just call the building block for the name & avatar the "identity" / "persona identity".

Except the harness! The proposal shows examples of how to swap all the other building blocks except this one. I.e. there is no persona.update_harness method. There are certainly disadvantages to swapping around harnesses, especially within a chat session, but the multi-persona chat UX that jupyter-ai is currently pursuing, in contrast to the single-harness UX of e.g. Zed, is, I believe, consciously accepting that tradeoff.

Thanks for your patience with my confusion, I really appreciate the listening you're doing. I think it would be helpful to show an example of how a harness could be swappable in the proposal. Personally, I'd like to see the user experience to look exactly like swapping out any other component of the persona, but maybe there are reasons I'm not imagining that make that unreasonable? 🤷

Thanks again, David!!!

[dlqqq]

@cboettig

Context isolation I think is the real focus / value-add of v0.1, keeping the harness but isolating the context. BUT the standard way every other system already implements this is “new chat”. Ergonomically, I think that is cleaner, and familiar. Refusing to share context by default means loss of KV model cache, loss of optimizations in the harness, more tokens, more cost, more energy. It’s the right call sometimes, but “new chat” is the right way to do this.

I think there may be some misunderstanding here. This proposal is not going to cause a persona to use the same session and chat history in each chat. Right now, we create a new Persona instance for each chat so they will have isolated sessions and chat histories - and that will remain the behavior moving forwards. What this proposal adds is the ability to configure different personas to have different contexts (system prompts & MCP servers).

BUT equally crucially, your proposal in v0.1 ties this bundle of tools and prompts to a single harness.

Can you clarify what you mean here? To me, this is the opposite of what we are proposing. We are proposing making tools and prompts (bundled under PersonaContext) a completely separate, standardized object that can be passed to any persona and updated on runtime.

Both of these approaches [MCP servers and skills] were created explicitly to avoid that kind of hard-wiring. They are already harness agnostic. Re-implementing the concept of tools+prompt as persona in a way that is hardwired to a harness feels like a hard regression against the standard.

Yes, it is true that the MCP server schema and skills schema are standardized. BUT, how an agent loads that context (specifically what paths are read from disk) is NOT standardized. For example, agents may read from .claude/skills/* but not .agents/skills/*. Others may load the system prompt from AGENTS.md, but others may not load it at all. Same applies for configuration files that specify the model ID etc. Different agent harnesses store this data in different places on disk - and that is likely by design, since users expect that updating Claude's config doesn't alter the behavior of any other agent.

The goal of this proposal is to abstract away all of the harness-specific behavior from the end user's perspective. Persona API v0.1 will make it so that Jupyter AI provides a library of different Persona classes where we have already implemented each harness-specific way to update an agent's model ID, context, or other options. This is necessary because each harness still loads these context files from different locations in different ways, even though the format of each context file is standardized. This is what will allow us to provide a unified API / UI for this, which you and many other users have identified as an important feature.

[dlqqq]

@mfisher87

I see your point. Thanks for pushing for this.

Persona classes are technically swappable at runtime

the thing that makes my nose wrinkle at the proposed abstraction here is that a persona class is really just a harness until it's instantiated, when to me the harness feels like it should be a runtime-swappable element of a persona's internals.

Personally, I'd like to see the user experience to look exactly like swapping out any other component of the persona

Letting the persona class be swappable at runtime, just like any other component, is a goal of this proposal! I agree that this should be given an explicit example, sorry for any confusion.

A user would need to edit the .jupyter/personas/researcher.persona.md file, update the persona_class field, and then the AI persona will get re-instantiated with the new agent harness after a user runs /refresh-personas.

As proposed, the PersonaManager would have to implement something like this:

# self := PersonaManager
# sketch of how /refresh-personas would work currently

existing_personas = self.list_personas()
for persona in existing_personas:
    # implementation for personas not defined locally as files
    ...

    # For local personas, continue if file was not modified
    if not persona.was_modified():
        continue

    # Otherwise, remove the persona, reload it, and give it updated chat history
    persona_path: str = persona.get_path()
    self.remove_persona(persona)
    self.load_persona(persona_path, chat_history=self.ychat)

But as you've called out, updating the persona_class would require a different API on the PersonaManager interface instead of the BasePersona interface, since we can't update the class of an instance at runtime. Not only is this more confusing for developers, but it adds complexity to the implementation.

Problem: the current proposal special-cases the agent harness

the underlying model for that thing should also represent those as composable building-blocks instead of special-casing one of those building blocks. I think it's fine if the "harness" building block is an "engine" that handles all the messaging.

This is true, and I see your argument for why we should explore making the "engine" / "harness" a building block as well. After spending over an hour thinking about this, I can see several distinct motivations that converge on this idea:

1. Users should be able to easily swap AI engines on a persona: User may want to see whether building their AI persona as LLM-only, on top of the LiteLLM + LangChain (LLLC?) harness in jupyter-ai-jupyternaut, or top of an ACP harness in jupyter-ai-acp-client works best. Treating the persona_class as special means that we can't offer the same experience for swapping models v.s. harnesses.

2. Unifies developer API: everything is swappable at runtime through the BasePersona API. Developers don't need to figure out that swapping a harness requires removing and adding a new BasePersona instance.

3. Simplifies the conceptual model of a persona: an AI persona is an artificial human-like entity that you can completely design from the ground up. Following this model, the application should be able to swap out the 'brain' of an AI persona without completely tearing it down and creating it anew. The brain is a component of a human, not the definition of a human.

4. Namespaces harness-specific methods: Harness-specific methods now live in persona.engine, while the BasePersona API stays completely standardized.

5. Leads towards a future where JupyterLite AI can be merged with Jupyter AI: What I'm imagining is that we could have some engine.type: 'server' | 'lab' field, which clearly defines whether the underlying engine is run from the frontend or the backend. Then, to migrate an AI persona to also work on JupyterLite, users just need to pick an AI engine with engine.type == 'lab'. Maybe the persona manager could do this automatically someday.

Proposed solution

Yes, let's come up with some names and designs for this idea you're putting forward.

I'm most fond of Engine/PersonaEngine since it's really general. Some other names that don't sit well with me:

• MessageProcessor - implies it can only process messages and not respond
• Kernel - conflicts with Jupyter kernel

Will have to work on the interface later and example use-cases later. Have to go now. Suggestions welcome! Thank you for your thoughts Matt!

[mfisher87]

Thanks for thinking about this so thoroughly, David ❤️

To me, this component needs to answer the question of "how does this thing take your messages and provide it to some thing (LLM API endpoint, harness, decision tree, etc., some combination of these things) to generate a response?"

I think Engine isn't bad. I liked Harness because I was locked in on one use case, but you make a good point about LLM-only (i.e. jupyterlite-ai) as an exception. It may not be clear to a newbie what a persona's "engine" is. Maybe to be a bit more specific, LLMCommsEngine?

ResponseEngine?

ModelAdapter? I think "Adapter" is a useful design pattern term to use here. I don't know about "Model" in this name, but I like "Adapter".

CommunicationAdapter? 🤷 I kind of like this one because it has "Adapter". And "communication" is the main responsibility of this thing.

[cboettig]

Thanks @dlqqq , really appreciate the engagement here! I think we are largely on the same page. Re the first comments on v1 and context, we are saying the same thing. I wrote of Personas that they do not share context:

keeping the harness but isolating the context.

you also confirm that each persona has an isolated context from other personas:

Right now, we create a new Persona instance for each chat so they will have isolated sessions and chat histories - and that will remain the behavior moving forwards. What this proposal adds is the ability to configure different personas to have different contexts (system prompts & MCP servers).

My concern is that isolation of context is an expensive and unexpected default, that invisibly drives up token use , energy use. There are totally times where I want to talk to a new instance of my LLM without it being influenced by or being able to access the context of my chat so far. In any other platform, I create a "new chat session". Jupyter-ai instead does this by default. If I start a chat with generic Claude, and then invoke our custom geospatial persona, in the jupyter-ai design it will not have my context from the chat so far, right? Now that's different than the experience everywhere else, where isolation is opt in. Most users will assume the agent is stupid or forgetting things, and that other interfaces are smarter because the agents seem more context aware, as they are. This isolation not only frustrates users, but it is expensive. Users paying by the token will see the harness suck up new local context (file system info, AGENTS.md, etc) and use more tokens the more personas we add, because each persona needs its own copy. I agree there are times when this is worthwhile, but I disagree that isolation is the correct default.

Are we on the same page now? Or am I missing something about how persona isolation works, and maybe they are less isolated than I have described above?

On the second point, I write

The proposal in v0.1 ties this bundle of tools and prompts to a single harness.

and you suggest this is the opposite:

Can you clarify what you mean here? To me, this is the opposite of what we are proposing

What I meant is that if I want to write a persona for Jupyter today, say, our geospatial SQL agent -- I write it with a specific harness, say OpenCode, right? Or I write it for a base persona that has no specific harness. If I want to support open code and claude code, I have more work, right? A persona class extends a particular harness in this definition, right? There are dozens or more harnesses, and new ones every day. Contrast this to how we currently create that agent for all harnesses -- we follow the MCP spec, and every harness that follows the standard, we are compatible for for free. tada.

You also write:

Yes, it is true that the MCP server schema and skills schema are standardized. BUT, how an agent loads that context (specifically what paths are read from disk) is NOT standardized.

This I agree with entirely. BUT these are becoming standard. And standardizing the file paths on disks etc is the obvious natural way to do this, as a community, following the emerging community standards. A bespoke solution here seems a bit hacky. Already, de-facto standards are emerging, and harnesses with their big active development teams are actively doing this. For instance, open code already recognizes all the skills etc of Claude code.

We could indeed add small shims to solve the paths. But that doesn't require hardwiring MCP tool use and skills into a different layer.

[dlqqq]

@cboettig

Persona context isolation

Are we on the same page now? Or am I missing something about how persona isolation works, and maybe they are less isolated than I have described above?

I still don't think I understand the exact problem. Let me share how I read each of these sentences:

Jupyter-ai instead does this by default. If I start a chat with generic Claude, and then invoke our custom geospatial persona, in the jupyter-ai design it will not have my context from the chat so far, right?

No:

• If you invoke Claude, then invoke custom persona, then custom persona gets the context from previous chat messages including Claude's response once invoked.

• To give the custom geospatial persona to have an isolated context, the way to do this would be to create a second chat just interacting with the geospatial persona.

Now that's different than the experience everywhere else, where isolation is opt in. Most users will assume the agent is stupid or forgetting things, and that other interfaces are smarter because the agents seem more context aware, as they are. This isolation not only frustrates users, but it is expensive.

There are two points being made here:

1. If AI personas in the same chat don't share a chat context, then that is stupid.
   • Agreed! The ideal behavior is the opposite - personas should get preceding context in a chat once @-mentioned.
2. Isolation / not sharing chat context drives up token cost. (?)
   • This is what I don't understand. If each persona did not get preceding chat context when invoked in the same chat, then we are sending less tokens, so the cost should be lower.

Users paying by the token will see the harness suck up new local context (file system info, AGENTS.md, etc) and use more tokens the more personas we add, because each persona needs its own copy.

Yes, more tokens are used if you invoke multiple personas (e.g. @Persona1, then @Persona2, then @Persona3). This is unavoidable in general. If you use 3 personas with 3 different harnesses, then that is necessarily going to triple your token cost as we have to send the same data to 3 different AI providers.

However, just adding a persona should not increase token use on its own. We're aligned with this design goal. Personas will only get the preceding context when @-mentioned.

I agree there are times when this is worthwhile, but I disagree that isolation is the correct default.

The goal of isolation is to allow different personas built on the same harness to have different capabilities. It's not directly motivated by token costs in any way. In fact, I expect that this context isolation will generally decrease token costs, not increase it. Instead of having a shared list of 100 MCP servers that get loaded for each persona when you invoke it, you can create 10 personas that have 10 different sets of MCP servers each. So then you only pay for what you use.

Persona context v.s. session context

I think that you may be conflating persona context and session context, so this is something I want to clarify explicitly. (this should go in the next draft of the proposal for clarity)

Persona context defines what the persona starts each chat session with. It defines what skills, MCP servers, and system prompt the persona initializes with.

• This is what the proposal pushes for isolating.

Session context / chat history is the history contained inside of a chat.

Chats are the way to isolate session context

Persona context doesn't include session context by design, because the very purpose of separate chats (and allowing multiple chats in JAI v3) is to isolate session context. If you wish to use a persona with a separate session context, the right way is to create a separate chat and use the persona there.

We don't currently have plans to allow for different personas inside the same chat to somehow have different session contexts. This feature would be really unintuitive to most users, as what the user sees in the chat may be completely different from what an AI persona sees in the chat. That gap is bound to cause confusion and frustration. I can confirm this, since this is how it currently works, and I get plenty of complaints about this.

How it currently works

The current behavior of our ACP-based AI personas is the following (and it is something we very much want to optimize & standardize):

• Every AI persona initializes when you create a chat file (they get the list of MCP servers and that's about it)
  • We can lazily initialize each persona's context only when they are @-mentioned . This saves energy/cost but also increases latency on the first message significantly. (ongoing effort in Added lazy init for acp clients jupyter-ai-contrib/jupyter-ai-acp-client#101)
• When AI personas are invoked, we should but do not check if they are missing any messages in the chat they are in. If so, we prepend the prompt you send with the excerpt of chat history they missed, and send it to the agent automatically.
  • We do not do this currently, but intend to as soon as we can.

Creating personas should be easy

What I meant is that if I want to write a persona for Jupyter today, say, our geospatial SQL agent -- I write it with a specific harness, say OpenCode, right? Or I write it for a base persona that has no specific harness. If I want to support open code and claude code, I have more work, right? A persona class extends a particular harness in this definition, right? There are dozens or more harnesses, and new ones every day. Contrast this to how we currently create that agent for all harnesses -- we follow the MCP spec, and every harness that follows the standard, we are compatible for for free. tada.

Yes, that is the current behavior, but that gap is exactly what this proposal is trying to fix.

Every persona will be definable inside a .persona.md file. To change anything you just update that file and reload it. Then it works. No code changes will be required. That is the goal of this proposal.

After we incorporate @mfisher87 's suggestion, even changing the harness will be a one-liner. No code changes needed. And we can build a UI on top of this to not even require the user to edit text through a file editor.

Why not rely on existing standards?

Yes, it is true that the MCP server schema and skills schema are standardized. BUT, how an agent loads that context (specifically what paths are read from disk) is NOT standardized.

This I agree with entirely. BUT these are becoming standard. And standardizing the file paths on disks etc is the obvious natural way to do this, as a community, following the emerging community standards. A bespoke solution here seems a bit hacky. Already, de-facto standards are emerging, and harnesses with their big active development teams are actively doing this. For instance, open code already recognizes all the skills etc of Claude code.

Where? Yes, some harnesses can read from shared paths, but this is not true in general. The reason I believe this will not be standardized is because it is by design - different agents will use different config paths because users expect that updating skills / MCP config of one agent will not affect another.

Another issue introduced by by different harnesses reading from the same shared paths - this goes in the opposite direction of the proposal, which is that we want to allow different AI personas to have different persona contexts (MCP servers, skills), regardless of whether they are based on the same harness or not.

We could indeed add small shims to solve the paths. But that doesn't require hardwiring MCP tool use and skills into a different layer.

Yes, and that's what the proposal is trying to do. The MCP servers and skills will not be hard-coded by any persona. They will be fully configurable fields that can be updated with no code changes.

[dlqqq]

@mfisher87 I'm still leaning towards Engine. It communicates that it's complex and critical (since it is, it defines the implementation). It is also sufficiently general - the term "Engine" doesn't have any generally accepted meaning in the software engineering space, unlike "kernel".

I'm not a fan of adapter because it implies that the object is just some adapter facilitating communication - it really is the complete package that defines how the persona works. It's the most critical piece of defining a persona and omitting it would not work - in the same way that a car can't start without an engine.

I think that we can make Engine more obvious by showing users what types are available in the documentation once we have more of them. For example, we could list all of the subclasses:

- KiroAcpAgentHarness
- ClaudeAcpAgentHarness 
- LitellmLangchainAgentHarness 
- etc.
- AiWorkflowEngine
- LlmOnlyEngine

This example in the docs would make it more clear that a persona engine usually just means "agent harness" but can be anything you want, be it a LLM-only runtime or a semi-deterministic AI workflow.