be able to call a persona from a cell magic

[joequant]

Problem

Would like to do

%%ai @persona

This will allow the user to push cell contents to different personas

Proposed Solution

I am trying to modify jupyter_magic_commands so that it gets a persona from the persona manager and then calls the persona. The problem is that the personas are stored in the server level and the magics are processed at the kernel level

Additional context

Take a look at

https://github.com/hkphysics/jupyter-ai-magic-commands/tree/dev/persona

[joequant]

So the basic architectural problem is that the cell magic is happening in the kernel, whereas all of the jupyter ai chat services are happening at the server level. The current cell magics calls the model via the litellm interface which bypasses all of the persona functionality.

Would like some feedback on the best thing to do. I am thinking that rather than pull in the persona manager code into the cell magic, that I put in calls in persona manager to allow a REST call with a chat message. This would keep a nice separation between the kernel and server, but would some additional coding in the persona manager. Any thoughts?

[srdas]

@joequant This is a nice idea, i.e., to enable each cell to use a different persona via %%ai magics. The interface you suggest (%%ai @persona) is quite simple and straightforward. You are likely aware that the persona API is under redesign, see https://link.springer.com/chapter/10.1057/9780230524545_6, and #1571. So sequencing your proposed contribution may be worth thinking through. You may want to attend the Jupyter-AI community call every week at 8:30 am Pacific time, via this zoom link: https://zoom.us/j/95228013874?pwd=Ep7HIk8t9JP6VToxt1Wj4P7K5PshC0.1 - persona design and related topics have bee under discussion recently.

[joequant]

@srdas Yeah thanks. One of the things that I am thinking through is how to do the design so maximize code reuse. What I think makes sense right now is to include a REST API to send a single message to the persona manager.

[srdas]

I think that makes sense, any magic command that include @persona has to check that the persona exists and then can issue a call to it that mimics what is offered in chat. It's a neat feature especially since it involves the notebook directly and is more Jupyter specific. It should maximize code reuse if the new API redesign mentioned above breaks anything in whatever you build for %%ai @persona.

[joequant]

I've implemented the procedure in jupyter-ai-contrib/jupyter-ai-magic-commands#9 the way I did this was to create a handler in jupyter-ai-magic-commands and then call it from the magic. One thing that I am interested in looking at are better ways of doing streaming since I would like to have the handler stream the response. But enjoy!!!!

Also this fix should be robust against code redesigns, since you can redo how personas work without messing with the REST.

[dlqqq]

The tricky thing is that Python kernels run in a separate process from the Jupyter server -- even if we providee a REST API for invoking personas, the kernel process doesn't have a way of invoking that REST API because it doesn't have the token credentials that are required by default.

That being said, we haven't looked into this issue super deeply, and it's worth revisiting to see if this is possible. Just wanted to chime in that this may be more technically complex than one would imagine just given the architecture of JupyterLab itself.

[dlqqq]

The easiest way is probably to have the kernel run the agent directly in the kernel somehow. This would limit what it can do (no access to JupyterLab environment, no way to integrate with other chats), but perhaps this meets your expectation of running a persona inside the notebook? Curiously what you want to do with this 'persona cell magic' idea.

[dlqqq]

Happy to work with you on the design once you're able to share details about what you want a persona inside a cell magic should do -- that will essentially define the technical design, so IMO we should start by working backwards.

[joequant]

I got it working. Take a look at my PR's. The thing that I want to do is to just be able use the persona functionalities from the cells. But I have two pull requests pending.

jupyter-ai-contrib/jupyter-ai-persona-manager#52

jupyter-ai-contrib/jupyter-ai-magic-commands#9

The use case is so that I can use the persona functionality to create custom agents and have them both work with chat and for in cell creation

[joequant]

Also I looked at having the agent run directly in the kernel and it was a code nightmare. Creating a handler in the persona manager and then calling it from the kernel was much cleaner. Getting the token credentials turns out to be easy (see the code for how I am doing it) since the kernel needs the authentication to talk to the server process anyway.

[joequant]

Also I have a blog to give more context on what I am doing

https://hkphysics.substack.com/

I have a series of changes that will allow the user to do

%%ai @Jupyternaut -m '{"model_id": "ollama_chat/glm-4.7-flash:latest", "model_args": {"memory_store":"false", "api_base": "http://ollama:11434"}}

[srdas]

As noted in your other PRs, the new code works for ACP-based personas, but not Jupyternaut: