agent-service-toolkit: Easily run your agent on a FastAPI service (template repo)

[JoshuaC215]

For self host or other use cases where LangGraph Cloud might not be a good fit, I found it was still somewhat painful to stand up a running service on a LangGraph agent. I built a template repo agent-service-toolkit to help alleviate some of this pain and accelerate some of my future projects with LangGraph. Hope it can be a helpful resource for the community too!

https://github.com/JoshuaC215/agent-service-toolkit

Try the running app: https://agent-service-toolkit.streamlit.app/

[adambouladxocialive]

Hello Joshua,
First of all great work...
When I try to run the streamlit app (streamlit run....), the execution exists after the first chat message.
I am using Ollama BTW.

[JoshuaC215]

Hi @adambouladxocialive , can you share more details about the error message and link to code? otherwise won't be able to take a look. Feel free to file an issue in the repo to discuss there with more detailed steps and info, it'll be the best way to follow up!

[jack-equimkt]

hey joshua, thank you so much for sharing such an awesome project ! I've been struggling to handle results from a tool call that synthesizes structured data with the with_structured_output() function. I can't figure out a way to stream normal messages, but NOT stream the tool call -- because what good is 'structured data' if it streams to the client interface. It seems like you handled your's pretty well. Any ideas?

[JoshuaC215]

Hey Jack, hard to say, I would take a look in more detail at my repo. There's some semi-complex logic for handling the events from langgraph/langchain and deciding what to pass to the user - specifically, look here: https://github.com/JoshuaC215/agent-service-toolkit/blob/bbfb26e7ce3d8a265ee725948b2f5dc657c77a10/src/service/service.py#L147-L195

This part has been somewhat fragile as LangChain / LangGraph continued to evolve. I'm not sure if there's a better way to handle it. Hope that helps.