Correct way to implement moderation

[darthShana]

Hi Team
I am trying to build a virtual medical receptionist for a medical clinic. I would like it to suggest suitable medical service to the human. Despite me saying in the agent prompt

"Your task to help visitors and guide them to an appropriate medial service offered by the clinic, but do NOT offer any medical advice"

I noticed sometimes the assistant will still suggest various treatments for the users queries. So I thought i would implement a moderation step to effectively filter this content out.

i created this graph

workflow.add_conditional_edges(
    "assistant",
    moderated_tools_condition,
)
workflow.add_conditional_edges(
    "tools",
    route_tool_results,
    {
        "assistant": "assistant",
        "display_results": "display_results"
    }
)
workflow.add_edge("display_results", "__end__")
workflow.add_edge("moderation", "__end__")

where the edge is

def moderated_tools_condition(state: AgentState) -> Literal["tools", "moderation"]:
    if messages := state.get("messages", []):
        ai_message = messages[-1]
        if hasattr(ai_message, "tool_calls") and len(ai_message.tool_calls) > 0:
            return "tools"
        return "moderation"

and moderation node is

def moderation(state: AgentState):
    if messages := state.get("messages", []):
        ai_message = messages[-1]
        return {"messages": [moderator.moderate(ai_message.content)]}

while this works it seems to get the state confused. As seen in this diagram

It seems to think the human has repeated the moderated content.
Should i be using this method? In which case there seems to be a bug here
Is there something different i should use?

[hinthornw]

So then moderate returns a new Human Message?

Assuming moderate looks something like:

def moderate(self, content: str):
    return HumanMessage(content="this has been moderated" + content)

You'll want to make sure the message id's are aligned so it isn't inserted after the initial response. Example here: https://langchain-ai.github.io/langgraph/tutorials/introduction/?h=add+mess#part-5-manually-updating-the-state

[darthShana]

oh i see
I had moderate return a string

    def moderate(self, content: str) -> str:

ill look through the example. in you sent