No useful traceback when using pydantic schema and failing validation

[eyurtsev]

Privileged issue

• I am a LangChain maintainer, or was asked directly by a LangChain maintainer to create an issue here.

Issue Content

The code below fails on entry to ok_node, but the stack trace has no information about which node failed.

Given that the same schema is often shared between all nodes in the graph (at least by default) -- it makes it impossible to determine where the run time error is occurring.

from langgraph.graph import StateGraph, START, END
from typing_extensions import TypedDict

from pydantic import BaseModel

# The overall state of the graph (this is the public state shared across nodes)
class OverallState(BaseModel):
    a: str

def bad_node(state: OverallState):
    return {
        "a": 123 # Invalid
    }

def ok_node(state: OverallState):
    return {
        "a": "goodbye"
    }


# Build the state graph
builder = StateGraph(OverallState)
builder.add_node(bad_node)
builder.add_node(ok_node)
builder.add_edge(START, "bad_node")
builder.add_edge("bad_node", "ok_node")
builder.add_edge("ok_node", END)
graph = builder.compile()

# Test the graph with a valid input
graph.invoke({ "a": "hello"})

Part of error trace

354 return tasks

File ~/.pyenv/versions/3.11.4/envs/core/lib/python3.11/site-packages/langgraph/pregel/algo.py:495, in prepare_single_task(task_path, task_id_checksum, checkpoint, processes, channels, managed, config, step, for_execution, store, checkpointer, manager)
485 if triggers := sorted(
486 chan
487 for chan in proc.triggers
(...)
492 > seen.get(chan, null_version)
493 ):
494 try:
--> 495 val = next(
496 _proc_input(
497 step, proc, managed, channels, for_execution=for_execution
498 )
499 )
500 except StopIteration:
501 return

File ~/.pyenv/versions/3.11.4/envs/core/lib/python3.11/site-packages/langgraph/pregel/algo.py:627, in _proc_input(step, proc, managed, channels, for_execution)
625 # If the process has a mapper, apply it to the value
626 if for_execution and proc.mapper is not None:
--> 627 val = proc.mapper(val)
629 yield val

File ~/.pyenv/versions/3.11.4/envs/core/lib/python3.11/site-packages/langgraph/graph/state.py:704, in _coerce_state(schema, input)
703 def _coerce_state(schema: Type[Any], input: dict[str, Any]) -> dict[str, Any]:
--> 704 return schema(**input)

File ~/.pyenv/versions/3.11.4/envs/core/lib/python3.11/site-packages/pydantic/main.py:212, in BaseModel.init(self, **data)
210 # __tracebackhide__ tells pytest and some other tools to omit this function from tracebacks
211 tracebackhide = True
--> 212 validated_self = self.pydantic_validator.validate_python(data, self_instance=self)
213 if self is not validated_self:
214 warnings.warn(
215 'A custom validator is returning a value other than self.\n'
216 "Returning anything other than self from a top level model validator isn't supported when validating via __init__.\n"
217 'See the model_validator docs (https://docs.pydantic.dev/latest/concepts/validators/#model-validators) for more details.',
218 category=None,
219 )

ValidationError: 1 validation error for OverallState
a
Input should be a valid string [type=string_type, input_value=123, input_type=int]
For further information visit https://errors.pydantic.dev/2.9/v/string_type

[FrancisGajitos]

Hey @eyurtsev,
I'm in a group of 5 students from the University of Toronto and we're interested in looking into this issue! Just wanted to ask if there's any other important information that could help us fix this error traceback?

Thanks!

[eyurtsev]

I'd follow the stack trace to identify the location where the exception is raised.

File [~/.pyenv/versions/3.11.4/envs/core/lib/python3.11/site-packages/langgraph/graph/state.py:704](http://localhost:8888/home/eugene/.pyenv/versions/3.11.4/envs/core/lib/python3.11/site-packages/langgraph/graph/state.py#line=703), in _coerce_state(schema, input)
703 def _coerce_state(schema: Type[Any], input: dict[str, Any]) -> dict[str, Any]:
--> 704 return schema(**input)

And here figure out whether it's possible to catch the error and re-raise with a better exception or error message that has information about the current execution step or the previous step that caused a bad state update.

This could be a 1 line change or more (i haven't looked at what information is available).

[FrancisGajitos]

Hey,

We have a couple of observations after trying to follow the stack trace:

We followed the stack trace and it eventually led us to a variable called "pydantic_validator" within the main.py class of the pydantic package.

File "C:\Python312\Lib\site-packages\pydantic\main.py", line 211, in __init__ validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ pydantic_core._pydantic_core.ValidationError: 1 validation error for OverallState

It seems like this pydantic_validator variable outputted a certain type of ValidationError, specifically flagging the input's type not matching a str field (see image below; from the Pydantic documentation).

It looks like the functionality to configure and output the error message was implemented in the Pydantic package rather than within a section of the Langgraph repository.

We're considering another solution that only involves modifying files within the Langgraph repository. In your above example, you add both nodes to your graph using the StateGraph.add_node() function. When we looked at how add_node was implemented in libs\langgraph\langgraph\graph\state.py, we noticed that there has already been some validation added (i.e., according to the docstring of one of the versions of add_node, a ValueError is raised is the key is already being used as a state key).

We're thinking to add another validation rule that would raise a more clear error message prior to the node being added to the StateGraph. Just to illustrate, here's the location in state.py where we're thinking to throw the error message.

    def add_node(
        self,
        node: Union[str, RunnableLike],
        action: Optional[RunnableLike] = None,
        *,
        metadata: Optional[dict[str, Any]] = None,
        input: Optional[Type[Any]] = None,
        retry: Optional[RetryPolicy] = None,
    ) -> Self:
        """Adds a new node to the state graph.

        Will take the name of the function/runnable as the node name.

        Args:
            node (Union[str, RunnableLike)]: The function or runnable this node will run.
            action (Optional[RunnableLike]): The action associated with the node. (default: None)
            metadata (Optional[dict[str, Any]]): The metadata associated with the node. (default: None)
            input (Optional[Type[Any]]): The input schema for the node. (default: the graph's input schema)
            retry (Optional[RetryPolicy]): The policy for retrying the node. (default: None)
        Raises:
            **ValueError: If the key is already being used as a state key.**

[EXAMPLE SECTION OF DOCSTRING]

        if not isinstance(node, str):
            action = node
            if isinstance(action, Runnable):
                node = action.get_name()
            else:
                node = getattr(action, "__name__", action.__class__.__name__)
            if node is None:
                raise ValueError(
                    "Node name must be provided if action is not a function"
                )
        if node in self.channels:
            raise ValueError(f"'{node}' is already being used as a state key")

        [PSEUDOCODE]

        for value in state_attribute_values:
            if any of the node's values for state attributes are invalid:
                raise ValueError(f"'{node}' has an invalid value for attribute {attribute} for state {state}")

[REST OF VALIDATIONS BELOW]

Although we're still looking into this issue, we just wanted to ask if you have any thoughts about this idea for a possible solution for this. We're still learning about how Langgraph works as a whole and we would appreciate any advice or feedback from you about our observations or solution. If you have any questions or need anything clarified from this post, please feel free to let me know.

Thanks in advance!