Added a mechanism to extract metadata from MCP tool call response #3339

anirbanbasu · 2025-11-04T22:26:15Z

feat: Added a mechanism to extract metadata from MCP tool call response.
feat: Added new MCP tools that attach metadata to MCP TextContent and to dict[str, Any] as a _meta key.
test: Added tests that run successfully.
TODOs
- Should really capture metadata from all MCP content types, not just TextContent.
- Need to support both overall tool result metadata and metadata in content blocks.
- Check that coverage is 100%.
- Must add additional tests to use ToolResult.meta from FastMCP 2.13.1 as documented at https://gofastmcp.com/servers/tools#toolresult-and-metadata.

feat: Added a new MCP tool that attaches metadata to MCP TextContent. test: Added a test to call the aforementioned tool (failing as of now).

anirbanbasu · 2025-11-04T22:27:07Z

@DouweM this is the PR in response to #3323 -- I am still working on it.

…Content.

anirbanbasu · 2025-11-05T12:06:04Z

Hi @DouweM, quoting you from #3323 (comment), I am pondering over whether it is better to edit the _map_tool_result_part in the MCPServer to handle the multi-modal content or modify the _call_tool method in the _agent_graph.py.

At present, if you take a look at the _map_tool_result_part method in this PR, I have only modified it to include the metadata if it is TextContent. I think I must also add the metadata if the response has multi-modal content. So, perhaps, changing this method to handle the multi-modal content is better instead of editing the _call_tool method.

todo: Exhaustive tests to improve coverage.

anirbanbasu · 2025-11-05T13:17:32Z

Quoting myself (#3339 (comment))

[...] perhaps, changing this [i.e., _map_tool_result_part ] method to handle the multi-modal content is better instead of editing the _call_tool method

Note that I just did that @DouweM. Need to add more tests to improve coverage.

DouweM · 2025-11-05T16:41:31Z

pydantic_ai_slim/pydantic_ai/mcp.py

-            return structured
+            if isinstance(structured, dict) and (
+                (len(structured) == 1 and 'result' in structured)
+                or (len(structured) == 2 and 'result' in structured and '_meta' in structured)


In the example at https://gofastmcp.com/servers/tools#toolresult-and-metadata, wouldn't the metadata be on result.meta? I don't think we should try to parse it directly from the result.structuredData

I agree with you regarding not modifying the structured result.

We may be, though, not thinking of the same metadata.

What you are referring to is a FastMCP tool call result metadata. In addition, the meta is an attribute of FastMCP ToolResult only from version 2.13.1 (according to https://gofastmcp.com/servers/tools#toolresult-and-metadata) while the version currently in use with Pydantic AI is 2.12.4.

What I have been referring to is the _meta in the latest MCP standard https://modelcontextprotocol.io/specification/2025-06-18/basic/index#meta. FastMCP wraps this in TextContent and other types of content too.

If we go with the FastMCP-specific meta then there is a possibility that a MCP server implemented without using that specific version (> 2.13.1) of FastMCP or implemented in a different language will not return the metadata in the expected ToolResult style object.

Having said that, there is a possibility that FastMCP is implementing what the upcoming MCP standard will be, as they seem to typically do. (I haven't dug through this in details.)

In summary:

for structured content, I think we could go with meta of ToolResult but I need to upgrade FastMCP for Pydantic AI to 2.13.1 or above;

however, we ought to support _meta aliased meta for each content type.

Regarding (1), is this something I should do by myself?

What are your thoughts?

Actually, I noticed that in my original issue #3323, I had referred to both the FastMCP metadata and the standards _meta. Sorry for the confusion.

Ideally, we should support both.

Fixed this in commit 39d47b5.

DouweM · 2025-11-05T16:43:01Z

pydantic_ai_slim/pydantic_ai/mcp.py

    async def _map_tool_result_part(
        self, part: mcp_types.ContentBlock
-    ) -> str | messages.BinaryContent | dict[str, Any] | list[Any]:
+    ) -> str | messages.ToolReturn | messages.BinaryContent | dict[str, Any] | list[Any]:


This is not going to work, because now the tool call could return a list of ToolReturns which is not supported: the tool needs to itself return a ToolReturn object.

I think we should build the list of output contents as we used to, and then if there's result.meta, return a ToolReturn with that metadata + the output content, instead of returning the output content directly

Okay, but as I mentioned in my comment above, what I have been referring to is the _meta in the latest MCP standard https://modelcontextprotocol.io/specification/2025-06-18/basic/index#meta. This can be present in each content block, it seems.

If we return a single meta, there is no way to know how to merge multiple _meta that may be present in the content blocks.

Rewrote the logic in commit 39d47b5.

pydantic_ai_slim/pydantic_ai/mcp.py

DouweM · 2025-11-07T14:59:43Z

@anirbanbasu I'll respond here at the top level with some thoughts because the 3 threads touch on overlapping topics:

There should be a way to access both tool-result metadata and tool-result-item metadata
MCP supports _meta on every object, so result.meta is not FastMCP specific, it's right in the MCP SDK: CallToolResult inherits from Result which has a meta field corresponding to _meta.
1. I think this is the natural place to read "tool-result metadata" from (and for any MCP server SDK to write metadata)
2. I don't think we should ever try to extract metadata from result.structuredData, as we should treat it as arbitrary data
If there is only tool-result metadata and no tool-result-item metadata, MCPServer.direct_call_tool should return it as ToolReturn.metadata
If there is no tool-result metadata, but there is a single tool-result-item (TextContent etc) that does have .meta, that should be the ToolReturn.metadata
If there are multiple pieces of metadata, on the tool-result and/or on one or more tool-result-items, we have a couple of options. Since ToolReturn.content will hold the tool-result-items in this case, perhaps ToolReturn.metadata should be a dictionary like {'result': <result metadata>, 'content': [<metadata for content index 0>, ...]]}
1. Another option is to leverage the existing process_tool_call hook, but (unfortunately?) it gets the processed tool result rather than the raw thing. Perhaps it could pass a flag into direct_call_tool to return the raw mcp.types.CallToolResult so metadata can be read off of it?

Let me know what you think. The fact that there can be metadata at multiple levels in MCP but not currently in Pydantic AI makes this tricky!

anirbanbasu · 2025-11-08T13:22:53Z

@DouweM thanks a lot for your thoughts. Here are my responses.

There should be a way to access both tool-result metadata and tool-result-item metadata

Yes, good idea. However, to pass the tool-result metadata, one needs to use FastMCP 2.13.1 (as mentioned in https://gofastmcp.com/servers/tools#toolresult-and-metadata) but this version is yet to be released. Thus, while I can parse tool-result metadata using the CallToolResult, I cannot create the tool to pass the metadata for pytest testing -- unless Pydantic AI uses the latest from the default branch of FastMCP's GitHub, which is a bad idea, in my understanding.

Related to this, I am attaching tool-result metadata using FastMCP's latest from GitHub in my MCP server template.

MCP supports _meta on every object, so result.meta is not FastMCP specific, it's right in the MCP SDK: CallToolResult inherits from Result which has a meta field corresponding to _meta.

Yes, indeed. Thanks for pointing out. That link effectively points to this documentation in a comment, which is the same as the June standard.

(2) i. I think this is the natural place to read "tool-result metadata" from (and for any MCP server SDK to write metadata)

Okay, but is it (i.e., _meta) also referring to the tool-result-item metadata?

(2) ii. I don't think we should ever try to extract metadata from result.structuredData, as we should treat it as arbitrary data

Yes, that was my mistake. Apologies. I will fix it. If the data is structured data then I can attach the metadata as I am doing in my MCP server template.

If there is only tool-result metadata and no tool-result-item metadata, MCPServer.direct_call_tool should return it as ToolReturn.metadata

Okay, makes sense.

If there is no tool-result metadata, but there is a single tool-result-item (TextContent etc) that does have .meta, that should be the ToolReturn.metadata

Okay, that's reasonable.

If there are multiple pieces of metadata, on the tool-result and/or on one or more tool-result-items, we have a couple of options. Since ToolReturn.content will hold the tool-result-items in this case, perhaps ToolReturn.metadata should be a dictionary like {'result': <result metadata>, 'content': [<metadata for content index 0>, ...]]}
i. Another option is to leverage the existing process_tool_call hook, but (unfortunately?) it gets the processed tool result rather than the raw thing. Perhaps it could pass a flag into direct_call_tool to return the raw mcp.types.CallToolResult so metadata can be read off of it?

The first option sounds reasonable as a default option even if the dictionary keys seem a bit hard-coded.

Instead of your suggested second option, how about letting the process_tool_call hook receive the processed ToolResult but alter it as necessary? This should be possible because the ToolReturn.metadata is just a dictionary after all.

…antic#3339 (comment)

anirbanbasu · 2025-11-09T09:23:10Z

@DouweM with the latest commit (39d47b5), I have attempted to implement 3, 4 and 5 from your message above (#3339 (comment)).

Notes.

I modified 5 a bit to return a dictionary instead of a list because there may be content parts that do not have corresponding metadata.
Modified _map_tool_result_part to return a tuple so that the second item in the tuple is the part metadata if it exists.
I am not convinced if the code (see snippet below, lines 303-309 of mcp.py) to handle the part content when it is not multi-modal is what is intended.

if isinstance(mapped_part, messages.BinaryContent):
   identifier = mapped_part.identifier

   return_values.append(f'See file {identifier}')
   user_contents.append([f'This is file {identifier}:', mapped_part])
else:
   user_contents.append(mapped_part)

Tests will definitely fail because they have not been correctly written yet.
Coverage will also fail for the same reason.

pydantic_ai_slim/pydantic_ai/mcp.py

anirbanbasu · 2025-11-13T22:08:04Z

@DouweM, I will let you know once I have completed all your requested changes. Sorry for delay.

…multi-modal content parsing.

…implified? experimental: Is the nested metadata assumption right? See line 475 in mcp.py

DouweM · 2025-11-17T20:25:16Z

@anirbanbasu Please have a look at the failing tests!

…o version 2.13.1 to be able to write more exhaustive tests for `meta` `ToolResult` in the MCP server.

anirbanbasu · 2025-11-18T02:34:08Z

@DouweM updated the tests and included some # pragma no-cover directives to parts of the metadata processing code that can only be tested once FastMCP is upgraded to 2.13.1. I shall write those test once FastMCP is upgraded.

DouweM · 2025-11-21T00:37:18Z

pydantic_ai_slim/pydantic_ai/_agent_graph.py

                    f'The return value of tool {tool_call.tool_name!r} contains invalid nested `ToolReturn` objects. '
                    f'`ToolReturn` should be used directly.'
                )
+            # TODO: Keep updated with the binary parsing in mcp.py


Suggested change

# TODO: Keep updated with the binary parsing in mcp.py

# TODO: Keep updated with the binary parsing in `mcp.py`

# or remove comment once https://github.com/pydantic/pydantic-ai/issues/3253 is done

DouweM · 2025-11-21T00:37:42Z

pydantic_ai_slim/pydantic_ai/mcp.py

+
+        parts_with_metadata = [await self._map_tool_result_part(part) for part in result.content]
+        parts_only = [part for part, _ in parts_with_metadata]
+        # any_part_has_metadata = any(metadata is not None for _, metadata in parts_with_metadata)


Not using this anymore?

DouweM · 2025-11-21T00:38:19Z

pydantic_ai_slim/pydantic_ai/mcp.py

+
+        # The following branching cannot be tested until FastMCP is updated to version 2.13.1
+        # such that the MCP server can generate ToolResult and result.meta can be specified.
+        # TODO: Add tests for the following branching once FastMCP is updated.


Adding a comment here so we don't lose this

DouweM · 2025-11-21T00:38:35Z

pydantic_ai_slim/pydantic_ai/mcp.py

+        # such that the MCP server can generate ToolResult and result.meta can be specified.
+        # TODO: Add tests for the following branching once FastMCP is updated.
+        if len(parts_metadata) > 0:
+            if result.meta is not None and len(result.meta) > 0:  # pragma: no cover


We should not have any no-covers if we can help it!

Edit: You already pointed out why you did that, never mind :)

Also is this equivalent to if result.meta?

DouweM · 2025-11-21T00:39:39Z

pydantic_ai_slim/pydantic_ai/mcp.py

+        # The following branching cannot be tested until FastMCP is updated to version 2.13.1
+        # such that the MCP server can generate ToolResult and result.meta can be specified.
+        # TODO: Add tests for the following branching once FastMCP is updated.
+        if len(parts_metadata) > 0:


Here too, I prefer if parts_metadata over specifically length-checking, unless we want to treat things like None and empty differently

DouweM · 2025-11-21T00:40:33Z

pydantic_ai_slim/pydantic_ai/mcp.py

+                else:
+                    return_metadata = {'content': parts_metadata}  # pragma: no cover
+        else:
+            if result.meta is not None and len(result.meta) > 0:  # pragma: no cover


This can be elif result.meta:

DouweM · 2025-11-21T00:41:01Z

pydantic_ai_slim/pydantic_ai/mcp.py

-                if len(resource_result.contents) == 1
-                else [self._get_content(resource) for resource in resource_result.contents]
-            )
+            # Check if metadata already exists. If so, merge it with nested the resource metadata.


Can we dedupe any of this with the above with some helper functions?

DouweM · 2025-11-21T00:41:20Z

tests/test_mcp.py

        assert result == snapshot(32.0)


+async def test_tool_response_metadata(run_context: RunContext[int]):


We'll want tests of every combination that we've covered up above

DouweM · 2025-11-21T00:45:04Z

@anirbanbasu Thanks Anirban, I think the logic is reasonable now, although I'm starting to have doubts about how brittle it will be to consume this metadata from code, as it's not type safe and will require lots of isinstance and key in dict checks.

I'm thinking we should add metadata fields to the multi-modal parts themselves, so we don't have to carry it all through the ToolReturn and then ToolReturnPart. Would you be up for making that change?

anirbanbasu · 2025-11-21T01:10:34Z

@DouweM I would agree that packaging the parts metadata in the way we are doing in a dictionary is brittle.

Having a metadata in the BinaryContent would be a good idea, if that is what you meant.

In addition, I wonder if there is any need to package metadata in one place in the return ToolResult. Thus, the ToolResult.meta should only have the top-level metadata, while parts contents, such as TextContent can have its own metadata.

What do you think?

Lastly, is there a plan to upgrade FastMCP to 2.13.1, now that it is an officially released version?

DouweM · 2025-11-21T18:28:50Z

Having a metadata in the BinaryContent would be a good idea, if that is what you meant.

@anirbanbasu Yep.

In addition, I wonder if there is any need to package metadata in one place in the return ToolResult. Thus, the ToolResult.meta should only have the top-level metadata, while parts contents, such as TextContent can have its own metadata.

What do you think?

Yep that's what I'm hoping we can do this way.

Lastly, is there a plan to upgrade FastMCP to 2.13.1, now that it is an officially released version?

Feel free to update!

anirbanbasu · 2025-11-22T00:59:10Z

@DouweM okay, I will modify the BinaryContent as discussed and keep parts metadata with the parts themselves.

However, we will run into an issue with TextContent because the current code discards its metadata and this cannot be packaged as BinaryContent. Perhaps, we should output TextContent as is?

Lastly, is there a plan to upgrade FastMCP to 2.13.1, now that it is an officially released version?

Feel free to update!

It seems that updating it to 2.13.1 is impossible for now due to other dependencies.

% uv lock -P fastmcp --dry-run                   
Resolved 381 packages in 191ms
Update fastmcp v2.12.4 -> v2.12.5

This would mean that an exhaustive test set cannot be implemented until we can get 2.13.1.

anirbanbasu added 2 commits November 5, 2025 07:21

feat: Added a mechanism to extract metadata from MCP tool call response.

4bd79c9

feat: Added a new MCP tool that attaches metadata to MCP TextContent. test: Added a test to call the aforementioned tool (failing as of now).

Merge branch 'main' into patch-3323

bb1c256

anirbanbasu added 5 commits November 5, 2025 08:39

Merge branch 'main' into patch-3323

98412c0

fix: Attempted fix to check the _meta tag presence in the MCP response.

5c3f58f

Merge branch 'pydantic:main' into patch-3323

73ba0a1

fix: Tests seem to work with the MCP metadata but are these exhaustive?

0e5ae00

chore: Improved metadata parsing for both structured content and Text…

ad73591

…Content.

chore: Added code to handle multi-modal content and metadata.

dfe81b6

todo: Exhaustive tests to improve coverage.

chore: Added no cover pragma to some portions of the code.

fd806f7

anirbanbasu marked this pull request as ready for review November 5, 2025 14:06

DouweM requested changes Nov 5, 2025

View reviewed changes

DouweM self-assigned this Nov 5, 2025

DouweM added the awaiting author revision label Nov 5, 2025

Merge branch 'main' into patch-3323

d99be83

Merge branch 'main' into patch-3323

f8feb5b

experimental: Potential implementation of 3, 4 and 5 described in pyd…

39d47b5

…antic#3339 (comment)

anirbanbasu requested a review from DouweM November 9, 2025 12:36

DouweM requested changes Nov 11, 2025

View reviewed changes

anirbanbasu added 3 commits November 12, 2025 22:21

Merge branch 'main' into patch-3323

6eea048

Merge branch 'main' into patch-3323

f54b127

in-progress: Addressed a number of comments from the changes requested.

7935568

anirbanbasu added 2 commits November 14, 2025 07:08

Merge branch 'main' into patch-3323

026b364

Merge branch 'main' into patch-3323

b8ce49c

anirbanbasu added 4 commits November 14, 2025 23:11

chore: MCP metadata parsing logic updated.

13fb859

chore: Added references between mcp.py and _agent_graph.py regarding …

46a7b87

…multi-modal content parsing.

Merge branch 'main' into patch-3323

41b3aa4

chore: Enabled parsing nested metadata from resource. Code could be s…

8154885

…implified? experimental: Is the nested metadata assumption right? See line 475 in mcp.py

anirbanbasu requested a review from DouweM November 14, 2025 23:06

Merge branch 'main' into patch-3323

0133802

anirbanbasu added 7 commits November 18, 2025 06:15

Merge branch 'main' into patch-3323

4a58898

fix: Corrected tests for MCP metadata.

137126a

fix: Added no cover pragma with TODO notes until FastMCP is updated t…

c5b3b57

…o version 2.13.1 to be able to write more exhaustive tests for `meta` `ToolResult` in the MCP server.

Merge branch 'main' into patch-3323

a0a1294

fix: Corrected no-cover directives in the MCP server.

0c8251d

fix: Added another no-cover directive.

55598da

fix: Correcting no-cover directives again.

9f0b28f

anirbanbasu added 2 commits November 19, 2025 21:55

Merge branch 'main' into patch-3323

a97b711

Merge branch 'main' into patch-3323

8267b71

DouweM mentioned this pull request Nov 20, 2025

Feature: MCP client Resources support #3024

Merged

DouweM requested changes Nov 21, 2025

View reviewed changes

anirbanbasu added 2 commits November 22, 2025 10:01

Merge branch 'main' into patch-3323

2674863

Merge branch 'main' into patch-3323

bf5e75c

	# TODO: Keep updated with the binary parsing in mcp.py
	# TODO: Keep updated with the binary parsing in `mcp.py`
	# or remove comment once https://github.com/pydantic/pydantic-ai/issues/3253 is done

		assert result == snapshot(32.0)


		async def test_tool_response_metadata(run_context: RunContext[int]):

Added a mechanism to extract metadata from MCP tool call response #3339

Are you sure you want to change the base?

Added a mechanism to extract metadata from MCP tool call response #3339

Conversation

anirbanbasu commented Nov 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

anirbanbasu commented Nov 4, 2025

Uh oh!

anirbanbasu commented Nov 5, 2025

Uh oh!

anirbanbasu commented Nov 5, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

DouweM commented Nov 7, 2025

Uh oh!

anirbanbasu commented Nov 8, 2025

Uh oh!

anirbanbasu commented Nov 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

anirbanbasu commented Nov 13, 2025

Uh oh!

DouweM commented Nov 17, 2025

Uh oh!

anirbanbasu commented Nov 18, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

DouweM Nov 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

DouweM commented Nov 21, 2025

Uh oh!

anirbanbasu commented Nov 21, 2025

Uh oh!

DouweM commented Nov 21, 2025

Uh oh!

anirbanbasu commented Nov 22, 2025

Uh oh!

Reviewers

anirbanbasu commented Nov 4, 2025 •

edited

Loading

anirbanbasu commented Nov 9, 2025 •

edited

Loading

DouweM Nov 21, 2025 •

edited

Loading