Howto: JSON transformation for REST output

[alex9434]

Dear all,

I want to include a "web-search" service that searches using a local SearXNG instance. Here is the JSON I use for "bulk import":

[
  {
    "name": "web-search",
    "displayName": "SearXNG web",
    "url": "http://172.17.0.1:5346/search?format=json&safesearch=1",
    "integration_type": "REST",
    "request_type": "GET",
    "description": "Search the internet with keywords",
    "headers": {
      "Accept": "application/json",
      "User-Agent": "MCP-Gateway/1.0"
    },
    "input_schema": {
      "type": "object",
      "properties": {
        "q": {
          "type": "string",
          "description": "The search query string. Required."
        },
        "language": {
          "type": "string",
          "description": "Language code. Optional, default is 'en'.",
          "default": "en"
        },
        "categories": {
          "type": "string",
          "description": "Text-based categories to search in: general, news, it_category, science, social media, map. Optional, default is 'general'.",
          "enum": ["general", "news", "it_category", "science", "social_media", "map"],
          "default": "general"
        }
      },
      "required": ["q"]
    },
    "tags": ["search", "api", "internal"]
  }
]

The issue with this definition is the output that the MCP gateway than provides to clients. The original reply from SearXNG looks like this:

{
  "query": "mcp context forge",
  "number_of_results": 0,
  "results": [
    {
      "url": "https://github.com/IBM/mcp-context-forge",
      "title": "IBM/mcp-context-forge - GitHub",
      "content": "ContextForge MCP Gateway is a feature-rich gateway, proxy and MCP Registry that federates MCP and REST services - unifying discovery, auth, rate-limiting, ...",
      "thumbnail": "",
      "engine": "google",
      "template": "default.html",
      "parsed_url": [
        "https",
        "github.com",
        "/IBM/mcp-context-forge",
        "",
        "",
        ""
      ],
      "img_src": "",
      "priority": "",
      "engines": [
        "google",
        "startpage"
      ],
      "positions": [
        1,
        1
      ],
      "score": 4.0,
      "category": "general",
      "publishedDate": null
    }
  ],
  "answers": [],
  "corrections": [],
  "infoboxes": [],
  "suggestions": [
    "IBM MCP Gateway",
    "Mcp context forge pypi",
    "MCP-Forge",
    "TrueFoundry MCP Gateway",
    "Lasso MCP Gateway",
    "MCP gateway GitHub"
  ],
  "unresponsive_engines": [
    [
      "brave",
      "Suspended: too many requests"
    ],
    [
      "duckduckgo",
      "CAPTCHA"
    ]
  ]
}

However, MCP Context Forge returns this:

{
  "jsonrpc": "2.0",
  "result": {
    "content": [
      {
        "type": "text",
        "text": "{\n  \"query\": \"context forge\",\n  \"number_of_results\": 0,\n  \"results\": [\n    {\n      \"url\": \"https://github.com/IBM/mcp-context-forge\",\n      \"title\": \"IBM/mcp-context-forge - GitHub\",\n      \"content\": \"ContextForge MCP Gateway is a feature-rich gateway, proxy and MCP Registry that federates MCP and REST services - unifying discovery, auth, rate-limiting, ...\",\n      \"thumbnail\": \"\",\n      \"engine\": \"google\",\n      \"template\": \"default.html\",\n      \"parsed_url\": [\n        \"https\",\n        \"github.com\",\n        \"/IBM/mcp-context-forge\",\n        \"\",\n        \"\",\n        \"\"\n      ],\n      \"img_src\": \"\",\n      \"priority\": \"\",\n      \"engines\": [\n        \"google\",\n        \"startpage\"\n      ],\n      \"positions\": [\n        1,\n        1\n      ],\n      \"score\": 4.0,\n      \"category\": \"general\",\n      \"publishedDate\": null\n    }\n  ],\n  \"answers\": [],\n  \"corrections\": [],\n  \"infoboxes\": [],\n  \"suggestions\": [\n    \"IBM MCP Gateway\",\n    \"Context forge example\",\n    \"Mcp context forge pypi\",\n    \"MCP Gateway Docker\",\n    \"MCP-Forge\",\n    \"Context Forge IBM\",\n    \"Context forge github\",\n    \"Open source MCP gateways\",\n    \"Lasso MCP Gateway\",\n    \"IBM MCP server\",\n    \"MCP gateway GitHub\",\n    \"MCP Gateway architecture\"\n  ],\n  \"unresponsive_engines\": [\n    [\n      \"brave\",\n      \"Suspended: too many requests\"\n    ],\n    [\n      \"duckduckgo\",\n      \"CAPTCHA\"\n    ]\n  ]\n}"
      }
    ],
    "isError": false
  },
  "id": 1769484634884
}

I think the issue is that I need to define an Output Schema (JSON) but I cannot find how to do it? Here is what I want:

1. Remove fields/values that are not needed by the LLM, e.g. "thumbnail", "engine", "template", "parsed_url"
2. Ideally keep the structured JSON format and do not mash everything in a text block.
   Can anyone help me with this? I am stuck. Is there any documentation on how to do this. I could not find anything.

[crivetimihai]

Hi @alex9434, thanks for the feature request. I can provide a concrete example for transformation patterns if you share your target output shape and whether this is at tool definition time or per-call transform.

[alex9434]

Hi @crivetimihai, that would be fantastic! I would suggest having a plugin (maybe similar to your json_repair) plugin, that strips unnecessary fields from noisy APIs to safe tokens. It might be actually quite common that APIs produce fields/values that are not helpful to the lllm and could be eliminated.
In my example I would suggest to reduce the json to:
{
"query": "mcp context forge",
"results": [
{
"url": "https://github.com/IBM/mcp-context-forge",
"title": "IBM/mcp-context-forge - GitHub",
"content": "ContextForge MCP Gateway is a feature-rich gateway, proxy and MCP Registry that federates MCP and REST services - unifying discovery, auth, rate-limiting, ...",
"score": 4.0,
"category": "general",
"publishedDate": null
}
],
"answers": [],
"corrections": [],
"infoboxes": [],
"suggestions": [
"IBM MCP Gateway",
"Mcp context forge pypi",
"MCP-Forge",
"TrueFoundry MCP Gateway",
"Lasso MCP Gateway",
"MCP gateway GitHub"
]
}
I think the ideal approach could be to "whitelist" fields that get passed through. All non-whitelisted fields/values get eliminated from the response. Please keep in mind that the results structure is repeated n-times. Is this something that should be done by a plugin?

[alex9434]

Hi @crivetimihai, I realized that this is indeed a feature request and not a discussion. So I opened a feature request and included the plugin that I wrote json_prune.py. Is this the right approach? Please let me know what you think.

[crivetimihai]

Hi @alex9434, good news — this is already supported via the jsonpath_filter field on tool definitions.

The jsonpath_filter uses jq syntax to transform REST API responses before they're returned to the MCP client. Here's how to use it for your SearXNG use case:

Updated tool definition with jq filter:

[
  {
    "name": "web-search",
    "displayName": "SearXNG web",
    "url": "http://172.17.0.1:5346/search?format=json&safesearch=1",
    "integration_type": "REST",
    "request_type": "GET",
    "description": "Search the internet with keywords",
    "headers": {
      "Accept": "application/json",
      "User-Agent": "MCP-Gateway/1.0"
    },
    "jsonpath_filter": "{query, results: [.results[] | {url, title, content, score, category}], suggestions}",
    "input_schema": {
      "type": "object",
      "properties": {
        "q": {
          "type": "string",
          "description": "The search query string. Required."
        },
        "language": {
          "type": "string",
          "description": "Language code. Optional, default is 'en'.",
          "default": "en"
        },
        "categories": {
          "type": "string",
          "description": "Categories to search in.",
          "enum": ["general", "news", "it_category", "science", "social_media", "map"],
          "default": "general"
        }
      },
      "required": ["q"]
    },
    "tags": ["search", "api", "internal"]
  }
]

The jsonpath_filter value {query, results: [.results[] | {url, title, content, score, category}], suggestions} will:

1. Keep only query, results, and suggestions from the top level
2. For each result, keep only url, title, content, score, and category
3. Strip thumbnail, engine, template, parsed_url, img_src, etc.

Note: The output will still be serialized as a text content block in the MCP response (this is how MCP tool results work), but the JSON inside will be clean and minimal. You can also add an output_schema for additional validation if needed.

You can set jsonpath_filter via the bulk import JSON, the API (POST /tools), or the Admin UI tool editor.

[itxashancode]

The previous comments from crivetimihai said that jsonpath_filter field on tool definitions is already supported, using jq syntax to transform REST API responses before they're returned to the MCP client. They gave an example for SearXNG use case. The user hasn't shown the updated tool definition yet; they just posted the original bulk import. The question is "Howto: JSON transformation for REST output". Provide a concrete example of jsonpath_filter usage for this specific case. Possibly show how to extract only the results array, or maybe map to a simpler shape. Not repeat what crivetimihai said, but build on it or correct if wrong. They said jsonpath_filter uses jq syntax. That's correct per the repo? I need to recall: In mcp-context-forge, tool definitions can have a "transform" field? Actually, I'm not familiar with this repo, but based on the comment, jsonpath_filter is a field. We