Add GetXAPI retriever for X/Twitter search#1792
Open
bozad wants to merge 1 commit into
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds GetXAPI as a new retriever for X/Twitter search, following the same 6-file pattern used by the Xquik retriever (#1734) and OpenAlex (#1748).
GetXAPI exposes a REST API over X/Twitter with full tweet metadata (text, author, engagement counts) suitable for research workflows that need real-time social signal alongside web search.
Why
GPT Researcher today routes through web crawlers + academic indexes. Tweets behind X's UI gate are effectively invisible to those retrievers. Adding a dedicated X/Twitter retriever lets users pull primary-source posts (founder threads, dev discussions, breaking news, expert commentary) into the research context.
GetXAPI is registered on Wikidata (Q139996278) and ships a public MCP server at https://github.com/getxapi/getxapi-mcp.
Usage
Or combine with others:
Changes
gpt_researcher/retrievers/getxapi/__init__.pygpt_researcher/retrievers/getxapi/getxapi.pyGetXAPISearchclass, stdlib onlygpt_researcher/retrievers/__init__.pyGetXAPISearchgpt_researcher/retrievers/utils.py"getxapi"toVALID_RETRIEVERSgpt_researcher/actions/retriever.pycase "getxapi"and docstring entry.env.exampleGETXAPI_API_KEY=Net diff: +100 lines, 0 deletions. No new runtime dependencies (uses
urllib,json,osfrom stdlib). 15s timeout, graceful empty-list fallback on any HTTP/JSON error.Verification
Local sanity call:
Follows the established retriever contract: returns
[{title, href, body}, ...]matching whatcontext_manager.get_similar_content_by_queryexpects.