Skip to content

Commit 7ec4dbe

Browse files
SauhaardWyuvalkamaniAryan ThakurManas1818aryan-thakur
authored
langchain[minor]: Add StackExchange API integration (#14002)
Implements [#12115](#12115) Who can review? @baskaryan , @eyurtsev , @hwchase17 Integrated Stack Exchange API into Langchain, enabling access to diverse communities within the platform. This addition enhances Langchain's capabilities by allowing users to query Stack Exchange for specialized information and engage in discussions. The integration provides seamless interaction with Stack Exchange content, offering content from varied knowledge repositories. A notebook example and test cases were included to demonstrate the functionality and reliability of this integration. - Add StackExchange as a tool. - Add unit test for the StackExchange wrapper and tool. - Add documentation for the StackExchange wrapper and tool. If you have time, could you please review the code and provide any feedback as necessary! My team is welcome to any suggestions. --------- Co-authored-by: Yuval Kamani <yuvalkamani@gmail.com> Co-authored-by: Aryan Thakur <aryanthakur@Aryans-MacBook-Pro.local> Co-authored-by: Manas1818 <79381912+manas1818@users.noreply.github.com> Co-authored-by: aryan-thakur <61063777+aryan-thakur@users.noreply.github.com> Co-authored-by: Bagatur <baskaryan@gmail.com>
1 parent d4405bc commit 7ec4dbe

12 files changed

Lines changed: 258 additions & 0 deletions

File tree

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
# Stack Exchange
2+
3+
>[Stack Exchange](https://en.wikipedia.org/wiki/Stack_Exchange) is a network of
4+
question-and-answer (Q&A) websites on topics in diverse fields, each site covering
5+
a specific topic, where questions, answers, and users are subject to a reputation award process.
6+
7+
This page covers how to use the `Stack Exchange API` within LangChain.
8+
9+
## Installation and Setup
10+
- Install requirements with
11+
```bash
12+
pip install stackapi
13+
```
14+
15+
## Wrappers
16+
17+
### Utility
18+
19+
There exists a StackExchangeAPIWrapper utility which wraps this API. To import this utility:
20+
21+
```python
22+
from langchain.utilities import StackExchangeAPIWrapper
23+
```
24+
25+
For a more detailed walkthrough of this wrapper, see [this notebook](/docs/integrations/tools/stackexchange).
26+
27+
### Tool
28+
29+
You can also easily load this wrapper as a Tool (to use with an Agent).
30+
You can do this with:
31+
```python
32+
from langchain.agents import load_tools
33+
tools = load_tools(["stackexchange"])
34+
```
35+
36+
For more information on tools, see [this page](/docs/modules/agents/tools/).
Lines changed: 74 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,74 @@
1+
{
2+
"cells": [
3+
{
4+
"cell_type": "markdown",
5+
"metadata": {},
6+
"source": [
7+
"# StackExchange\n",
8+
"\n",
9+
"This notebook goes over how to use the stack exchange component.\n",
10+
"\n",
11+
"All you need to do is install stackapi:\n",
12+
"1. pip install stackapi\n",
13+
"\n"
14+
]
15+
},
16+
{
17+
"cell_type": "code",
18+
"execution_count": null,
19+
"metadata": {},
20+
"outputs": [],
21+
"source": [
22+
"pip install stackapi"
23+
]
24+
},
25+
{
26+
"cell_type": "code",
27+
"execution_count": null,
28+
"metadata": {},
29+
"outputs": [],
30+
"source": [
31+
"from langchain.utilities import StackExchangeAPIWrapper"
32+
]
33+
},
34+
{
35+
"cell_type": "code",
36+
"execution_count": null,
37+
"metadata": {},
38+
"outputs": [],
39+
"source": [
40+
"stackexchange = StackExchangeAPIWrapper()"
41+
]
42+
},
43+
{
44+
"cell_type": "code",
45+
"execution_count": null,
46+
"metadata": {},
47+
"outputs": [],
48+
"source": [
49+
"stackexchange.run(\"zsh: command not found: python\")"
50+
]
51+
}
52+
],
53+
"metadata": {
54+
"kernelspec": {
55+
"display_name": "Python 3",
56+
"language": "python",
57+
"name": "python3"
58+
},
59+
"language_info": {
60+
"codemirror_mode": {
61+
"name": "ipython",
62+
"version": 3
63+
},
64+
"file_extension": ".py",
65+
"mimetype": "text/x-python",
66+
"name": "python",
67+
"nbconvert_exporter": "python",
68+
"pygments_lexer": "ipython3",
69+
"version": "3.8.8"
70+
}
71+
},
72+
"nbformat": 4,
73+
"nbformat_minor": 2
74+
}

libs/langchain/langchain/agents/load_tools.py

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -53,6 +53,7 @@
5353
from langchain.tools.searx_search.tool import SearxSearchResults, SearxSearchRun
5454
from langchain.tools.shell.tool import ShellTool
5555
from langchain.tools.sleep.tool import SleepTool
56+
from langchain.tools.stackexchange.tool import StackExchangeTool
5657
from langchain.tools.wikipedia.tool import WikipediaQueryRun
5758
from langchain.tools.wolfram_alpha.tool import WolframAlphaQueryRun
5859
from langchain.tools.openweathermap.tool import OpenWeatherMapQueryRun
@@ -73,6 +74,7 @@
7374
from langchain.utilities.searchapi import SearchApiAPIWrapper
7475
from langchain.utilities.searx_search import SearxSearchWrapper
7576
from langchain.utilities.serpapi import SerpAPIWrapper
77+
from langchain.utilities.stackexchange import StackExchangeAPIWrapper
7678
from langchain.utilities.twilio import TwilioAPIWrapper
7779
from langchain.utilities.wikipedia import WikipediaAPIWrapper
7880
from langchain.utilities.wolfram_alpha import WolframAlphaAPIWrapper
@@ -269,6 +271,10 @@ def _get_serpapi(**kwargs: Any) -> BaseTool:
269271
)
270272

271273

274+
def _get_stackexchange(**kwargs: Any) -> BaseTool:
275+
return StackExchangeTool(api_wrapper=StackExchangeAPIWrapper(**kwargs))
276+
277+
272278
def _get_dalle_image_generator(**kwargs: Any) -> Tool:
273279
return Tool(
274280
"Dall-E-Image-Generator",
@@ -397,6 +403,7 @@ def _get_google_cloud_texttospeech(**kwargs: Any) -> BaseTool:
397403
_get_lambda_api,
398404
["awslambda_tool_name", "awslambda_tool_description", "function_name"],
399405
),
406+
"stackexchange": (_get_stackexchange, []),
400407
"sceneXplain": (_get_scenexplain, []),
401408
"graphql": (_get_graphql_tool, ["graphql_endpoint"]),
402409
"openweathermap-api": (_get_openweathermap, ["openweathermap_api_key"]),

libs/langchain/langchain/tools/__init__.py

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -612,6 +612,12 @@ def _import_sql_database_tool_QuerySQLDataBaseTool() -> Any:
612612
return QuerySQLDataBaseTool
613613

614614

615+
def _import_stackexchange_tool() -> Any:
616+
from langchain.tools.stackexchange.tool import StackExchangeTool
617+
618+
return StackExchangeTool
619+
620+
615621
def _import_steamship_image_generation() -> Any:
616622
from langchain.tools.steamship_image_generation import SteamshipImageGenerationTool
617623

@@ -871,6 +877,8 @@ def __getattr__(name: str) -> Any:
871877
return _import_sql_database_tool_QuerySQLCheckerTool()
872878
elif name == "QuerySQLDataBaseTool":
873879
return _import_sql_database_tool_QuerySQLDataBaseTool()
880+
elif name == "StackExchangeTool":
881+
return _import_stackexchange_tool()
874882
elif name == "SteamshipImageGenerationTool":
875883
return _import_steamship_image_generation()
876884
elif name == "VectorStoreQATool":
@@ -992,6 +1000,7 @@ def __getattr__(name: str) -> Any:
9921000
"ShellTool",
9931001
"SleepTool",
9941002
"StdInInquireTool",
1003+
"StackExchangeTool",
9951004
"SteamshipImageGenerationTool",
9961005
"StructuredTool",
9971006
"Tool",
Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
"""StackExchange API toolkit."""
Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
"""Tool for the Wikipedia API."""
2+
3+
from typing import Optional
4+
5+
from langchain.callbacks.manager import CallbackManagerForToolRun
6+
from langchain.tools.base import BaseTool
7+
from langchain.utilities.stackexchange import StackExchangeAPIWrapper
8+
9+
10+
class StackExchangeTool(BaseTool):
11+
"""Tool that uses StackExchange"""
12+
13+
name: str = "StackExchange"
14+
description: str = (
15+
"A wrapper around StackExchange. "
16+
"Useful for when you need to answer specific programming questions"
17+
"code excerpts, code examples and solutions"
18+
"Input should be a fully formed question."
19+
)
20+
api_wrapper: StackExchangeAPIWrapper
21+
22+
def _run(
23+
self,
24+
query: str,
25+
run_manager: Optional[CallbackManagerForToolRun] = None,
26+
) -> str:
27+
"""Use the Stack Exchange tool."""
28+
return self.api_wrapper.run(query)

libs/langchain/langchain/utilities/__init__.py

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -188,6 +188,12 @@ def _import_sql_database() -> Any:
188188
return SQLDatabase
189189

190190

191+
def _import_stackexchange() -> Any:
192+
from langchain.utilities.stackexchange import StackExchangeAPIWrapper
193+
194+
return StackExchangeAPIWrapper
195+
196+
191197
def _import_tensorflow_datasets() -> Any:
192198
from langchain.utilities.tensorflow_datasets import TensorflowDatasets
193199

@@ -277,6 +283,8 @@ def __getattr__(name: str) -> Any:
277283
return _import_serpapi()
278284
elif name == "SparkSQL":
279285
return _import_spark_sql()
286+
elif name == "StackExchangeAPIWrapper":
287+
return _import_stackexchange()
280288
elif name == "SQLDatabase":
281289
return _import_sql_database()
282290
elif name == "TensorflowDatasets":
@@ -326,6 +334,7 @@ def __getattr__(name: str) -> Any:
326334
"SearxSearchWrapper",
327335
"SerpAPIWrapper",
328336
"SparkSQL",
337+
"StackExchangeAPIWrapper",
329338
"TensorflowDatasets",
330339
"TextRequestsWrapper",
331340
"TwilioAPIWrapper",
Lines changed: 68 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,68 @@
1+
import html
2+
from typing import Any, Dict, Literal
3+
4+
from langchain.pydantic_v1 import BaseModel, Field, root_validator
5+
6+
7+
class StackExchangeAPIWrapper(BaseModel):
8+
"""Wrapper for Stack Exchange API."""
9+
10+
client: Any #: :meta private:
11+
max_results: int = 3
12+
"""Max number of results to include in output."""
13+
query_type: Literal["all", "title", "body"] = "all"
14+
"""Which part of StackOverflows items to match against. One of 'all', 'title',
15+
'body'. Defaults to 'all'.
16+
"""
17+
fetch_params: Dict[str, Any] = Field(default_factory=dict)
18+
"""Additional params to pass to StackApi.fetch."""
19+
result_separator: str = "\n\n"
20+
"""Separator between question,answer pairs."""
21+
22+
@root_validator()
23+
def validate_environment(cls, values: Dict) -> Dict:
24+
"""Validate that the required Python package exists."""
25+
try:
26+
from stackapi import StackAPI
27+
28+
values["client"] = StackAPI("stackoverflow")
29+
except ImportError:
30+
raise ImportError(
31+
"The 'stackapi' Python package is not installed. "
32+
"Please install it with `pip install stackapi`."
33+
)
34+
return values
35+
36+
def run(self, query: str) -> str:
37+
"""Run query through StackExchange API and parse results."""
38+
39+
query_key = "q" if self.query_type == "all" else self.query_type
40+
output = self.client.fetch(
41+
"search/excerpts", **{query_key: query}, **self.fetch_params
42+
)
43+
if len(output["items"]) < 1:
44+
return f"No relevant results found for '{query}' on Stack Overflow."
45+
questions = [
46+
item for item in output["items"] if item["item_type"] == "question"
47+
][: self.max_results]
48+
answers = [item for item in output["items"] if item["item_type"] == "answer"]
49+
results = []
50+
for question in questions:
51+
res_text = f"Question: {question['title']}\n{question['excerpt']}"
52+
relevant_answers = [
53+
answer
54+
for answer in answers
55+
if answer["question_id"] == question["question_id"]
56+
]
57+
accepted_answers = [
58+
answer for answer in relevant_answers if answer["is_accepted"]
59+
]
60+
if relevant_answers:
61+
top_answer = (
62+
accepted_answers[0] if accepted_answers else relevant_answers[0]
63+
)
64+
excerpt = html.unescape(top_answer["excerpt"])
65+
res_text += f"\nAnswer: {excerpt}"
66+
results.append(res_text)
67+
68+
return self.result_separator.join(results)
Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
"""Integration test for Stack Exchange."""
2+
from langchain.utilities import StackExchangeAPIWrapper
3+
4+
5+
def test_call() -> None:
6+
"""Test that call runs."""
7+
stackexchange = StackExchangeAPIWrapper()
8+
output = stackexchange.run("zsh: command not found: python")
9+
assert output != "hello"
10+
11+
12+
def test_failure() -> None:
13+
"""Test that call that doesn't run."""
14+
stackexchange = StackExchangeAPIWrapper()
15+
output = stackexchange.run("sjefbsmnf")
16+
assert output == "No relevant results found for 'sjefbsmnf' on Stack Overflow"
17+
18+
19+
def test_success() -> None:
20+
"""Test that call that doesn't run."""
21+
stackexchange = StackExchangeAPIWrapper()
22+
output = stackexchange.run("zsh: command not found: python")
23+
assert "zsh: command not found: python" in output

libs/langchain/tests/unit_tests/tools/test_imports.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -94,6 +94,7 @@
9494
"SearxSearchRun",
9595
"ShellTool",
9696
"SleepTool",
97+
"StackExchangeTool",
9798
"StdInInquireTool",
9899
"SteamshipImageGenerationTool",
99100
"StructuredTool",

0 commit comments

Comments
 (0)