Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 9 additions & 5 deletions src/oss/javascript/integrations/vectorstores/clickhouse.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -9,12 +9,12 @@ description: "Integrate with the ClickHouse vector store using LangChain JavaScr
Only available on Node.js.
</Tip>

[ClickHouse](https://clickhouse.com/) is a robust and open-source columnar database that is used for handling analytical queries and efficient storage, ClickHouse is designed to provide a powerful combination of vector search and analytics.
[ClickHouse](https://clickhouse.com/) is an open-source columnar database for analytics that also supports vector search. For background on ClickHouse vector search (including approximate indexes), see [Exact and Approximate Vector Search](https://clickhouse.com/docs/engines/table-engines/mergetree-family/annindexes).

## Setup

1. Launch a ClickHouse cluster. Refer to the [ClickHouse Installation Guide](https://clickhouse.com/docs/en/getting-started/install/) for details.
2. After launching a ClickHouse cluster, retrieve the `Connection Details` from the cluster's `Actions` menu. You will need the host, port, username, and password.
1. Launch a ClickHouse cluster. Refer to the [ClickHouse installation guide](https://clickhouse.com/docs/getting-started/install/) for details.
2. After launching a ClickHouse cluster, retrieve the connection details. You will need the host, port, username, and password.
3. Install the required Node.js peer dependency for ClickHouse in your workspace.

You will need to install the following peer dependencies:
Expand Down Expand Up @@ -47,7 +47,9 @@ const vectorStore = await ClickHouseStore.fromTexts(
new OpenAIEmbeddings(),
{
host: process.env.CLICKHOUSE_HOST || "localhost",
port: process.env.CLICKHOUSE_PORT || 8443,
port: process.env.CLICKHOUSE_PORT
? Number.parseInt(process.env.CLICKHOUSE_PORT, 10)
: 8443,
username: process.env.CLICKHOUSE_USER || "username",
password: process.env.CLICKHOUSE_PASSWORD || "password",
database: process.env.CLICKHOUSE_DATABASE || "default",
Expand Down Expand Up @@ -81,7 +83,9 @@ const vectorStore = await ClickHouseStore.fromExistingIndex(
new OpenAIEmbeddings(),
{
host: process.env.CLICKHOUSE_HOST || "localhost",
port: process.env.CLICKHOUSE_PORT || 8443,
port: process.env.CLICKHOUSE_PORT
? Number.parseInt(process.env.CLICKHOUSE_PORT, 10)
: 8443,
username: process.env.CLICKHOUSE_USER || "username",
password: process.env.CLICKHOUSE_PASSWORD || "password",
database: process.env.CLICKHOUSE_DATABASE || "default",
Expand Down
12 changes: 4 additions & 8 deletions src/oss/javascript/integrations/vectorstores/singlestore.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -3,17 +3,13 @@ title: "SingleStore integration"
description: "Integrate with the SingleStore using LangChain JavaScript."
---

[SingleStoreDB](https://singlestore.com/) is a robust, high-performance distributed SQL database solution designed to excel in both [cloud](https://www.singlestore.com/cloud/) and on-premises environments. Boasting a versatile feature set, it offers seamless deployment options while delivering unparalleled performance.
[SingleStore](https://singlestore.com/) is a distributed SQL database for transactional and analytical workloads. You can run it in the [cloud](https://www.singlestore.com/cloud/) or on premises.

A standout feature of SingleStoreDB is its advanced support for vector storage and operations, making it an ideal choice for applications requiring intricate AI capabilities such as text similarity matching. With built-in vector functions like [dot_product](https://docs.singlestore.com/managed-service/en/reference/sql-reference/vector-functions/dot_product.html) and [euclidean_distance](https://docs.singlestore.com/managed-service/en/reference/sql-reference/vector-functions/euclidean_distance.html), SingleStoreDB empowers developers to implement sophisticated algorithms efficiently.
SingleStore supports vector storage and similarity search alongside SQL. It includes vector functions such as [dot_product](https://docs.singlestore.com/managed-service/en/reference/sql-reference/vector-functions/dot_product.html) and [euclidean_distance](https://docs.singlestore.com/managed-service/en/reference/sql-reference/vector-functions/euclidean_distance.html). For table design, indexing, and query patterns, see [working with vector data](https://docs.singlestore.com/managed-service/en/developer-resources/functional-extensions/working-with-vector-data.html) in the SingleStore documentation.

For developers keen on leveraging vector data within SingleStoreDB, a comprehensive tutorial is available, guiding them through the intricacies of [working with vector data](https://docs.singlestore.com/managed-service/en/developer-resources/functional-extensions/working-with-vector-data.html). This tutorial delves into the Vector Store within SingleStoreDB, showcasing its ability to facilitate searches based on vector similarity. Leveraging vector indexes, queries can be executed with remarkable speed, enabling swift retrieval of relevant data.
You can also combine vector search with [full-text indexing based on Lucene](https://docs.singlestore.com/cloud/developer-resources/functional-extensions/working-with-full-text-search/) and filter on document metadata. Depending on your workload, you can prefilter on text or vectors, or combine scores (for example, with a weighted sum).

Moreover, SingleStoreDB's Vector Store seamlessly integrates with [full-text indexing based on Lucene](https://docs.singlestore.com/cloud/developer-resources/functional-extensions/working-with-full-text-search/), enabling powerful text similarity searches. Users can filter search results based on selected fields of document metadata objects, enhancing query precision.

What sets SingleStoreDB apart is its ability to combine vector and full-text searches in various ways, offering flexibility and versatility. Whether prefiltering by text or vector similarity and selecting the most relevant data, or employing a weighted sum approach to compute a final similarity score, developers have multiple options at their disposal.

In essence, SingleStoreDB provides a comprehensive solution for managing and querying vector data, offering unparalleled performance and flexibility for AI-driven applications.
Use the following sections to connect SingleStore to LangChain.js.

<Tip>
**Compatibility**
Expand Down
4 changes: 2 additions & 2 deletions src/oss/python/integrations/providers/databricks.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,12 @@ title: "Databricks integrations"
description: "Integrate with Databricks using LangChain Python."
---

> [Databricks](https://www.databricks.com/) Intelligence Platform is the world's first data intelligence platform powered by generative AI. Infuse AI into every facet of your business.
> [Databricks](https://www.databricks.com/) provides a data and AI platform (often called the Databricks Lakehouse) for analytics, machine learning, and generative AI workloads.

Databricks embraces the LangChain ecosystem in various ways:

1. 🚀 **Model Serving** - Access state-of-the-art LLMs, such as DBRX, Llama3, Mixtral, or your fine-tuned models on [Databricks Model Serving](https://www.databricks.com/product/model-serving), via a highly available and low-latency inference endpoint. LangChain provides LLM (`Databricks`), Chat Model (`ChatDatabricks`), and Embeddings (`DatabricksEmbeddings`) implementations, streamlining the integration of your models hosted on Databricks Model Serving with your LangChain applications.
2. 📃 **Vector Search** - [Databricks Vector Search](https://www.databricks.com/product/machine-learning/vector-search) is a serverless vector database seamlessly integrated within the Databricks Platform. Using `DatabricksVectorSearch`, you can incorporate the highly scalable and reliable similarity search engine into your LangChain applications.
2. 📃 **Vector Search** - [Databricks Vector Search](https://www.databricks.com/product/machine-learning/vector-search) is a serverless vector database seamlessly integrated within the Databricks Platform. Use the `DatabricksVectorSearch` class to connect LangChain to vector search indexes in your Databricks account.
3. 📊 **MLflow** - [MLflow](https://mlflow.org/) is an open-source platform to manage full the ML lifecycle, including experiment management, evaluation, tracing, deployment, and more. [MLflow's LangChain Integration](/oss/integrations/providers/mlflow_tracking) streamlines the process of developing and operating modern compound ML systems.
4. 🌐 **SQL Database** - [Databricks SQL](https://www.databricks.com/product/databricks-sql) is integrated with `SQLDatabase` in LangChain, allowing you to access the auto-optimizing, exceptionally performant data warehouse.
5. 💡 **Open Models** - Databricks open sources models, such as [DBRX](https://www.databricks.com/blog/introducing-dbrx-new-state-art-open-llm), which are available through the [Hugging Face Hub](https://huggingface.co/databricks/dbrx-instruct). These models can be directly utilized with LangChain, leveraging its integration with the `transformers` library.
Expand Down
3 changes: 2 additions & 1 deletion src/oss/python/integrations/providers/datadog.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,8 @@ patch(langchain=True)
# patch(langchain=True, aiohttp=True)

# to include underlying OpenAI spans from the OpenAI integration
# patch(langchain=True, openai=True)patch_all
# patch(langchain=True, openai=True)
# patch_all()
```

See the [APM Python library documentation](https://ddtrace.readthedocs.io/en/stable/installation_quickstart.html) for more advanced usage.
Expand Down
19 changes: 9 additions & 10 deletions src/oss/python/integrations/vectorstores/clickhouse.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -3,16 +3,16 @@ title: "ClickHouse integration"
description: "Integrate with the ClickHouse vector store using LangChain Python."
---

> [ClickHouse](https://clickhouse.com/) is the fastest and most resource efficient open-source database for real-time apps and analytics with full SQL support and a wide range of functions to assist users in writing analytical queries. Lately added data structures and distance search functions (like `L2Distance`) as well as [approximate nearest neighbor search indexes](https://clickhouse.com/docs/en/engines/table-engines/mergetree-family/annindexes) enable ClickHouse to be used as a high performance and scalable vector database to store and search vectors with SQL.
> [ClickHouse](https://clickhouse.com/) is an open-source database for real-time apps and analytics with full SQL support. ClickHouse supports exact vector search (for example, using distance functions like `L2Distance`) and approximate vector search using vector similarity indexes (available in ClickHouse 25.8+). For details, see [Exact and Approximate Vector Search](https://clickhouse.com/docs/engines/table-engines/mergetree-family/annindexes).

This notebook shows how to use functionality related to the `ClickHouse` vector store.
This page shows how to use functionality related to the `ClickHouse` vector store.

## Setup

First set up a local clickhouse server with docker:

```python
! docker run -d -p 8123:8123 -p 9000:9000 --name langchain-clickhouse-server --ulimit nofile=262144:262144 -e CLICKHOUSE_SKIP_USER_SETUP=1 clickhouse/clickhouse-server:25.7
! docker run -d -p 8123:8123 -p 9000:9000 --name langchain-clickhouse-server --ulimit nofile=262144:262144 -e CLICKHOUSE_SKIP_USER_SETUP=1 clickhouse/clickhouse-server:26.2
```

You'll need to install `langchain-community` and `clickhouse-connect` to use this integration
Expand Down Expand Up @@ -153,9 +153,8 @@ Performing a simple similarity search can be done as follows:
results = vector_store.similarity_search(
"LangChain provides abstractions to make working with LLMs easy", k=2
)
for res in results:
page_content, metadata = res
print(f"* {page_content} [{metadata}]")
for doc in results:
print(f"* {doc.page_content} [{doc.metadata}]")
```

#### Similarity search with score
Expand All @@ -174,7 +173,7 @@ You can have direct access to ClickHouse SQL where statement. You can write `WHE

**NOTE**: Please be aware of SQL injection, this interface must not be directly called by end-user.

If you custimized your `column_map` under your setting, you search with filter like this:
If you customized your `column_map` in your settings, you can search with a filter like this:

```python
meta = vector_store.metadata_column
Expand All @@ -195,14 +194,14 @@ There are a variety of other search methods that are not covered in this noteboo

You can also transform the vector store into a retriever for easier usage in your chains.

Here is how to transform your vector store into a retriever and then invoke the retreiever with a simple query and filter.
Here is how to transform your vector store into a retriever and then invoke the retriever with a simple query and filter.

```python
retriever = vector_store.as_retriever(
search_type="similarity_score_threshold",
search_kwargs={"k": 1, "score_threshold": 0.5},
search_kwargs={"k": 1, "score_threshold": 0.5, "where_str": "metadata.source = 'news'"},
)
retriever.invoke("Stealing from the bank is a crime", filter={"source": "news"})
retriever.invoke("Stealing from the bank is a crime")
```

## Usage for retrieval-augmented generation
Expand Down
2 changes: 1 addition & 1 deletion src/oss/python/integrations/vectorstores/couchbase.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ title: "Couchbase integration"
description: "Integrate with the Couchbase vector store using LangChain Python."
---

[Couchbase](http://couchbase.com/) is an award-winning distributed NoSQL cloud database that delivers unmatched versatility, performance, scalability, and financial value for all of your cloud, mobile, AI, and edge computing applications. Couchbase embraces AI with coding assistance for developers and vector search for their applications.
[Couchbase](http://couchbase.com/) is a distributed NoSQL database for operational workloads across cloud, mobile, and edge deployments. It supports vector search for applications that need similarity search together with key-value and JSON document access.

Couchbase provides two different vector store implementations for LangChain:

Expand Down
12 changes: 4 additions & 8 deletions src/oss/python/integrations/vectorstores/singlestore.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -3,17 +3,13 @@ title: "SingleStoreVectorStore integration"
description: "Integrate with the SingleStoreVectorStore using LangChain Python."
---

>[SingleStore](https://singlestore.com/) is a robust, high-performance distributed SQL database solution designed to excel in both [cloud](https://www.singlestore.com/cloud/) and on-premises environments. Boasting a versatile feature set, it offers seamless deployment options while delivering unparalleled performance.
> [SingleStore](https://singlestore.com/) is a distributed SQL database for transactional and analytical workloads. You can run it in the [cloud](https://www.singlestore.com/cloud/) or on premises.

A standout feature of SingleStore is its advanced support for vector storage and operations, making it an ideal choice for applications requiring intricate AI capabilities such as text similarity matching. With built-in vector functions like [dot_product](https://docs.singlestore.com/managed-service/en/reference/sql-reference/vector-functions/dot_product.html) and [euclidean_distance](https://docs.singlestore.com/managed-service/en/reference/sql-reference/vector-functions/euclidean_distance.html), SingleStore empowers developers to implement sophisticated algorithms efficiently.
SingleStore supports vector storage and similarity search alongside SQL. It includes vector functions such as [dot_product](https://docs.singlestore.com/managed-service/en/reference/sql-reference/vector-functions/dot_product.html) and [euclidean_distance](https://docs.singlestore.com/managed-service/en/reference/sql-reference/vector-functions/euclidean_distance.html). For table design, indexing, and query patterns, see [working with vector data](https://docs.singlestore.com/managed-service/en/developer-resources/functional-extensions/working-with-vector-data.html) in the SingleStore documentation.

For developers keen on leveraging vector data within SingleStore, a comprehensive tutorial is available, guiding them through the intricacies of [working with vector data](https://docs.singlestore.com/managed-service/en/developer-resources/functional-extensions/working-with-vector-data.html). This tutorial delves into the Vector Store within SingleStoreDB, showcasing its ability to facilitate searches based on vector similarity. Leveraging vector indexes, queries can be executed with remarkable speed, enabling swift retrieval of relevant data.
You can also combine vector search with [full-text indexing based on Lucene](https://docs.singlestore.com/cloud/developer-resources/functional-extensions/working-with-full-text-search/) and filter on document metadata. Depending on your workload, you can prefilter on text or vectors, or combine scores (for example, with a weighted sum).

Moreover, SingleStore's Vector Store seamlessly integrates with [full-text indexing based on Lucene](https://docs.singlestore.com/cloud/developer-resources/functional-extensions/working-with-full-text-search/), enabling powerful text similarity searches. Users can filter search results based on selected fields of document metadata objects, enhancing query precision.

What sets SingleStore apart is its ability to combine vector and full-text searches in various ways, offering flexibility and versatility. Whether prefiltering by text or vector similarity and selecting the most relevant data, or employing a weighted sum approach to compute a final similarity score, developers have multiple options at their disposal.

In essence, SingleStore provides a comprehensive solution for managing and querying vector data, offering unparalleled performance and flexibility for AI-driven applications.
Use the following sections to connect SingleStore to LangChain.

| Class | Package | JS support |
| :--- | :--- | :---: |
Expand Down
8 changes: 4 additions & 4 deletions src/oss/python/integrations/vectorstores/vectara.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,11 @@ title: "Vectara integration"
description: "Integrate with the Vectara vector store using LangChain Python."
---

[Vectara](https://vectara.com/) is the trusted AI Assistant and Agent platform which focuses on enterprise readiness for mission-critical applications.
Vectara serverless RAG-as-a-service provides all the components of RAG behind an easy-to-use API, including:
[Vectara](https://vectara.com/) is a RAG platform that exposes indexing, retrieval, and related services through an API.
Vectara serverless RAG-as-a-service provides the main components of RAG behind an API, including:

1. A way to extract text from files (PDF, PPT, DOCX, etc)
2. ML-based chunking that provides state of the art performance.
2. ML-based chunking for text splits
3. The [Boomerang](https://vectara.com/how-boomerang-takes-retrieval-augmented-generation-to-the-next-level-via-grounded-generation/) embeddings model.
4. Its own internal vector database where text chunks and embedding vectors are stored.
5. A query service that automatically encodes the query into embedding, and retrieves the most relevant text segments, including support for [Hybrid Search](https://docs.vectara.com/docs/api-reference/search-apis/lexical-matching) as well as multiple reranking options such as the [multi-lingual relevance reranker](https://www.vectara.com/blog/deep-dive-into-vectara-multilingual-reranker-v1-state-of-the-art-reranker-across-100-languages), [MMR](https://vectara.com/get-diverse-results-and-comprehensive-summaries-with-vectaras-mmr-reranker/), [UDF reranker](https://www.vectara.com/blog/rag-with-user-defined-functions-based-reranking).
Expand All @@ -19,7 +19,7 @@ For more information:
- [API Playground](https://docs.vectara.com/docs/rest-api/)
- [Quickstart](https://docs.vectara.com/docs/quickstart)

This notebook shows how to use the basic retrieval functionality, when utilizing Vectara just as a Vector Store (without summarization), incuding: `similarity_search` and `similarity_search_with_score` as well as using the LangChain `as_retriever` functionality.
This notebook shows how to use the basic retrieval functionality when you use Vectara only as a vector store (without summarization), including: `similarity_search` and `similarity_search_with_score`, and the LangChain `as_retriever` functionality.

## Setup

Expand Down
Loading
Loading