Skip to content

singlestore.mdx

Latest commit

 

History

History
206 lines (172 loc) · 8.03 KB

singlestore.mdx

File metadata and controls

206 lines (172 loc) · 8.03 KB
title SingleStore integration
description Integrate with the SingleStore using LangChain JavaScript.

SingleStore is a distributed SQL database for transactional and analytical workloads. You can run it in the cloud or on premises.

SingleStore supports vector storage and similarity search alongside SQL. It includes vector functions such as dot_product and euclidean_distance. For table design, indexing, and query patterns, see working with vector data in the SingleStore documentation.

You can also combine vector search with full-text indexing based on Lucene and filter on document metadata. Depending on your workload, you can prefilter on text or vectors, or combine scores (for example, with a weighted sum).

Use the following sections to connect SingleStore to LangChain.js.

**Compatibility**

Only available on Node.js.

LangChain.js requires the mysql2 library to create a connection to a SingleStoreDB instance.

Setup

  1. Establish a SingleStoreDB environment. You have the flexibility to choose between Cloud-based or On-Premise editions.
  2. Install the mysql2 JS client
npm install -S mysql2

Usage

SingleStoreVectorStore manages a connection pool. It is recommended to call await store.end(); before terminating your application to assure all connections are appropriately closed and prevent any possible resource leaks.

Standard usage

See [this section for general instructions on installing LangChain packages](/oss/langchain/install). 
npm install @langchain/openai @langchain/community @langchain/core
import { SingleStoreVectorStore } from "@langchain/community/vectorstores/singlestore";
import { OpenAIEmbeddings } from "@langchain/openai";

export const run = async () => {
  const vectorStore = await SingleStoreVectorStore.fromTexts(
    ["Hello world", "Bye bye", "hello nice world"],
    [{ id: 2 }, { id: 1 }, { id: 3 }],
    new OpenAIEmbeddings(),
    {
      connectionOptions: {
        host: process.env.SINGLESTORE_HOST,
        port: Number(process.env.SINGLESTORE_PORT),
        user: process.env.SINGLESTORE_USERNAME,
        password: process.env.SINGLESTORE_PASSWORD,
        database: process.env.SINGLESTORE_DATABASE,
      },
    }
  );

  const resultOne = await vectorStore.similaritySearch("hello world", 1);
  console.log(resultOne);
  await vectorStore.end();
};

Metadata filtering

import { SingleStoreVectorStore } from "@langchain/community/vectorstores/singlestore";
import { OpenAIEmbeddings } from "@langchain/openai";

export const run = async () => {
  const vectorStore = await SingleStoreVectorStore.fromTexts(
    ["Good afternoon", "Bye bye", "Boa tarde!", "Até logo!"],
    [
      { id: 1, language: "English" },
      { id: 2, language: "English" },
      { id: 3, language: "Portuguese" },
      { id: 4, language: "Portuguese" },
    ],
    new OpenAIEmbeddings(),
    {
      connectionOptions: {
        host: process.env.SINGLESTORE_HOST,
        port: Number(process.env.SINGLESTORE_PORT),
        user: process.env.SINGLESTORE_USERNAME,
        password: process.env.SINGLESTORE_PASSWORD,
        database: process.env.SINGLESTORE_DATABASE,
      },
      distanceMetric: "EUCLIDEAN_DISTANCE",
    }
  );

  const resultOne = await vectorStore.similaritySearch("greetings", 1, {
    language: "Portuguese",
  });
  console.log(resultOne);
  await vectorStore.end();
};

Vector indexes

Enhance your search efficiency with SingleStore DB version 8.5 or above by leveraging ANN vector indexes. By setting useVectorIndex: true during vector store object creation, you can activate this feature. Additionally, if your vectors differ in dimensionality from the default OpenAI embedding size of 1536, ensure to specify the vectorSize parameter accordingly.

Hybrid search

import { SingleStoreVectorStore } from "@langchain/community/vectorstores/singlestore";
import { OpenAIEmbeddings } from "@langchain/openai";

export const run = async () => {
  const vectorStore = await SingleStoreVectorStore.fromTexts(
    [
      "In the parched desert, a sudden rainstorm brought relief, as the droplets danced upon the thirsty earth, rejuvenating the landscape with the sweet scent of petrichor.",
      "Amidst the bustling cityscape, the rain fell relentlessly, creating a symphony of pitter-patter on the pavement, while umbrellas bloomed like colorful flowers in a sea of gray.",
      "High in the mountains, the rain transformed into a delicate mist, enveloping the peaks in a mystical veil, where each droplet seemed to whisper secrets to the ancient rocks below.",
      "Blanketing the countryside in a soft, pristine layer, the snowfall painted a serene tableau, muffling the world in a tranquil hush as delicate flakes settled upon the branches of trees like nature's own lacework.",
      "In the urban landscape, snow descended, transforming bustling streets into a winter wonderland, where the laughter of children echoed amidst the flurry of snowballs and the twinkle of holiday lights.",
      "Atop the rugged peaks, snow fell with an unyielding intensity, sculpting the landscape into a pristine alpine paradise, where the frozen crystals shimmered under the moonlight, casting a spell of enchantment over the wilderness below.",
    ],
    [
      { category: "rain" },
      { category: "rain" },
      { category: "rain" },
      { category: "snow" },
      { category: "snow" },
      { category: "snow" },
    ],
    new OpenAIEmbeddings(),
    {
      connectionOptions: {
        host: process.env.SINGLESTORE_HOST,
        port: Number(process.env.SINGLESTORE_PORT),
        user: process.env.SINGLESTORE_USERNAME,
        password: process.env.SINGLESTORE_PASSWORD,
        database: process.env.SINGLESTORE_DATABASE,
      },
      distanceMetric: "DOT_PRODUCT",
      useVectorIndex: true,
      useFullTextIndex: true,
    }
  );

  const resultOne = await vectorStore.similaritySearch(
    "rainstorm in parched desert, rain",
    1,
    { category: "rain" }
  );
  console.log(resultOne[0].pageContent);

  await vectorStore.setSearchConfig({
    searchStrategy: "TEXT_ONLY",
  });
  const resultTwo = await vectorStore.similaritySearch(
    "rainstorm in parched desert, rain",
    1
  );
  console.log(resultTwo[0].pageContent);

  await vectorStore.setSearchConfig({
    searchStrategy: "FILTER_BY_TEXT",
    filterThreshold: 0.1,
  });
  const resultThree = await vectorStore.similaritySearch(
    "rainstorm in parched desert, rain",
    1
  );
  console.log(resultThree[0].pageContent);

  await vectorStore.setSearchConfig({
    searchStrategy: "FILTER_BY_VECTOR",
    filterThreshold: 0.1,
  });
  const resultFour = await vectorStore.similaritySearch(
    "rainstorm in parched desert, rain",
    1
  );
  console.log(resultFour[0].pageContent);

  await vectorStore.setSearchConfig({
    searchStrategy: "WEIGHTED_SUM",
    textWeight: 0.2,
    vectorWeight: 0.8,
    vectorselectCountMultiplier: 10,
  });
  const resultFive = await vectorStore.similaritySearch(
    "rainstorm in parched desert, rain",
    1
  );
  console.log(resultFive[0].pageContent);

  await vectorStore.end();
};

Related