Skip to content

Commit 53fad30

Browse files
Milvus-doc-botMilvus-doc-bot
Milvus-doc-bot
authored and
Milvus-doc-bot
committed
Release new docs to master
1 parent aeb60f0 commit 53fad30

File tree

5 files changed

+849
-1
lines changed

5 files changed

+849
-1
lines changed

‎v2.5.x/site/en/integrations/integration_with_mindsdb.md

+157
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,157 @@
1+
---
2+
id: integration_with_mindsdb.md
3+
summary: This tutorial demonstrates how to integrate Milvus with MindsDB, enabling you to leverage MindsDB's AI capabilities with Milvus's vector database functionality through SQL-like operations for managing and querying vector embeddings.
4+
title: Integrate Milvus with MindsDB
5+
---
6+
7+
# Integrate Milvus with MindsDB
8+
9+
[MindsDB](https://docs.mindsdb.com/what-is-mindsdb) is a powerful tool for integrating AI applications with diverse enterprise data sources. It acts as a federated query engine that brings order to data sprawl while meticulously answering queries across both structured and unstructured data. Whether your data is scattered across SaaS applications, databases, or data warehouses, MindsDB can connect and query it all using standard SQL. It features state-of-the-art autonomous RAG systems through Knowledge Bases, supports hundreds of data sources, and provides flexible deployment options from local development to cloud environments.
10+
11+
This tutorial demonstrates how to integrate Milvus with MindsDB, enabling you to leverage MindsDB's AI capabilities with Milvus's vector database functionality through SQL-like operations for managing and querying vector embeddings.
12+
13+
<div class="alert note">
14+
15+
This tutorial mainly refers to the official documentation of the [MindsDB Milvus Handler](https://github.com/mindsdb/mindsdb/tree/main/mindsdb/integrations/handlers/milvus_handler). If you find any outdated parts in this tutorial, you can prioritize following the official documentation and create an issue for us.
16+
17+
</div>
18+
19+
20+
## Install MindsDB
21+
22+
Before we start, install MindsDB locally via [Docker](https://docs.mindsdb.com/setup/self-hosted/docker) or [Docker Desktop](https://docs.mindsdb.com/setup/self-hosted/docker-desktop).
23+
24+
Before proceeding, ensure you have a solid understanding of the fundamental concepts and operations of both MindsDB and Milvus.
25+
26+
27+
## Arguments Introduction
28+
The required arguments to establish a connection are:
29+
30+
* `uri`: uri for milvus database, can be set to local ".db" file or docker or cloud service
31+
* `token`: token to support docker or cloud service according to uri option
32+
33+
The optional arguments to establish a connection are:
34+
35+
These are used for `SELECT` queries:
36+
* `search_default_limit`: default limit to be passed in select statements (default=100)
37+
* `search_metric_type`: metric type used for searches (default="L2")
38+
* `search_ignore_growing`: whether to ignore growing segments during similarity searches (default=False)
39+
* `search_params`: specific to the `search_metric_type` (default={"nprobe": 10})
40+
41+
These are used for `CREATE` queries:
42+
* `create_auto_id`: whether to auto generate id when inserting records with no ID (default=False)
43+
* `create_id_max_len`: maximum length of the id field when creating a table (default=64)
44+
* `create_embedding_dim`: embedding dimension for creating table (default=8)
45+
* `create_dynamic_field`: whether or not the created tables have dynamic fields or not (default=True)
46+
* `create_content_max_len`: max length of the content column (default=200)
47+
* `create_content_default_value`: default value of content column (default='')
48+
* `create_schema_description`: description of the created schemas (default='')
49+
* `create_alias`: alias of the created schemas (default='default')
50+
* `create_index_params`: parameters of the index created on embeddings column (default={})
51+
* `create_index_metric_type`: metric used to create the index (default='L2')
52+
* `create_index_type`: the type of index (default='AUTOINDEX')
53+
54+
55+
## Usage
56+
57+
Before continuing, make sure that `pymilvus` version is same as this [pinned version](https://github.com/mindsdb/mindsdb/blob/main/mindsdb/integrations/handlers/milvus_handler/requirements.txt). If you find any issues with version compatibility, you can roll back your version of pymilvus, or customize it in this [requirement file](https://github.com/mindsdb/mindsdb/tree/main/mindsdb/integrations/handlers/milvus_handler).
58+
59+
### Creating connection
60+
61+
In order to make use of this handler and connect to a Milvus server in MindsDB, the following syntax can be used:
62+
63+
```sql
64+
CREATE DATABASE milvus_datasource
65+
WITH
66+
ENGINE = 'milvus',
67+
PARAMETERS = {
68+
"uri": "./milvus_local.db",
69+
"token": "",
70+
"create_embedding_dim": 3,
71+
"create_auto_id": true
72+
};
73+
```
74+
75+
> - If you only need a local vector database for small scale data or prototyping, setting the uri as a local file, e.g.`./milvus.db`, is the most convenient method, as it automatically utilizes [Milvus Lite](https://milvus.io/docs/milvus_lite.md) to store all data in this file.
76+
> - For larger scale data and traffic in production, you can set up a Milvus server on [Docker or Kubernetes](https://milvus.io/docs/install-overview.md). In this setup, please use the server address and port as your `uri`, e.g.`http://localhost:19530`. If you enable the authentication feature on Milvus, set the `token` as `"<your_username>:<your_password>"`, otherwise there is no need to set the token.
77+
> - You can also use fully managed Milvus on [Zilliz Cloud](https://zilliz.com/cloud). Simply set the `uri` and `token` to the [Public Endpoint and API key](https://docs.zilliz.com/docs/on-zilliz-cloud-console#cluster-details) of your Zilliz Cloud instance.
78+
79+
80+
### Dropping connection
81+
82+
To drop the connection, use this command
83+
84+
```sql
85+
DROP DATABASE milvus_datasource;
86+
```
87+
88+
### Creating tables
89+
90+
To insert data from a pre-existing table, use `CREATE`
91+
92+
```sql
93+
CREATE TABLE milvus_datasource.test
94+
(SELECT * FROM sqlitedb.test);
95+
```
96+
97+
### Dropping collections
98+
99+
Dropping a collection is not supported
100+
101+
### Querying and selecting
102+
103+
To query database using a search vector, you can use `search_vector` in `WHERE` clause
104+
105+
Caveats:
106+
- If you omit `LIMIT`, the `search_default_limit` is used since Milvus requires it
107+
- Metadata column is not supported, but if the collection has dynamic schema enabled, you can query like normal, see the example below
108+
- Dynamic fields cannot be displayed but can be queried
109+
110+
```sql
111+
SELECT * from milvus_datasource.test
112+
WHERE search_vector = '[3.0, 1.0, 2.0, 4.5]'
113+
LIMIT 10;
114+
```
115+
116+
If you omit the `search_vector`, this becomes a basic search and `LIMIT` or `search_default_limit` amount of entries in collection are returned
117+
118+
```sql
119+
SELECT * from milvus_datasource.test
120+
```
121+
122+
You can use `WHERE` clause on dynamic fields like normal SQL
123+
124+
```sql
125+
SELECT * FROM milvus_datasource.createtest
126+
WHERE category = "science";
127+
```
128+
129+
### Deleting records
130+
131+
You can delete entries using `DELETE` just like in SQL.
132+
133+
Caveats:
134+
- Milvus only supports deleting entities with clearly specified primary keys
135+
- You can only use `IN` operator
136+
137+
```sql
138+
DELETE FROM milvus_datasource.test
139+
WHERE id IN (1, 2, 3);
140+
```
141+
142+
### Inserting records
143+
144+
You can also insert individual rows like so:
145+
146+
```sql
147+
INSERT INTO milvus_test.testable (id,content,metadata,embeddings)
148+
VALUES ("id3", 'this is a test', '{"test": "test"}', '[1.0, 8.0, 9.0]');
149+
```
150+
151+
### Updating
152+
153+
Updating records is not supported by Milvus API. You can try using combination of `DELETE` and `INSERT`
154+
155+
---
156+
157+
For more details and examples, please refer to the [MindsDB Official Documentation](https://docs.mindsdb.com/what-is-mindsdb).

‎v2.5.x/site/en/integrations/integrations_overview.md

+3-1
Original file line numberDiff line numberDiff line change
@@ -66,4 +66,6 @@ This page provides a list of tutorials for you to interact with Milvus and third
6666
| [Building RAG with Milvus and Firecrawl](build_RAG_with_milvus_and_firecrawl.md) | Data Sources | Milvus, Firecrawl |
6767
| [Build RAG with Llama Stack with Milvus](llama_stack_with_milvus.md) | Orchestration | Milvus, Llama Stack |
6868
| [RAG with Milvus and LlamaIndex Async API](llamaindex_milvus_async.md) | Ochestration | Milvus, LlamaIndex |
69-
69+
| [Integrate Milvus with MindsDB](integration_with_mindsdb.md) | Knowledge Engineering | Milvus, MindsDB |
70+
| [MCP + Milvus: Connecting AI with Vector Databases](milvus_and_mcp.md) | Agents | Milvus, MCP |
71+
| [Milvus Integration with OpenAI Agents: A Step-by-Step Guide](openai_agents_milvus.md) | Agents | Milvus, OpenAI |

0 commit comments

Comments
 (0)