llms.txt

# @zilliz/milvus2-sdk-node

> Official Node.js/TypeScript SDK for Milvus and Zilliz Cloud vector databases. Use it for semantic search, RAG retrieval, recommendation, hybrid dense+sparse retrieval, metadata filtering, and collection/index/user/resource management. Package version in this repo: `2.6.14`. Requires Node.js 18+.

## Install

```bash
npm install @zilliz/milvus2-sdk-node
# or
yarn add @zilliz/milvus2-sdk-node
```

## Client Choice

- Use `MilvusClient` for the full gRPC API and best performance in normal Node.js runtimes.
- Use `HttpClient` for HTTP/REST compatibility and environments where gRPC/raw TCP is unavailable or undesirable, such as Cloudflare Workers, Vercel Edge, and many serverless deployments.
- Zilliz Cloud commonly uses `token`; local Milvus can use no auth or `username`/`password`.

```typescript
import { MilvusClient, HttpClient } from '@zilliz/milvus2-sdk-node';

const grpcClient = new MilvusClient({ address: 'localhost:19530' });
await grpcClient.connectPromise;

const cloudClient = new MilvusClient({
  address: 'https://your-instance.zillizcloud.com',
  token: 'your-api-key',
});
await cloudClient.connectPromise;

const httpClient = new HttpClient({
  endpoint: 'localhost:19530',
  token: 'optional-token',
});
```

## Key Constraints and Pitfalls

- Call `loadCollection` after `createCollection` and before `search`/`query`.
- Dense vector dimensions must match the collection schema at insert and search time.
- `DataType.VarChar` requires `max_length`.
- `DataType.Array` requires `element_type` and `max_capacity`; VarChar array elements also need `max_length`.
- `DataType.SparseFloatVector` does not require `dim`.
- Int64 values beyond `Number.MAX_SAFE_INTEGER` (`2^53`) should be passed as strings.
- `search` performs vector ANN search and returns results ordered by score/distance; `query` performs scalar filtering without a vector.
- Use `filter` for scalar expressions. Some older APIs may also accept `expr`, but prefer `filter`.
- `autoID: true` means primary keys are generated by Milvus; do not provide that primary key in inserted rows.

## Compatibility

| Milvus version | Recommended SDK |
| --- | --- |
| v2.6.0+ | latest (`@zilliz/milvus2-sdk-node@latest`) |
| v2.5.0+ | `@zilliz/milvus2-sdk-node@2.5.12` |
| v2.4.0+ | `@zilliz/milvus2-sdk-node@2.4.9` |
| v2.3.0+ | `@zilliz/milvus2-sdk-node@2.3.5` |

The proto files in this repo track Milvus 3.0 development, but only use APIs exported by the TypeScript SDK in `milvus/` and documented here/README.

## Task: Basic Vector Search / RAG Retrieval

```typescript
import { MilvusClient, DataType, MetricType } from '@zilliz/milvus2-sdk-node';

const client = new MilvusClient({ address: 'localhost:19530' });
await client.connectPromise;

await client.createCollection({
  collection_name: 'documents',
  fields: [
    { name: 'id', data_type: DataType.Int64, is_primary_key: true, autoID: true },
    { name: 'vector', data_type: DataType.FloatVector, dim: 1536 },
    { name: 'text', data_type: DataType.VarChar, max_length: 4000 },
    { name: 'source', data_type: DataType.VarChar, max_length: 256 },
  ],
  index_params: [
    {
      field_name: 'vector',
      index_type: 'HNSW',
      metric_type: MetricType.COSINE,
      params: { M: 16, efConstruction: 256 },
    },
  ],
});

await client.loadCollection({ collection_name: 'documents' });

await client.insert({
  collection_name: 'documents',
  data: [
    { vector: embedding1, text: 'document content...', source: 'file1.pdf' },
    { vector: embedding2, text: 'another document...', source: 'file2.pdf' },
  ],
});

const res = await client.search({
  collection_name: 'documents',
  data: [queryEmbedding],
  limit: 5,
  output_fields: ['text', 'source'],
});

console.log(res.results); // [{ id, score, text, source }, ...]
```

## Task: Vector Search with Metadata Filter

```typescript
const res = await client.search({
  collection_name: 'products',
  data: [queryVector],
  limit: 20,
  filter: 'category == "electronics" AND price < 500 AND brand IN ["Apple", "Samsung"]',
  output_fields: ['name', 'price', 'category', 'brand'],
});
```

Filter expression syntax:

```text
age > 18
status == "active"
price <= 99.9
category IN ["books", "music"]
title LIKE "hello%"
ARRAY_CONTAINS(tags, "ai")
ARRAY_LENGTH(tags) > 3
metadata["key"] == "value"
metadata["nested"]["field"] > 10
NOT (status == "deleted")
```

Parameterized filters are supported:

```typescript
await client.query({
  collection_name: 'documents',
  filter: 'year >= {min_year}',
  filter_params: { min_year: 2024 },
  output_fields: ['text'],
});
```

## Task: Query, Get, Count, Delete

```typescript
const rows = await client.query({
  collection_name: 'documents',
  filter: 'source == "file1.pdf"',
  output_fields: ['text', 'source'],
  limit: 100,
});

const byIds = await client.get({
  collection_name: 'documents',
  ids: [1, 2, 3],
  output_fields: ['text'],
});

const count = await client.count({
  collection_name: 'documents',
  filter: 'source != ""',
});

await client.delete({ collection_name: 'documents', ids: [1, 2, 3] });
await client.delete({ collection_name: 'documents', filter: 'source == "old.pdf"' });
```

## Task: Upsert and Partial Array Update

Use `upsert` to insert or replace rows by primary key. For array fields, `field_ops` can do partial updates.

```typescript
import {
  DataType,
  FieldPartialUpdateOpType,
} from '@zilliz/milvus2-sdk-node';

await client.createCollection({
  collection_name: 'articles',
  fields: [
    { name: 'id', data_type: DataType.Int64, is_primary_key: true },
    { name: 'vector', data_type: DataType.FloatVector, dim: 128 },
    { name: 'tags', data_type: DataType.Array, element_type: DataType.VarChar, max_length: 64, max_capacity: 20 },
  ],
});

await client.upsert({
  collection_name: 'articles',
  data: [{ id: 1, tags: ['featured'] }],
  field_ops: [
    { field_name: 'tags', op: FieldPartialUpdateOpType.ARRAY_APPEND },
  ],
});
```

Valid partial array ops: `REPLACE`, `ARRAY_APPEND`, `ARRAY_REMOVE`.

## Task: Rich Schema (JSON, Array, Dynamic Fields)

```typescript
await client.createCollection({
  collection_name: 'articles',
  fields: [
    { name: 'id', data_type: DataType.Int64, is_primary_key: true, autoID: true },
    { name: 'title', data_type: DataType.VarChar, max_length: 512 },
    { name: 'embedding', data_type: DataType.FloatVector, dim: 768 },
    { name: 'tags', data_type: DataType.Array, element_type: DataType.VarChar, max_length: 64, max_capacity: 20 },
    { name: 'metadata', data_type: DataType.JSON },
  ],
  enable_dynamic_field: true,
  index_params: [
    {
      field_name: 'embedding',
      index_type: 'HNSW',
      metric_type: MetricType.COSINE,
      params: { M: 16, efConstruction: 256 },
    },
  ],
});

await client.loadCollection({ collection_name: 'articles' });

await client.insert({
  collection_name: 'articles',
  data: [
    {
      title: 'Milvus guide',
      embedding: Array(768).fill(0.1),
      tags: ['vector-db', 'rag'],
      metadata: { author: 'zilliz', year: 2026 },
      dynamic_extra_field: 'stored in dynamic field',
    },
  ],
});
```

## Task: Multi-Tenant Partition Key

Partition keys automatically route rows and filtered searches by tenant without manual partition management.

```typescript
await client.createCollection({
  collection_name: 'tenant_data',
  fields: [
    { name: 'id', data_type: DataType.Int64, is_primary_key: true, autoID: true },
    { name: 'tenant_id', data_type: DataType.VarChar, max_length: 64, is_partition_key: true },
    { name: 'vector', data_type: DataType.FloatVector, dim: 128 },
    { name: 'content', data_type: DataType.VarChar, max_length: 2000 },
  ],
  num_partitions: 16,
});

await client.loadCollection({ collection_name: 'tenant_data' });

await client.insert({
  collection_name: 'tenant_data',
  data: [{ tenant_id: 'user_123', vector: Array(128).fill(0.1), content: 'hello' }],
});

const res = await client.search({
  collection_name: 'tenant_data',
  data: [queryVector],
  filter: 'tenant_id == "user_123"',
  limit: 10,
  output_fields: ['content'],
});
```

## Task: Sparse Vector Search

Sparse vectors are objects mapping dimension index to weight.

```typescript
await client.createCollection({
  collection_name: 'sparse_docs',
  fields: [
    { name: 'id', data_type: DataType.Int64, is_primary_key: true, autoID: true },
    { name: 'sparse_vector', data_type: DataType.SparseFloatVector },
    { name: 'text', data_type: DataType.VarChar, max_length: 5000 },
  ],
  index_params: [
    {
      field_name: 'sparse_vector',
      index_type: 'SPARSE_INVERTED_INDEX',
      metric_type: MetricType.IP,
    },
  ],
});

await client.loadCollection({ collection_name: 'sparse_docs' });

await client.insert({
  collection_name: 'sparse_docs',
  data: [
    { sparse_vector: { 0: 0.5, 10: 0.3, 200: 0.8 }, text: 'machine learning' },
    { sparse_vector: { 1: 0.1, 50: 0.9 }, text: 'database systems' },
  ],
});

const res = await client.search({
  collection_name: 'sparse_docs',
  data: [{ 0: 0.5, 10: 0.3 }],
  anns_field: 'sparse_vector',
  limit: 10,
  output_fields: ['text'],
});
```

## Task: Full-Text Search / BM25 Functions

Milvus can generate sparse BM25 representations from text fields with functions. Define analyzer-enabled input text and a sparse output field.

```typescript
import { DataType, FunctionType, MetricType } from '@zilliz/milvus2-sdk-node';

await client.createCollection({
  collection_name: 'text_search',
  fields: [
    { name: 'id', data_type: DataType.Int64, is_primary_key: true, autoID: true },
    {
      name: 'text',
      data_type: DataType.VarChar,
      max_length: 5000,
      enable_analyzer: true,
    },
    { name: 'sparse', data_type: DataType.SparseFloatVector },
  ],
  functions: [
    {
      name: 'bm25_fn',
      type: FunctionType.BM25,
      input_field_names: ['text'],
      output_field_names: ['sparse'],
      params: {},
    },
  ],
  index_params: [
    {
      field_name: 'sparse',
      index_type: 'SPARSE_INVERTED_INDEX',
      metric_type: MetricType.BM25,
    },
  ],
});
```

## Task: Hybrid Search (Multiple Vector Requests + Rerank)

`hybridSearch` combines multiple ANN requests, commonly dense vectors plus sparse/BM25 vectors, then reranks with `WeightedRanker` or `RRFRanker`.

```typescript
import { WeightedRanker, RRFRanker } from '@zilliz/milvus2-sdk-node';

const res = await client.hybridSearch({
  collection_name: 'hybrid_docs',
  requests: [
    {
      anns_field: 'dense_vector',
      data: [denseQueryVector],
      limit: 50,
      params: { ef: 64 },
    },
    {
      anns_field: 'sparse_vector',
      data: [sparseQueryVector],
      limit: 50,
    },
  ],
  rerank: WeightedRanker([0.7, 0.3]), // or RRFRanker(60)
  limit: 10,
  output_fields: ['text', 'source'],
});
```

## Task: Iterators for Large Result Sets

```typescript
const queryIter = await client.queryIterator({
  collection_name: 'documents',
  filter: 'status == "active"',
  batchSize: 1000,
  limit: 100000,
  output_fields: ['text', 'source'],
});

for await (const batch of queryIter) {
  processBatch(batch);
}

const searchIter = await client.searchIterator({
  collection_name: 'documents',
  data: queryVector,
  batchSize: 100,
  limit: 5000,
  output_fields: ['text'],
});

for await (const batch of searchIter) {
  processBatch(batch);
}
```

## Task: Bulk Data Import

Use `BulkWriter` to generate Milvus-compatible JSON or Parquet files locally, then import files server-side from storage accessible by Milvus.

```typescript
import { BulkWriter, DataType } from '@zilliz/milvus2-sdk-node';

const writer = new BulkWriter({
  schema: {
    fields: [
      { name: 'id', data_type: DataType.Int64, is_primary_key: true },
      { name: 'vector', data_type: DataType.FloatVector, dim: 128 },
    ],
  },
  format: 'parquet', // or 'json'
  localPath: './bulk_data',
});

for (const row of largeDataset) {
  await writer.append(row);
}

const batchFiles = await writer.close();

const task = await client.bulkInsert({
  collection_name: 'my_collection',
  files: ['data.parquet'],
});

await client.getImportState({ taskId: task.task_id });
```

## Task: External Collections (gRPC)

External collections can map Milvus fields to external data and refresh external data sources. Use only when your Milvus deployment supports this feature.

```typescript
await client.createCollection({
  collection_name: 'external_docs',
  external_source: 's3://bucket/path/',
  external_spec: JSON.stringify({ format: 'parquet' }),
  fields: [
    { name: 'id', data_type: DataType.Int64, is_primary_key: true, external_field: 'doc_id' },
    { name: 'vector', data_type: DataType.FloatVector, dim: 768, external_field: 'embedding' },
    { name: 'text', data_type: DataType.VarChar, max_length: 4000, external_field: 'body' },
  ],
});

const refresh = await client.refreshExternalCollection({
  collection_name: 'external_docs',
});

await client.getRefreshExternalCollectionProgress({ job_id: refresh.job_id });
await client.listRefreshExternalCollectionJobs({ collection_name: 'external_docs' });
```

## Task: Snapshot APIs (gRPC)

Snapshot APIs are available for deployments that support Milvus snapshot management.

```typescript
await client.createSnapshot({ collection_name: 'documents', snapshot_name: 'snap_001' });
await client.listSnapshots({ collection_name: 'documents' });
await client.describeSnapshot({ collection_name: 'documents', snapshot_name: 'snap_001' });

const restore = await client.restoreSnapshot({
  collection_name: 'documents_restored',
  snapshot_name: 'snap_001',
});

await client.getRestoreSnapshotState({ job_id: restore.job_id });
await client.listRestoreSnapshotJobs();
await client.dropSnapshot({ collection_name: 'documents', snapshot_name: 'snap_001' });
```

Related methods include `pinSnapshotData` and `unpinSnapshotData`.

## Task: Collection and Index Management

```typescript
await client.hasCollection({ collection_name });
await client.describeCollection({ collection_name });
await client.batchDescribeCollections({ collection_names: [collection_name] });
await client.showCollections();
await client.loadCollection({ collection_name });
await client.releaseCollection({ collection_name });
await client.refreshLoad({ collection_name });
await client.renameCollection({ collection_name, new_collection_name });
await client.truncateCollection({ collection_name });
await client.dropCollection({ collection_name });

await client.alterCollectionProperties({ collection_name, properties: { 'collection.ttl.seconds': 3600 } });
await client.dropCollectionProperties({ collection_name, delete_keys: ['collection.ttl.seconds'] });
await client.alterCollectionFieldProperties({ collection_name, field_name: 'vector', properties: { 'mmap.enabled': true } });
await client.addCollectionFunction({ collection_name, functions });
await client.dropCollectionFunction({ collection_name, functions });

await client.createIndex({ collection_name, field_name: 'vector', index_type: 'HNSW', metric_type: 'COSINE' });
await client.describeIndex({ collection_name, field_name: 'vector' });
await client.listIndexes({ collection_name });
await client.getIndexState({ collection_name, field_name: 'vector' });
await client.dropIndex({ collection_name, field_name: 'vector' });
```

## Task: Database, Partition, Alias, RBAC, Resource Groups

```typescript
await client.createDatabase({ db_name: 'tenant_db' });
await client.listDatabases();
await client.describeDatabase({ db_name: 'tenant_db' });
await client.dropDatabase({ db_name: 'tenant_db' });

await client.createPartition({ collection_name, partition_name: 'p1' });
await client.listPartitions({ collection_name });
await client.loadPartitions({ collection_name, partition_names: ['p1'] });
await client.dropPartition({ collection_name, partition_name: 'p1' });

await client.createAlias({ collection_name, alias: 'current_docs' });
await client.describeAlias({ alias: 'current_docs' });
await client.dropAlias({ alias: 'current_docs' });

await client.createUser({ username: 'alice', password: 'secret' });
await client.createRole({ roleName: 'reader' });
await client.operateUserRole({ username: 'alice', roleName: 'reader', type: 'AddUserToRole' });

await client.createResourceGroup({ resource_group: 'rg1' });
await client.listResourceGroups();
await client.dropResourceGroup({ resource_group: 'rg1' });
```

## HTTP Client Notes

`HttpClient` supports REST-oriented collection/vector/index/partition/alias/import/user/role operations and shares a similar data-operation style with `MilvusClient`.

```typescript
const client = new HttpClient({
  endpoint: 'localhost:19530',
  username: 'root',
  password: 'milvus',
  database: 'default',
  timeout: 60000,
});

await client.createCollection({ collection_name: 'docs', dimension: 128, metric_type: 'COSINE' });
await client.insert({ collection_name: 'docs', data: [{ id: 1, vector: Array(128).fill(0.1) }] });
await client.search({ collection_name: 'docs', data: [Array(128).fill(0.1)], limit: 10 });
await client.query({ collection_name: 'docs', filter: 'id > 0', limit: 100 });
```

Some HTTP examples and types use camelCase REST fields (`collectionName`, `fieldName`, `metricType`) internally, while the SDK accepts the public SDK-style snake_case in many high-level methods. Prefer examples from `README.md` and `docs/src/content/docs/http-client.mdx` when targeting HTTP specifically.

## Quick Reference: Data Types

| Use case | DataType | Notes |
| --- | --- | --- |
| Primary key | `DataType.Int64`, `DataType.VarChar` | Set `is_primary_key: true` |
| Dense embedding | `DataType.FloatVector` | Set `dim` |
| Binary embedding | `DataType.BinaryVector` | Set `dim`; binary metrics apply |
| Half precision | `DataType.Float16Vector`, `DataType.BFloat16Vector` | Set `dim` |
| Quantized dense | `DataType.Int8Vector` | Set `dim` |
| Sparse embedding | `DataType.SparseFloatVector` | No `dim` |
| Text | `DataType.VarChar` | Set `max_length` |
| Metadata | `DataType.JSON` | Filter with `metadata["key"]` |
| Lists | `DataType.Array` | Set `element_type` + `max_capacity` |
| Numeric | `DataType.Int8`, `Int16`, `Int32`, `Int64`, `Float`, `Double` | Use string for unsafe Int64 |
| Boolean | `DataType.Bool` |  |
| Geo/time | `DataType.Geometry`, `DataType.Timestamptz` | Deployment/version dependent |
| Advanced nested/vector arrays | `DataType.Struct`, `DataType.ArrayOfVector` | Deployment/version dependent |

## Quick Reference: Metric Types

| Metric | Best for | Score meaning |
| --- | --- | --- |
| `MetricType.COSINE` | Normalized embeddings | Higher = more similar |
| `MetricType.L2` | Euclidean distance | Lower = more similar |
| `MetricType.IP` | Inner product / normalized vectors | Higher = more similar |
| `MetricType.BM25` | Sparse full-text relevance | Higher = more relevant |
| `MetricType.HAMMING`, `MetricType.JACCARD` | Binary vectors | Lower/metric-specific |
| `MetricType.TANIMOTO`, `SUBSTRUCTURE`, `SUPERSTRUCTURE`, `MHJACCARD` | Specialized similarity | Deployment/version dependent |
| `MetricType.MAX_SIM*`, `DTW_*` | Advanced multi-vector/sequence search | Deployment/version dependent |

## Quick Reference: Index Types

Common values: `AUTOINDEX`, `HNSW`, `FLAT`, `IVF_FLAT`, `IVF_SQ8`, `IVF_PQ`, `DISKANN`, `BIN_FLAT`, `BIN_IVF_FLAT`, `SPARSE_INVERTED_INDEX`, `SPARSE_WAND`.

## Error Handling

```typescript
import { ErrorCode } from '@zilliz/milvus2-sdk-node';

try {
  const res = await client.createCollection({ /* ... */ });
  if (res.error_code !== ErrorCode.SUCCESS) {
    throw new Error(res.reason);
  }
} catch (err) {
  console.error('Milvus transport or SDK error:', err);
}
```

## Full API Reference

See `README.md` for current method signatures and examples. Source lives under `milvus/`; generated docs live under `docs/`.