Skip to content

README.md

Latest commit

 

History

History
276 lines (220 loc) · 12.1 KB

README.md

File metadata and controls

276 lines (220 loc) · 12.1 KB
Raw

😈 ZBSearch

ZBSearch (zee bee search) is a zero-bs fork of Orama maintained by the original Orama team.

After Michele's departure, the entire engineering team left Orama and reassembled to maintain this popular project with no external influence. No VC, no business incentives, no bs. Just open source software. Help us get 10.5k stars back!

Highlighted features

Installation

You can install ZBSearch using npm, yarn, pnpm, bun:

npm i zbsearch

Or import it directly in a browser module:

<html>
  <body>
    <script type="module">
      import { create, insert, search } from 'https://cdn.jsdelivr.net/npm/zbsearch@latest/+esm'
    </script>
  </body>
</html>

With Deno, you can just use the same CDN URL or use npm specifiers:

import { create, search, insert } from 'npm:zbsearch'

Read the complete documentation at https://zbsearch.dev.

Usage

ZBSearch is quite simple to use. The first thing to do is to create a new database instance and set an indexing schema:

import { create, insert, remove, search, searchVector } from 'zbsearch'

const db = create({
  schema: {
    name: 'string',
    description: 'string',
    price: 'number',
    embedding: 'vector[1536]', // Vector size must be expressed during schema initialization
    meta: {
      rating: 'number',
    },
  },
})

insert(db, {
  name: 'Noise cancelling headphones',
  description: 'Best noise cancelling headphones on the market',
  price: 99.99,
  embedding: [0.2432, 0.9431, 0.5322, 0.4234, ...],
  meta: {
    rating: 4.5
  }
})

const results = search(db, {
  term: 'Best headphones'
})

// {
//   elapsed: {
//     raw: 21492,
//     formatted: '21μs',
//   },
//   hits: [
//     {
//       id: '41013877-56',
//       score: 0.925085832971998432,
//       document: {
//         name: 'Noise cancelling headphones',
//         description: 'Best noise cancelling headphones on the market',
//         price: 99.99,
//         embedding: [0.2432, 0.9431, 0.5322, 0.4234, ...],
//         meta: {
//           rating: 4.5
//         }
//       }
//     }
//   ],
//   count: 1
// }

ZBSearch currently supports 10 different data types:

Type Description Example
string A string of characters. 'Hello world'
number A numeric value, either float or integer. 42
boolean A boolean value. true
enum An enum value. 'drama'
geopoint A geopoint value. { lat: 40.7128, lon: 74.0060 }
string[] An array of strings. ['red', 'green', 'blue']
number[] An array of numbers. [42, 91, 28.5]
boolean[] An array of booleans. [true, false, false]
enum[] An array of enums. ['comedy', 'action', 'romance']
vector[<size>] A vector of numbers to perform vector search on. [0.403, 0.192, 0.830]

Schema inference (optional schema)

The schema is optional. If you omit it, ZBSearch infers the type of every document property on first sight and indexes it on the fly:

const db = create()

insert(db, { name: 'Noise cancelling headphones', price: 99.99, meta: { rating: 4.5 } })
// db.schema is now: { name: 'string', price: 'number', meta: { rating: 'number' } }

Types lock on first sight (a later conflicting value is rejected with SCHEMA_VALIDATION_FAILURE), and objects shaped like { lat, lon } are inferred as geopoint. Vectors are the only types that are never inferred - a number[] is indistinguishable from an embedding — so embedding properties must still be declared, and they can be the only thing you declare:

const db = create({
  schema: { embedding: 'vector[1536]' },
  inferSchema: true, // a provided schema is strict by default; this opts undeclared fields into inference
})

Passing a schema keeps the classic strict behavior (undeclared fields are stored but not indexed). See the official docs for the full inference rules.

Vector and Hybrid Search Support

ZBSearch supports both vector and hybrid search by just setting mode: 'vector' when performing search.

To perform this kind of search, you'll need to provide text embeddings at search time:

import { create, insertMultiple, search } from 'zbsearch'

const db = create({
  schema: {
    title: 'string',
    embedding: 'vector[5]', // we are using a 5-dimensional vector.
  },
});

insertMultiple(db, [
  { title: 'The Prestige', embedding: [0.938293, 0.284951, 0.348264, 0.948276, 0.56472] },
  { title: 'Barbie', embedding: [0.192839, 0.028471, 0.284738, 0.937463, 0.092827] },
  { title: 'Oppenheimer', embedding: [0.827391, 0.927381, 0.001982, 0.983821, 0.294841] },
])

const results = search(db, {
  // Search mode. Can be 'vector', 'hybrid', or 'fulltext'
  mode: 'vector',
  vector: {
    // The vector (text embedding) to use for search
    value: [0.938292, 0.284961, 0.248264, 0.748276, 0.26472],
    // The schema property where ZBSearch should compare embeddings
    property: 'embedding',
  },
  // Minimum similarity to determine a match. Defaults to `0.8`
  similarity: 0.85,
  // Defaults to `false`. Setting to 'true' will return the embeddings in the response (which can be very large).
  includeVectors: true,
})

Have trouble generating embeddings for vector and hybrid search? Try our zbsearch/plugin-embeddings plugin!

import { create } from 'zbsearch'
import { pluginEmbeddings } from '@zbsearch/plugin-embeddings'
import '@tensorflow/tfjs-node' // Or any other appropriate TensorflowJS backend, like @tensorflow/tfjs-backend-webgl

const plugin = await pluginEmbeddings({
  embeddings: {
    // Schema property used to store generated embeddings
    defaultProperty: 'embeddings',
    onInsert: {
      // Generate embeddings at insert-time
      generate: true,
      // properties to use for generating embeddings at insert time.
      // Will be concatenated to generate a unique embedding.
      properties: ['description'],
      verbose: true,
    }
  }
})

const db = create({
  schema: {
    description: 'string',
    // ZBSearch generates 512-dimensions vectors.
    // When using zbsearch/plugin-embeddings, set the property where you want to store embeddings as `vector[512]`.
    embeddings: 'vector[512]'
  },
  plugins: [plugin]
})

// ZBSearch will generate and store embeddings at insert-time!
await insert(db, { description: 'Classroom Headphones Bulk 5 Pack, Student On Ear Color Varieties' })
await insert(db, { description: 'Kids Wired Headphones for School Students K-12' })
await insert(db, { description: 'Kids Headphones Bulk 5-Pack for K-12 School' })
await insert(db, { description: 'Bose QuietComfort Bluetooth Headphones' })

// ZBSearch will also generate and use embeddings at search time when search mode is set to "vector" or "hybrid"!
const searchResults = await search(db, {
  term: 'Headphones for 12th grade students',
  mode: 'vector',
  similarity: 0.75,
})

Read the complete documentation.

Official Docs

Read the complete documentation at https://zbsearch.dev.

Official ZBSearch Plugins

Plugins extend the engine itself — how it ranks, where it stores its data, what it can index.

Write your own plugin: https://www.zbsearch.dev/docs/zbsearch/plugins/writing-your-own-plugins

Official ZBSearch Integrations

Integrations do the opposite of plugins: they take the engine as it is and wire it into somewhere you already work. Everything here runs entirely in the visitor's browser — no server to run, no index to host, and no query ever leaves the page.

The framework plugins are built on the lower layers: they index content with docs-index and render one of the search boxes, which mark up matches with highlight. Reach for those directly only when assembling your own search experience.

See all integrations: https://zbsearch.dev/docs/zbsearch/integrations

License

ZBSearch is licensed under the Apache 2.0 license.