Restructure tests

Author: dimamik

.github/workflows/ci.yml

@@ -23,7 +23,7 @@ jobs:
     # Additional services can be defined here if required.
     services:
       db:
-        image: postgres:15
+        image: postgres:17
         ports:
           - 5432/tcp
         env:
@@ -86,6 +86,11 @@ jobs:
     - name: Install dependencies
       run: mix deps.get --check-locked
 
+    - name: Install pg_textsearch extension
+      run: |
+        docker exec ${{ job.services.db.id }} bash -lc "apt-get update && apt-get install -y build-essential postgresql-server-dev-17"
+        docker cp ./pg_textsearch ${{ job.services.db.id }}:/tmp/pg_textsearch
+        docker exec ${{ job.services.db.id }} bash -lc "cd /tmp/pg_textsearch && make && make install"
     # TODO These steps can be moved to `mix.exs`
 
     # Step: Compile the project treating any warnings as errors.

CHANGELOG.md

@@ -1,3 +1,30 @@
+# v0.6.0
+
+## New 🔥
+
+**BM25 Full-Text Search** is now available via the new `Torus.bm25/5` macro!
+
+[BM25](https://en.wikipedia.org/wiki/Okapi_BM25) is a modern ranking algorithm that generally provides superior relevance scoring compared to traditional TF-IDF (used by `full_text/5`). This integration uses the [pg_textsearch](https://github.com/timescale/pg_textsearch) extension by Timescale.
+
+Key features:
+- State-of-the-art BM25 ranking with configurable parameters (k1, b)
+- Fast top-k queries via Block-Max WAND optimization
+- Simple syntax: `Post |> Torus.bm25([p], p.body, "search term") |> limit(10)`
+- Score selection and filtering capabilities
+- Pre-filtering with `<@>>` operator
+- Language support through PostgreSQL text search configurations
+
+Requirements:
+- PostgreSQL 17+
+- pg_textsearch extension installed
+- BM25 index on the search column
+
+See `Torus.bm25/5` documentation for complete details and migration guide.
+
+**When to use BM25 vs full_text:**
+- Use `bm25/5` for superior single-column search with modern relevance ranking
+- Use `full_text/5` for multi-column search with weights or when using stored tsvector columns
+
 # v0.5.3
 
 ## Fixes

README.md

@@ -38,7 +38,7 @@ Post
 
 See [`full_text/5`](https://hexdocs.pm/torus/Torus.html#full_text/5) for more details.
 
-## 6 types of search:
+## 7 types of search:
 
 1. **Pattern matching**: Searches for a specific pattern in a string.
 
@@ -84,10 +84,28 @@ See [`full_text/5`](https://hexdocs.pm/torus/Torus.html#full_text/5) for more de
    ["Diagon Bombshell"]
    ```
 
-   Use it when you don’t care about spelling, the documents are long, or if you need to order the results by rank.
+   Use it when you don't care about spelling, the documents are long, you need multi-column search with weights, or if you need to order the results by rank.
 
    See [`full_text/5`](https://hexdocs.pm/torus/Torus.html#full_text/5) for more details.
 
+1. **BM25 full text**: Modern BM25 ranking algorithm for superior relevance scoring using the [pg_textsearch](https://github.com/timescale/pg_textsearch) extension. BM25 generally provides better ranking than traditional built-in TF-IDF full text search and is optimized for top-k queries.
+
+   ```elixir
+   iex> insert_post!(body: "PostgreSQL is a powerful relational database system")
+   ...> insert_post!(body: "BM25 is a ranking function used in search engines")
+   ...> insert_post!(body: "Full text search enables finding relevant documents")
+   ...> Post
+   ...> |> Torus.bm25([p], p.body, "database system")
+   ...> |> limit(10)
+   ...> |> select([p], p.body)
+   ...> |> Repo.all()
+   ["PostgreSQL is a powerful relational database system"]
+   ```
+
+   Use it when you need state-of-the-art relevance ranking for single-column search, especially with LIMIT clauses. Requires PostgreSQL 17+.
+
+   See [`bm25/5`](https://hexdocs.pm/torus/Torus.html#bm25/5) for more details.
+
 1. **Semantic Search**: Understands the contextual meaning of queries to match and retrieve related content utilizing natural language processing. Read more about semantic search in [Semantic search with Torus guide](/guides/semantic_search.md).
 
    ```elixir
@@ -131,7 +149,7 @@ Torus offers a few helpers to debug, explain, and analyze your queries before us
 
 ## Torus support
 
-For now, Torus supports pattern match, similarity, full-text, and semantic search, with plans to expand support further. These docs will be updated with more examples on which search type to choose and how to make them more performant (by adding indexes or using specific functions).
+For now, Torus supports pattern match, similarity, full-text (TF-IDF and BM25), and semantic search, with plans to expand support further. These docs will be updated with more examples on which search type to choose and how to make them more performant (by adding indexes or using specific functions).
 
 <!-- MDOC -->