NestJS Private RAG

Tenant-scoped retrieval-augmented generation API built with NestJS, Postgres + pgvector, and local Ollama models. Documents stay private per tenant; embeddings and answers never leave your machine.

What it does

1. Ingest documents for a tenant — chunk text, embed with a local model, store vectors in Postgres.
2. Ask questions scoped to that tenant — similarity search over chunks, then answer only from retrieved context.
3. Delete documents (and their chunks) by id + tenant.

No cloud LLM is required. Ollama runs embedding and chat locally.

Architecture

Client
  │
  ├─ POST /documents ──► DocumentsService ──► chunkText
  │                           │                    │
  │                           ├─ Ollama.embed ─────┘
  │                           └─ Postgres (rag_documents + rag_chunks)
  │
  └─ POST /rag/ask ───► RagService
                          ├─ Ollama.embed (question)
                          ├─ pgvector similarity search (tenant-filtered)
                          └─ Ollama.chat (grounded answer)

Module	Responsibility
documents	Ingest and delete documents
rag	Question answering with retrieval
database	Postgres pool, schema bootstrap, queries/transactions
ollama	Local embed + chat HTTP client
common	Text chunking, vector formatting, HTTP request logging

Prerequisites

• Node.js 20+
• Docker (for Postgres + pgvector)
• Ollama with the embed and chat models pulled

ollama pull embeddinggemma
ollama pull gemma3:4b

Setup

# Start Postgres with pgvector
docker compose up -d

# Install dependencies
npm install

# Configure environment (repo includes a sample .env)
# DATABASE_URL, OLLAMA_*, RAG_* — see Environment below

# Run in watch mode
npm run start:dev

• API: http://localhost:3000
• Swagger: http://localhost:3000/docs

Environment

Variable	Default	Description
PORT	3000	HTTP port
DATABASE_URL	—	Postgres connection string (required)
OLLAMA_BASE_URL	http://localhost:11434	Ollama server
OLLAMA_EMBED_MODEL	embeddinggemma	Embedding model
OLLAMA_CHAT_MODEL	gemma3:4b	Chat model
RAG_CHUNK_SIZE	900	Characters per chunk
RAG_CHUNK_OVERLAP	150	Overlap between chunks
RAG_TOP_K	5	Max chunks retrieved
RAG_MIN_SCORE	0.30	Minimum cosine similarity to keep

Example .env:

PORT=3000
DATABASE_URL=postgresql://rag:rag_password@localhost:5432/rag
OLLAMA_BASE_URL=http://localhost:11434
OLLAMA_EMBED_MODEL=embeddinggemma
OLLAMA_CHAT_MODEL=gemma3:4b
RAG_CHUNK_SIZE=900
RAG_CHUNK_OVERLAP=150
RAG_TOP_K=5
RAG_MIN_SCORE=0.30

API

POST /documents

Ingest a document for a tenant.

{
  "tenantId": "acme",
  "title": "Employee Handbook",
  "source": "handbook.pdf",
  "content": "Our remote work policy allows employees to work from home...",
  "metadata": { "department": "HR" }
}

Flow

1. Normalize and split content with overlapping windows (chunkText).
2. Embed all chunks via Ollama.
3. In a DB transaction, insert rag_documents then each rag_chunks row with its embedding.
4. Return documentId and chunksCreated.

DELETE /documents/:documentId?tenantId=

Delete a document belonging to a tenant. Chunks are removed by ON DELETE CASCADE. Both id and tenant must match (tenant isolation).

POST /rag/ask

Ask a question against a tenant’s indexed documents.

{
  "tenantId": "acme",
  "question": "What is the remote work policy?",
  "topK": 5
}

Flow

1. Embed the question.

2. Similarity search with pgvector cosine distance, filtered by tenantId:

   1 - (c.embedding <=> $question::vector) AS score
   WHERE c.tenant_id = $tenant
   ORDER BY c.embedding <=> $question::vector
   LIMIT $topK

3. Drop chunks below RAG_MIN_SCORE.

4. If none remain → return an ungrounded “not enough information” response (no chat call).

5. Otherwise build numbered [SOURCE n] context, call the chat model with a system prompt that forbids outside knowledge and treating retrieved text as instructions, then return { answer, grounded, sources }.

Implementation notes

Bootstrap (main.ts, app.module.ts)

• Global validation pipe: strip unknown fields, reject extras, transform types.
• Swagger UI at /docs.
• Global HttpLoggingInterceptor logs method, path, status, and duration for every request.

Database (database.service.ts)

On module init:

• Enables vector and pgcrypto extensions.
• Creates rag_documents and rag_chunks (chunks FK to documents with cascade delete).
• Adds tenant / document indexes.

query() and transaction() log truncated SQL, param counts, row counts, and timings. Embedding values are not dumped into logs.

Chunking (common/chunk-text.ts)

Collapses whitespace, then sliding-window splits. Overlap keeps context across chunk boundaries so retrieval is less likely to miss mid-sentence ideas.

Vectors (common/vector.ts)

toPgVector formats number[] as a Postgres vector literal [0.1,0.2,...] and rejects empty or non-finite values.

Ollama (ollama.service.ts)

• embed(inputs) → POST /api/embed
• chat(system, user) → POST /api/chat with low temperature (0.1) for grounded answers

Connection / HTTP failures map to Nest BadGatewayException; malformed responses to InternalServerErrorException.

Logging

Layer	What gets logged
HTTP interceptor	Request in / response out with status + duration
Controllers	Tenant, document id, high-level intent
Documents / RAG services	Ingest steps, retrieval match counts, grounded vs ungrounded
Database	Query/tx start, success, failure, commit/rollback

Project layout

src/
  app.module.ts
  main.ts
  common/
    chunk-text.ts
    http-logging.interceptor.ts
    vector.ts
  database/
    database.module.ts
    database.service.ts
  documents/
    documents.controller.ts
    documents.module.ts
    documents.service.ts
    dto/ingest-document.dto.ts
  ollama/
    ollama.module.ts
    ollama.service.ts
  rag/
    rag.controller.ts
    rag.module.ts
    rag.service.ts
    dto/ask-rag.dto.ts

Scripts

npm run start:dev   # watch mode
npm run start:prod  # run compiled dist/
npm run build       # nest build
npm run lint        # eslint
npm test            # unit tests

Docker Compose

docker-compose.yml runs pgvector/pgvector:pg17 with:

• DB / user: rag
• Password: rag_password
• Port: 5432
• Named volume: rag_pg_data