Marginalia

Marginalia is a self-hosted RAG app for PDFs:

• Upload PDF documents
• Generate embeddings and index chunks in Postgres + pgvector
• Ask questions across your library with cited sources
• Browse auto-generated notes per document

The backend is FastAPI and the frontend is Streamlit.

Quickstart

1. Clone and configure:

git clone <your-repo-url> marginalia
cd marginalia
cp .env.example .env

2. (Optional) Add your Anthropic key to .env for answer/note generation:

ANTHROPIC_API_KEY=your_key_here

3. Start services:

docker compose up --build

4. Open:

• Frontend: http://localhost:8501
• Backend API: http://localhost:8000

API Endpoints

• POST /api/documents — upload PDF
• GET /api/documents — list documents
• GET /api/documents/{id}/note — get generated note
• POST /api/query — ask a question
• GET /api/metrics — retrieval/query metrics
• GET /api/health — health check

Project Structure

marginalia/
├── backend/
│   ├── app/
│   ├── sql/init.sql
│   ├── requirements.txt
│   └── Dockerfile
├── frontend/
│   ├── app.py
│   ├── requirements.txt
│   └── Dockerfile
├── docker-compose.yml
└── .env.example

Notes

• Without ANTHROPIC_API_KEY, ingestion and retrieval still work, but LLM-generated answers/notes are unavailable (unless you configure a local model).
• First run downloads embedding/reranker models and caches them in Docker volumes.

Local model (optional)

Marginalia can run a very small local model inside the backend container for on-device answer generation. Current supported provider: gpt4all.

Setup summary:

1. Install native runtime deps (Dockerfile updated). The backend image creates /app/models for mounting model files.

2. Download a gpt4all-compatible quantized model and place it in backend/models, e.g.:

   mkdir -p backend/models

   download example model (replace URL with desired build):

   wget -O backend/models/gpt4all.bin https://example.com/path/to/gpt4all-quantized.bin

3. Configure .env (or exported env vars):

   LOCAL_MODEL_PROVIDER=gpt4all LOCAL_MODEL_PATH=/app/models/gpt4all.bin LOCAL_MODEL_MAX_TOKENS=512

4. Rebuild and start the stack:

   docker compose up --build

5. Use the local-only endpoint to query:

   POST /api/local-query JSON body: { "question": "...", "top_k": 5 }

Notes:

• If a local model isn't configured or fails to load, /api/local-query will return 503 with an explanatory message.
• If you also set ANTHROPIC_API_KEY, the normal /api/query endpoint will prefer the local model when LOCAL_MODEL_PROVIDER is set, otherwise fall back to Anthropic.
• Adjust model download source and mounting strategy to match the runtime you choose.