Skip to content

thijs-hakkenberg/nexus

Repository files navigation

Nexus - Personal CRM

A self-hosted CRM system that aggregates communications from multiple channels (email, Telegram, WhatsApp) with a contact graph, LLM-powered processing, vector embeddings, and DAV integration.

Features

  • Multi-channel communication aggregation: Email (IMAP), Telegram, WhatsApp
  • LLM-powered classification: Intent detection, priority assessment, action items via Ollama
  • Hybrid search: Combined text and semantic search with intelligent result ranking
  • Contact graph: Automatic contact resolution and linking
  • Contact deduplication: LLM-assisted duplicate detection and merge with bidirectional CardDAV sync
  • Git-based storage: Version-controlled message archive with symlink aggregates
  • DAV integration: CalDAV/CardDAV sync for contacts and calendar
  • Web UI: Phoenix LiveView for real-time updates

Requirements

  • Elixir 1.17+
  • OTP 26+
  • SQLite 3.35+
  • Git
  • Ollama (for LLM features)

Setup

  1. Install dependencies:

    mix deps.get
  2. Create and migrate the database:

    mix ecto.setup
  3. Install frontend assets:

    mix assets.setup
  4. Create environment configuration:

    cp .env.example .env
    # Edit .env with your credentials (DAV, email accounts, etc.)
  5. Start the Phoenix server:

    mix phx.server

    Or use the dev helper script:

    ./start_dev.sh

Visit localhost:4000 from your browser.

Configuration

Email Accounts

Configure email accounts via environment variables or config/runtime.exs:

config :nexus,
  email: [
    accounts: [
      %{
        name: "personal",
        host: "imap.gmail.com",
        port: 993,
        username: "your@gmail.com",
        password: "app_password",
        folders: ["INBOX", "[Gmail]/Sent Mail"],
        poll_interval: 300_000  # 5 minutes
      }
    ]
  ]

Ollama

Ensure Ollama is running with the required models:

# Install models (see tested models below)
ollama pull gemma3:4b
ollama pull nomic-embed-text

# Start Ollama
ollama serve

Tested LLM Models

Model Size Classification Avg Time Notes
gemma3:4b 2.5GB Excellent 5.3s Recommended - best accuracy/speed balance
gemma3:2b 1.6GB Good ~3s Reasonable balance of speed/quality
gemma3:270m 270MB Poor ~1s Too small for nuanced classification
ministral-3:8b 6.0GB Excellent 13s Great quality but slower
lfm2.5-thinking:1.2b 731MB Poor 2.9s Fast but misclassifies (e.g., invoices as requests)
qwen3-vl:8b 6.1GB Failed - Vision model, doesn't output valid JSON
gpt-oss:20b 13GB Failed - Doesn't follow JSON output format
glm-4.7-flash 19GB Failed - Doesn't follow JSON output format

Embedding model: nomic-embed-text:latest - works well for semantic search.

Evaluating Models

Test different models on your own emails:

mix evaluate_models --account gmail --limit 10
mix evaluate_models --account gmail --limit 5 --models "gemma3:4b,ministral-3:8b"

Configure models via environment variables:

export OLLAMA_CLASSIFY_MODEL="gemma3:4b"
export OLLAMA_EMBED_MODEL="nomic-embed-text"

DAV Integration

Configure CardDAV/CalDAV via environment variables:

export CARDDAV_URL="https://your.server/carddav/user/addressbook/"
export CALDAV_URL="https://your.server/caldav/user/calendar/"
export DAV_USER="username"
export DAV_PASSWORD="password"

# For self-signed certificates (e.g., Synology NAS)
export CARDDAV_SSL_VERIFY="none"

Or via config/runtime.exs:

config :nexus,
  dav: [
    carddav: [
      url: "https://your.caldav.server/carddav/",
      username: "user",
      password: "password"
    ]
  ]

CardDAV sync runs automatically every 5 minutes, syncing contacts bidirectionally between Nexus and your CardDAV server (accessible on iOS/Mac/Android).

Historical Email Import

Import existing emails from your IMAP account (Yugo only handles new emails via IDLE):

# In IEx or via mix run
# Import last 50 emails from INBOX (default)
Nexus.Channels.IMAP.import_historical("gmail")

# Import last 100 emails
Nexus.Channels.IMAP.import_historical("gmail", limit: 100)

# Import from a specific folder
Nexus.Channels.IMAP.import_historical("gmail", folder: "[Gmail]/All Mail", limit: 200)

Imported emails are processed through the full pipeline: parsed, stored in Git, classified by LLM, embedded for semantic search, and indexed in SQLite. Contacts are automatically created for unknown senders.

Message Reprocessing

Reprocess messages when LLM was unavailable or to re-run with a better model:

# Check message counts
Nexus.Messages.Reprocessor.count_all()      # Total messages
Nexus.Messages.Reprocessor.count_pending()  # Messages with default classification

# List pending messages
Nexus.Messages.Reprocessor.list_pending(limit: 10)

# Reprocess only messages with default classification (failed LLM)
Nexus.Messages.Reprocessor.reprocess_defaults()

# Reprocess ALL messages (e.g., after switching to better model)
Nexus.Messages.Reprocessor.reprocess_all(limit: 100)

# Reprocess a specific message
Nexus.Messages.Reprocessor.reprocess_one("msg_abc123")

Messages are identified as "pending" when they have the default classification (intent: "information", priority: "medium"), which indicates the LLM classification likely failed.

Architecture

┌─────────────────────────────────────────────────────────────────────┐
│                    Application Supervisor                            │
├─────────────────────────────────────────────────────────────────────┤
│  ┌─────────────────┐  ┌─────────────────┐  ┌─────────────────┐      │
│  │  IMAP Supervisor │  │  DAV Supervisor  │  │ Channel Supervisor│      │
│  └────────┬────────┘  └────────┬────────┘  └────────┬────────┘      │
│           │                     │                     │               │
│           ▼                     ▼                     ▼               │
├───────────────────────────────────────────────────────────────────────┤
│                        Message Pipeline (GenStage)                   │
│  Normalizer → GitStore → Classifier → Embedder → SymlinkManager     │
├───────────────────────────────────────────────────────────────────────┤
│                        Storage Layer                                  │
│  ┌─────────────────────┐  ┌─────────────────────────────────┐       │
│  │ Git Repository      │  │ SQLite (Metadata + Vectors)     │       │
│  └─────────────────────┘  └─────────────────────────────────┘       │
└───────────────────────────────────────────────────────────────────────┘

Directory Structure

nexus/
├── lib/
│   ├── nexus/                 # Core application
│   │   ├── channels/          # Message sources (IMAP, Telegram, etc.)
│   │   ├── pipeline/          # Processing pipeline stages
│   │   ├── storage/           # Git and SQLite storage
│   │   ├── contacts/          # Contact management
│   │   ├── llm/               # Ollama integration
│   │   └── dav/               # CalDAV/CardDAV
│   ├── nexus_web/             # Phoenix web app
│   └── nexus_cli/             # CLI (future)
├── data/
│   ├── messages.git/          # Git storage
│   └── nexus.db               # SQLite database
└── test/

Git Storage Structure

Messages are stored using Hive-style date partitioning with symlink aggregates:

messages.git/
├── raw/                              # Primary storage
│   └── year=2024/month=01/day=25/
│       ├── {msg_id}.eml
│       ├── {msg_id}.md
│       └── {msg_id}.json
├── by_contact/                       # Symlink aggregate
│   └── contact={id}/
├── by_topic/                         # Symlink aggregate
├── by_channel/                       # Symlink aggregate
└── by_action/                        # Symlink aggregate

MCP Server (AI Assistant Integration)

Nexus includes an MCP (Model Context Protocol) server for integration with AI assistants like Claude.

Starting the MCP Server

mix run --no-halt -e "Nexus.MCP.Server.start()"

Available Tools

Tool Description
search_messages Search messages (hybrid/text/semantic mode)
import_emails Backfill historical emails from IMAP
reprocess_messages Re-run LLM classification on messages
create_contact Create a new contact

Available Resources

Resource URI
Messages nexus://messages?limit=50&filter=all|action|high
Single Message nexus://messages/{id}
Contacts nexus://contacts
Status nexus://status

Claude Code Configuration

Add the MCP server to Claude Code:

cd /path/to/nexus
claude mcp add nexus -- mix run --no-halt -e "Nexus.MCP.Server.start()"

Or with explicit working directory:

claude mcp add nexus --dir /path/to/nexus -- mix run --no-halt -e "Nexus.MCP.Server.start()"

Claude Desktop Configuration

Add to ~/Library/Application Support/Claude/claude_desktop_config.json:

{
  "mcpServers": {
    "nexus": {
      "command": "mix",
      "args": ["run", "--no-halt", "-e", "Nexus.MCP.Server.start()"],
      "cwd": "/path/to/nexus"
    }
  }
}

Testing

# Run all tests
mix test

# Run with coverage
mix coveralls

# Run integration tests (requires external services)
mix test --include integration

License

MIT

About

An AI first elixer based CRM for self-hosted personal productivity

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors

Languages