Use FastEmbed as default embedding - no API key needed

[automationpi]

Summary

Switches the default embedding model from OpenAI text-embedding-3-small (requires API key) to FastEmbed bge-small-en-v1.5 (runs locally on CPU, no API key).

This simplifies Tier 2 setup from 4 secrets to 2:

• Before: ANTHROPIC_API_KEY + OPENAI_API_KEY + QDRANT_URL + QDRANT_API_KEY
• After: ANTHROPIC_API_KEY + QDRANT_URL + QDRANT_API_KEY

OpenAI and Cohere remain available as optional plugins via pip install mergelore-indexer[openai] or mergelore-indexer[cohere].

Changes

• Added FastEmbed provider (ONNX-based, CPU, 3 model options)
• Changed default from text-embedding-3-small to bge-small-en-v1.5
• Moved openai/cohere to optional dependencies
• Lazy imports for openai/cohere with clear error messages
• Updated config, factory, tests, workflows, and sample files
• 25 Python tests passing

Embedding models

• bge-small-en-v1.5 - 384d, fast, no API key (new default)
• all-MiniLM-L6-v2 - 384d, well-tested, no API key
• nomic-embed-text-v1.5 - 768d, best quality, no API key
• text-embedding-3-small - 1536d, OpenAI, requires key (optional)
• embed-english-v3.0 - 1024d, Cohere, requires key (optional)

[github-actions]

mergelore findings

Reviewed 0 past PRs · 0 relevant decisions found · confidence threshold 0.5

⚠️ Potential conflicts

Removal of Nomic embedding support with name change · confidence: 0.75

The PR removes support for 'nomic-embed-text' model but introduces a new 'nomic-embed-text-v1.5' model via FastEmbed. While the new version offers compatibility, the old model name is no longer supported. Any existing deployments explicitly using 'nomic-embed-text' will break. The factory.py previously had explicit NomicEmbedding class support; this is now consolidated into FastEmbed. Tests were updated, but the breaking change in model naming should be documented clearly.

/mergelore-acknowledge · /mergelore-override [reason] · /mergelore-update-record

Dependency restructuring: embedding backends moved to optional dependencies · confidence: 0.85

pyproject.toml shows that 'openai' and 'cohere' packages are moved from base dependencies to optional dependencies ('openai' and 'cohere' extras). However, the factory.py now uses lazy imports (ImportError handling) for these optional packages. This creates a gap: the base 'mergelore-indexer' install will no longer include OpenAI/Cohere support by default, which is a breaking change if users relied on the automatic availability. Tests handle this with try/except for ImportError, but users upgrading may experience runtime failures when requesting these models without the optional extras.

/mergelore-acknowledge · /mergelore-override [reason] · /mergelore-update-record

Default embedding model changed from text-embedding-3-small to bge-small-en-v1.5 · confidence: 0.90

The default embedding model is changed from 'text-embedding-3-small' (OpenAI, requires API key) to 'bge-small-en-v1.5' (FastEmbed local). While this simplifies the default deployment (no API key needed), it is a breaking change for any existing deployments that relied on OpenAI embeddings without explicitly setting MERGELORE_EMBED_MODEL. Vector dimensions also change (1536 → 384), which may affect downstream vector search compatibility if not re-indexed.

/mergelore-acknowledge · /mergelore-override [reason] · /mergelore-update-record

---

ℹ️ Related context

• Test refactoring reduces validation coverage for embed model selection (confidence: 0.65): In test_embed_factory.py, tests for OpenAI and Cohere providers now accept both ValueError and ImportError. This relaxes validation specificity: previously, tests explicitly verified that missing API keys raised ValueError with specific messages. Now, ImportError (from lazy import) is also accepted, making it harder to distinguish between 'API key missing' vs 'optional package not installed'. While functionally acceptable, it reduces precision in error reporting and testing.

---

mergelore v0.1.0 · docs