Hello, been working on adding cloud provider support to osgrep as an alternative to local models. Wanted to check if you'd be interested in a PR for this.
What I've implemented:
- Cloud embeddings via OpenRouter (Qwen3-Embedding-8B)
- Cloud reranking via ZeroEntropy (zerank-2)
- .env file support via dotenv for API key management
- Automatic skip of ~150MB model download when using cloud providers
- Windows compatibility fixes (removed Unix-only build scripts, would need to test before merging)
- Parallel workers for cloud providers - cranked up to 14 concurrent workers (vs 4 for local) since there's no local ONNX memory overhead. Makes indexing crazy fast.
How it works:
Set provider + keys
export OSGREP_EMBED_PROVIDER=qwen
export OSGREP_RERANK_PROVIDER=zeroentropy
export QWEN_API_KEY=your-key
export ZEROENTROPY_API_KEY=your-key
Use normally - no local model download needed
osgrep "query"
Cost: OpenRouter's Qwen3-Embedding-8B is dirt cheap ($0,1/M)+ SOTA reranker zerank-2 for ($0,025/M), and the parallel batching keeps API calls efficient. Indexed a full repo for around 1 cent.
Current limitations (needs more work):
- Models are hardcoded - can't swap models without code changes
- Vector dimensions tied to models - switching models requires full re-index (384 local vs 4096 cloud for Qwen's model)
- No index metadata validation - if you switch providers, old index silently breaks
- .env loads from CWD - not from install location, so users need .env in each project or use system env vars
- Mixed provider mode still downloads both models (could be more granular)
What I'd need to do before a proper PR:
- Store model info in index metadata
- Add mismatch detection on startup (warn + force re-index)
- Make it model agnostic (needs more thought due to vector dimension mismatches)
- Add tests for cloud providers
- Clean up code / match project style (lot of it was written by Opus so I'd need to manually verify everything before merging)
lmk if it's something that you guys are interested in, would love to make it PR ready.
Branch: https://github.com/anessbelbati/osgrep
Hello, been working on adding cloud provider support to osgrep as an alternative to local models. Wanted to check if you'd be interested in a PR for this.
What I've implemented:
How it works:
Set provider + keys
export OSGREP_EMBED_PROVIDER=qwen
export OSGREP_RERANK_PROVIDER=zeroentropy
export QWEN_API_KEY=your-key
export ZEROENTROPY_API_KEY=your-key
Use normally - no local model download needed
osgrep "query"
Cost: OpenRouter's Qwen3-Embedding-8B is dirt cheap ($0,1/M)+ SOTA reranker zerank-2 for ($0,025/M), and the parallel batching keeps API calls efficient. Indexed a full repo for around 1 cent.
Current limitations (needs more work):
What I'd need to do before a proper PR:
lmk if it's something that you guys are interested in, would love to make it PR ready.
Branch: https://github.com/anessbelbati/osgrep