Merge pull request #25 from llbbl/feat/unit-testing

Remove cloud embedding config and add just tasks

Author: llbbl

.env.example

@@ -4,10 +4,3 @@ TURSO_DB_URL=libsql://your-database.turso.io
 TURSO_AUTH_TOKEN=your-auth-token-here
 
 # Note: If credentials are not set, scripts with :local suffix will use file:local.db instead
-
-# Embedding Provider (local, gemini, or openai)
-EMBEDDING_PROVIDER=local
-
-# Optional: API Keys for cloud embedding providers
-# GEMINI_API_KEY=your-gemini-api-key
-# OPENAI_API_KEY=your-openai-api-key

.github/workflows/ci.yml

@@ -26,8 +26,13 @@ jobs:
       - name: Install dependencies
         run: pnpm install
 
-      - name: Run tests
-        run: pnpm test --run
+      - name: Run tests with coverage
+        run: pnpm test:coverage --run
+
+      - name: Upload coverage reports to Codecov
+        uses: codecov/codecov-action@v5
+        with:
+          token: ${{ secrets.CODECOV_TOKEN }}
 
       - name: Initialize local database
         run: pnpm db:init:local

CLAUDE.md

@@ -52,7 +52,7 @@ pnpm format     # Format all files
 
 ### Content Flow
 1. **Markdown → Database**: Content in `./content` is indexed into Turso via `scripts/index-content.ts`
-2. **libsql-search**: Handles embedding generation (local/Gemini/OpenAI), vector storage, and semantic search
+2. **libsql-search**: Handles embedding generation (local), vector storage, and semantic search
 3. **Static Generation**: Article pages are pre-rendered at build time using `getStaticPaths()`
 4. **Server Search**: Search API runs server-side at `/api/search.json` (requires `output: 'server'` with Node.js adapter)
 
@@ -73,29 +73,20 @@ pnpm format     # Format all files
 Optional in `.env`:
 - `TURSO_DB_URL`: Turso database URL (libsql://...) - if not set, uses local libSQL file
 - `TURSO_AUTH_TOKEN`: Turso authentication token - if not set, uses local libSQL file
-- `EMBEDDING_PROVIDER`: "local" (default), "gemini", or "openai"
-- Optional: `GEMINI_API_KEY` or `OPENAI_API_KEY` (if using cloud providers)
+
+Embeddings run locally by default; no API keys are required.
 
 **Local Development**: If Turso credentials aren't provided, the project automatically falls back to a local SQLite file (`local.db`) for database operations. This is useful for CI builds and local development without cloud dependencies.
 
 ## Critical Configuration
 
-### Server-Side Rendering + Dual Adapter Support
+### Server-Side Rendering + Node Adapter
 The search API endpoint requires SSR with an adapter. The configuration uses:
 - `output: 'server'` - Enables server-side rendering
-- **Dual adapter support** - Conditionally uses Node.js or Cloudflare adapter based on `ADAPTER` env var
-  - Default: `node({ mode: 'standalone' })` - For traditional Node.js deployments
-  - Cloudflare: `cloudflare()` - For Cloudflare Workers (set `ADAPTER=cloudflare`)
+- `node({ mode: 'standalone' })` - For traditional Node.js deployments
 - Article pages marked with `prerender: true` are pre-rendered as static HTML
 - Search API marked with `prerender: false` runs server-side
 
-**Adapter Selection** (astro.config.mjs:14-16):
-```js
-const adapter = process.env.ADAPTER === 'cloudflare'
-  ? cloudflare()
-  : node({ mode: 'standalone' });
-```
-
 **Never** remove the adapter or change output to 'static', or the search API will break.
 
 ### Content Structure
@@ -129,11 +120,11 @@ The project relies heavily on libsql-search. When modifying search behavior:
 3. Update `src/pages/api/search.json.ts` for search query changes
 4. Maintain embedding dimension consistency (768) across indexing and search
 
-### Customizing Embedding Providers
-To switch providers, update `.env` and ensure API keys are set. The dimension (768) must match across:
+### Customizing Embeddings
+The embedding dimension (768) must match across:
 - `scripts/index-content.ts` (createTable and indexContent)
-- Search API (automatically uses same provider)
-- Re-index content after switching providers
+- Search API
+- Re-index content after changing the embedding model or dimension
 
 ### Styling
 - Uses Tailwind CSS 4 via Vite plugin
@@ -152,7 +143,6 @@ To switch providers, update `.env` and ensure API keys are set. The dimension (7
 
 ## Deployment Options
 
-> **Note**: Cloudflare Workers/Pages deployment is being developed in the `cloudflare-workers` branch.
 
 ### Node.js Platforms (Vercel, Netlify, etc.)
 
@@ -232,4 +222,4 @@ chore: bump dependencies
 ### Release Process
 1. Make commits following the convention above
 2. Create and push a version tag: `git tag v1.2.3 && git push --tags`
-3. GitHub Actions will automatically generate changelog and create release
+3. GitHub Actions will automatically generate changelog and create release

Dockerfile

@@ -7,7 +7,6 @@ FROM node:20-slim AS builder
 # Build arguments for Turso credentials (required for indexing and pre-rendering)
 ARG TURSO_DB_URL
 ARG TURSO_AUTH_TOKEN
-ARG EMBEDDING_PROVIDER=local
 
 # Install pnpm
 RUN corepack enable && corepack prepare pnpm@latest --activate
@@ -27,7 +26,6 @@ COPY . .
 # Set environment variables for build
 ENV TURSO_DB_URL=$TURSO_DB_URL
 ENV TURSO_AUTH_TOKEN=$TURSO_AUTH_TOKEN
-ENV EMBEDDING_PROVIDER=$EMBEDDING_PROVIDER
 
 # Index content to Turso database (env vars already set via ENV directives)
 RUN pnpm exec tsx scripts/init-db.ts && pnpm exec tsx scripts/index-content.ts

Makefile

@@ -1,5 +1,7 @@
 # semantic-docs
 
+[![Coverage](https://img.shields.io/codecov/c/github/llbbl/semantic-docs?label=coverage)](https://codecov.io/gh/llbbl/semantic-docs) [![CI](https://github.com/llbbl/semantic-docs/actions/workflows/ci.yml/badge.svg)](https://github.com/llbbl/semantic-docs/actions/workflows/ci.yml) [![Release](https://img.shields.io/github/v/release/llbbl/semantic-docs)](https://github.com/llbbl/semantic-docs/releases)
+
 Documentation theme with semantic vector search.
 
 A beautiful, dark-mode documentation theme powered by [libsql-search](https://github.com/llbbl/libsql-search) for semantic search capabilities. Perfect for technical documentation, knowledge bases, and content-heavy sites.
@@ -31,6 +33,16 @@ Or use as a template on GitHub.
 pnpm install
 ```
 
+Optional: use the `justfile` task runner for common commands:
+
+```bash
+just
+just dev
+just test
+```
+
+See `docs/just.md` for the full list of recipes.
+
 ### 3. Set Up Environment
 
 Copy `.env.example` to `.env` and add your credentials:
@@ -44,7 +56,6 @@ Edit `.env`:
 ```env
 TURSO_DB_URL=libsql://your-database.turso.io
 TURSO_AUTH_TOKEN=your-auth-token
-EMBEDDING_PROVIDER=local
 ```
 
 **Get Turso credentials:**
@@ -130,21 +141,9 @@ const { title = "Your Site Name", description = "Your description" } = Astro.pro
 
 Edit `src/styles/global.css` to change the color scheme. The theme uses OKLCH colors for smooth gradients and perceptual uniformity.
 
-### Change Embedding Provider
-
-**Use Gemini** (free tier: 1,500 requests/day):
-
-```env
-EMBEDDING_PROVIDER=gemini
-GEMINI_API_KEY=your-key
-```
-
-**Use OpenAI** (paid):
+### Embeddings
 
-```env
-EMBEDDING_PROVIDER=openai
-OPENAI_API_KEY=your-key
-```
+Semantic search uses local embeddings by default, so no API keys are required.
 
 ## Project Structure
 
@@ -178,8 +177,6 @@ semantic-docs/
 
 ## Deployment
 
-> **Note**: Cloudflare Workers/Pages deployment support is currently in development on the `cloudflare-workers` branch.
-
 ### Container-Based Platforms (Recommended)
 
 This project is designed to run on platforms that support Docker containers, such as:
@@ -278,21 +275,14 @@ pnpm preview
 
 First run downloads ~50MB model. Subsequent runs use cache.
 
-Use Gemini for faster embeddings:
-
-```env
-EMBEDDING_PROVIDER=gemini
-GEMINI_API_KEY=your-key
-```
-
 ## Tech Stack
 
 - **Framework**: [Astro](https://astro.build) 5
 - **Search**: [libsql-search](https://github.com/llbbl/libsql-search)
 - **Database**: [Turso](https://turso.tech) (libSQL)
 - **Styling**: [Tailwind CSS](https://tailwindcss.com) 4
 - **UI**: React islands for interactivity
-- **Embeddings**: Xenova, Gemini, or OpenAI
+- **Embeddings**: Xenova (local)
 
 ## License
 

README.md

@@ -117,15 +117,12 @@ const embedding = await embedder(text, {
 console.log(embedding.data);  // [0.123, -0.456, 0.789, ...]
 ```
 
-### Popular Models
+### Popular Local Models
 
 | Model | Dimensions | Speed | Quality |
 |-------|------------|-------|---------|
 | all-MiniLM-L6-v2 | 384 | Fast | Good |
 | all-mpnet-base-v2 | 768 | Medium | Better |
-| text-embedding-3-small (OpenAI) | 1536 | API | Excellent |
-| text-embedding-ada-002 (OpenAI) | 1536 | API | Excellent |
-| textembedding-gecko (Gemini) | 768 | API | Excellent |
 
 ## Implementation in Astro Vault
 
@@ -351,7 +348,7 @@ async function hybridSearch(query: string) {
 - **E-commerce**: Product codes (full-text) + descriptions (semantic)
 - **Code search**: Function names (full-text) + purpose (semantic)
 
-## Embedding Providers
+## Embeddings
 
 ### Local (Xenova Transformers)
 ```typescript
@@ -364,31 +361,6 @@ const embedder = await pipeline('feature-extraction',
 const embedding = await embedder(text);
 ```
 
-### OpenAI
-```typescript
-// Pros: High quality, fast
-// Cons: Costs money, rate limits
-
-import { OpenAI } from 'openai';
-const openai = new OpenAI();
-const response = await openai.embeddings.create({
-  model: 'text-embedding-3-small',
-  input: text,
-});
-const embedding = response.data[0].embedding;
-```
-
-### Gemini
-```typescript
-// Pros: High quality, generous free tier
-// Cons: Rate limits
-
-import { GoogleGenerativeAI } from '@google/generative-ai';
-const genAI = new GoogleGenerativeAI(process.env.GEMINI_API_KEY);
-const model = genAI.getGenerativeModel({ model: 'embedding-001' });
-const result = await model.embedContent(text);
-const embedding = result.embedding.values;
-```
 
 ## Use Cases
 
@@ -409,5 +381,4 @@ const embedding = result.embedding.values;
 
 - **Xenova Transformers**: [huggingface.co/docs/transformers.js](https://huggingface.co/docs/transformers.js)
 - **Sentence Transformers**: [sbert.net](https://www.sbert.net/)
-- **OpenAI Embeddings**: [platform.openai.com/docs/guides/embeddings](https://platform.openai.com/docs/guides/embeddings)
 - **Vector Search Explained**: [pinecone.io/learn/vector-database](https://www.pinecone.io/learn/vector-database/)

content/features/semantic-search.md

@@ -41,7 +41,7 @@ Check out the README for customization options:
 
 - Change colors
 - Update site title
-- Configure embedding providers
+- Tune search behavior
 - And more!
 
 ---

content/getting-started/welcome.md

@@ -13,7 +13,7 @@ Semantic Docs is a modern documentation theme built with Astro, featuring semant
 
 ### Semantic Vector Search
 - **Vector embeddings**: Content is indexed with 768-dimension embeddings
-- **Three embedding providers**: Local (onnxruntime), Gemini, or OpenAI
+- **Local embeddings**: Runs on-device with no API keys required
 - **Fast semantic search**: Natural language queries return relevant results
 - **Edge-optimized**: Runs on Turso's edge database for low latency
 
@@ -116,12 +116,6 @@ semantic-docs/
 TURSO_DB_URL=libsql://your-db.turso.io
 TURSO_AUTH_TOKEN=your-token
 
-# Embedding provider (optional, defaults to "local")
-EMBEDDING_PROVIDER=local  # or "gemini" or "openai"
-
-# API keys (if using cloud embeddings)
-GEMINI_API_KEY=your-key
-OPENAI_API_KEY=your-key
 ```
 
 ### Astro Configuration

content/theme/overview.md

@@ -50,12 +50,10 @@ For multi-server deployments, consider:
    ```
 
 2. **Edge rate limiting** (Platform-specific)
-   - Cloudflare Workers: Use Durable Objects
    - Vercel: Use Edge Config or KV
    - Netlify: Use Blobs
 
 3. **WAF/CDN rate limiting**
-   - Cloudflare: Configure rate limiting rules
    - AWS CloudFront: Lambda@Edge
    - Fastly: VCL rate limiting
 
@@ -72,15 +70,6 @@ The API limits:
 - Risk: CPU abuse
 - Mitigation: Rate limiting sufficient
 
-**Gemini Provider** (Free tier: 1,500 req/day)
-- Risk: API quota exhaustion
-- Mitigation: Consider stricter rate limits (5-10 req/min)
-
-**OpenAI Provider** (Paid)
-- Risk: Cost abuse
-- Mitigation: Monitor usage, alert on anomalies
-- Recommendation: Use OpenAI's own rate limiting
-
 ### Turso Database Limits
 
 Free tier limits: