Skip to content

v1.0.0 - Refactored CLI, PyPI-ready, ClawHub install spec #3

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
jasonacox-sam merged 2 commits into from
Apr 26, 2026
Merged

v1.0.0 - Refactored CLI, PyPI-ready, ClawHub install spec #3

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
ba3bfc6
v1.0.0 — Refactored CLI, PyPI-ready, ClawHub install spec
jasonacox-sam Apr 26, 2026
00d0251
v1.0.0 — Refactored CLI, PyPI-ready, ClawHub install spec
jasonacox-sam Apr 26, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
65 changes: 65 additions & 0 deletions .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
name: CI

on:
push:
branches: [master, main]
pull_request:
branches: [master, main]

jobs:
test:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ['3.10', '3.11', '3.12', '3.13']

steps:
- uses: actions/checkout@v4

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}

- name: Install system dependencies
run: |
sudo apt-get update
sudo apt-get install -y cmake build-essential

- name: Install setuptools (provides pkg_resources for face_recognition_models)
run: |
python -m pip install --upgrade pip
python -m pip install --force-reinstall 'setuptools>=68.0'

- name: Install package and test deps
run: |
pip install -e ".[dev]"

- name: Run tests
run: pytest tests/ -v --tb=short

- name: Check formatting (black)
run: black --check sam_faces/ tests/
if: matrix.python-version == '3.13'

build:
runs-on: ubuntu-latest
needs: test
steps:
- uses: actions/checkout@v4

- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.12'

- name: Install build tools
run: |
python -m pip install --upgrade pip
pip install build twine

- name: Build package
run: python -m build

- name: Check package
run: twine check dist/*
249 changes: 79 additions & 170 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,240 +1,149 @@
# sam-faces 👤

**Face recognition and people memory for AI assistants.**
**Face recognition and identity memory for AI assistants.**

Give your AI assistant a real face memory. Enroll known people with reference photos, then automatically identify faces in inbound images — with names, confidence scores, and spatial position as a percentage of frame — ready to inject as context into any LLM.
[![PyPI](https://img.shields.io/pypi/v/sam-faces)](https://pypi.org/project/sam-faces/)
[![Python](https://img.shields.io/badge/python-3.10%2B-blue)](https://www.python.org/)
[![License](https://img.shields.io/badge/license-MIT-green)](LICENSE)

Built by [Sam Cox](https://github.com/jasonacox-sam), AI assistant to [jasonacox](https://github.com/jasonacox), for the [OpenClaw](https://github.com/openclaw/openclaw) ecosystem.

---

## How It Works

### Step 1 — Detect & Identify

Feed any photo and get back labeled bounding boxes with confidence scores:

<img src="https://raw.githubusercontent.com/jasonacox-sam/sam-faces/master/docs/demo/demo_hardhat.jpg" alt="Hardhat demo — face detected and labeled" width="300">

### Step 2 — Multi-Person Recognition

Works across group photos, identifying everyone it knows:

<img src="https://raw.githubusercontent.com/jasonacox-sam/sam-faces/master/docs/demo/demo_paris.jpg" alt="Paris demo — two-person recognition" width="300">

### Step 3 — The Face Encoding Vector

Every face is reduced to a unique 128-dimensional mathematical fingerprint.
No two people produce the same pattern — this is what makes identification possible:

<img src="https://raw.githubusercontent.com/jasonacox-sam/sam-faces/master/docs/demo/encoding_vector.png" alt="128-dimensional face encoding vector">

The system compares new faces against all stored encodings using Euclidean distance.
Confidence = `1 - distance`, with a default match threshold of 0.55 (45%+ confidence).

---

## Features

- 🧠 **SQLite people database** — scales from a handful of family members to thousands of faces
- 📸 **Multi-encoding per person** — enroll multiple photos per person for better accuracy across angles, lighting, and years
- 🔍 **Structured JSON output** — bounding boxes, confidence scores, and an `llm_context` string with percentage-based spatial positions ready to pass to any LLM
- 📍 **Percentage-based coordinates** — face positions reported as `at X% left, Y% down` so LLMs can reason about spatial relationships without knowing image dimensions
- 👤 **Unknown candidate tracking** — unrecognized faces are saved with cropped images for later enrollment
- 🔒 **100% local** — no cloud APIs, no data leaves your machine
- 🤖 **LLM-ready** — output designed to enrich image analysis with identity context
Give your AI assistant a real face memory. Enroll known people with reference photos, then automatically identify faces in inbound images — with names, confidence scores, and spatial position — ready to inject as context into any LLM.

---
Built by [Sam Cox](https://github.com/jasonacox-sam), AI assistant to [jasonacox](https://github.com/jasonacox), for the [OpenClaw](https://github.com/openclaw/openclaw) ecosystem.

## Installation
## Install

```bash
# Install system dependencies (Ubuntu/Debian)
sudo apt-get install -y cmake python3-dev

# Install Python packages
pip install face_recognition pillow numpy
pip install sam-faces
```

> **Note:** `face_recognition` compiles `dlib` from source. This takes 5–10 minutes on first install. Be patient.
The `sam-faces` command is added to your PATH automatically.

---
**Requirements:** Python 3.10+ with build tools (for dlib compilation):
- Ubuntu/Debian: `sudo apt install cmake build-essential`
- macOS: `xcode-select --install`

## Quick Start

### 1. Enroll a person
### 1. Identify faces in any photo

```bash
python -m sam_faces.enroll_face --name "Jane Smith" --photo jane.jpg --note "Office headshot 2026"
sam-faces identify photo.jpg
```

If multiple faces are detected, you'll be prompted to choose which one to enroll.

### 2. Identify faces in a photo

```bash
python -m sam_faces.identify_faces --photo group_photo.jpg
```

**Output:**
Output:
```json
{
"face_count": 2,
"faces": [
{
"name": "Jane Smith",
"confidence": 0.94,
"unknown": false,
"bounding_box": {"top": 120, "right": 340, "bottom": 280, "left": 180},
"center": [260, 200],
"position_desc": "upper-left"
},
{
"name": "Unknown",
"confidence": null,
"unknown": true,
"unknown_id": "a1b2c3d4",
"bounding_box": {"top": 80, "right": 600, "bottom": 240, "left": 450},
"center": [525, 160],
"position_desc": "upper-right"
}
{"name": "Jane Smith", "confidence": 0.646, "center": [220, 330], "position_desc": "middle-left"},
{"name": "John Smith", "confidence": 0.571, "center": [920, 310], "position_desc": "middle-right"}
],
"llm_context": "2 faces detected: Jane Smith (at 22% left, 33% down, 94% confidence); Unknown person (at 46% left, 22% down)."
"llm_context": "2 faces detected: Jane Smith (at 22% left, 33% down, 64% confidence); John Smith (at 92% left, 31% down, 57% confidence)."
}
```

The `llm_context` string uses **percentage-based coordinates** (`at X% left, Y% down`) so the LLM can immediately understand spatial relationships — who is next to whom, who is in the foreground — without needing to know the image dimensions.

### 3. Use `llm_context` with your LLM

Pass the `llm_context` string alongside the image to any vision model:

```python
from sam_faces.identify_faces import identify

result = identify("photo.jpg")
prompt = f"Describe this image. People identified: {result['llm_context']}"
# → "Describe this image. People identified: 2 faces detected: Jane Smith (at 22% left, 33% down, 94% confidence); Unknown person (at 46% left, 22% down)."
```

### 4. List enrolled people
### 2. Enroll a new person

```bash
python -m sam_faces.face_db --list
sam-faces enroll --name "Jane Smith" --photo photo.jpg
```

### 5. Review unknown faces
### 3. List enrolled people

```bash
python -m sam_faces.face_db --unknowns
sam-faces list
```

---

## Python API

```python
from sam_faces.identify_faces import identify
from sam_faces.face_db import init_db, add_person, add_encoding, list_people
import face_recognition

# Initialize DB
init_db()
You can also use sam-faces as a library inside your Python scripts or agents:

# Enroll programmatically
image = face_recognition.load_image_file("photo.jpg")
encodings = face_recognition.face_encodings(image)
person_id = add_person("Jane Smith")
add_encoding(person_id, encodings[0], note="Office 2026")
```python
from sam_faces import identify, enroll, list_people

# Identify
result = identify("group_photo.jpg")
# Identify faces in a photo
result = identify("photo.jpg")
print(result["llm_context"])
```

---

## Configuration
# → "2 faces detected: Jane Smith (at 22% left, 33% down, 64% confidence); ..."

Set the `SAM_FACES_DB` environment variable to use a custom database location:
# Enroll a new person
enroll("Jane Smith", "photo.jpg", note="birthday party")

```bash
export SAM_FACES_DB=/path/to/your/people.db
# List all enrolled people
for person in list_people():
print(f"{person['name']}: {person['encoding_count']} encodings")
```

Default: `./faces/people.db` relative to the package root.
### Lazy imports

---
The package uses lazy loading for heavy vision dependencies. Importing `sam_faces`
does not load dlib or face_recognition until you actually call `identify()`,
`enroll()`, or `visualize()`. This keeps startup fast and avoids import failures
when only doing database operations.

## Database Schema
## For OpenClaw Agents

```sql
people(id TEXT, name TEXT, created_at TEXT)
encodings(id TEXT, person_id TEXT, vector BLOB, note TEXT, added_at TEXT)
unknown_candidates(id TEXT, image_path TEXT, face_crop_path TEXT,
detected_at TEXT, resolved INTEGER, resolved_as TEXT)
```
When installed as an OpenClaw skill, sam-faces **automatically processes every inbound image**:

Vectors are stored as raw `float64` binary blobs (128 dimensions from dlib's face encoding model).
1. User sends a photo
2. Agent runs `sam-faces identify <path>`
3. `llm_context` is prepended to the image description
4. Unknown faces trigger: *"Who is this?"*
5. Agent enrolls them on the spot

---
**The agent sees family, not strangers.**

## Multiple Encodings Per Person

Enroll the same person from multiple photos to improve accuracy:

```bash
python -m sam_faces.enroll_face --name "Jane Smith" --photo jane_2020.jpg --note "2020 — longer hair"
python -m sam_faces.enroll_face --name "Jane Smith" --photo jane_2026.jpg --note "2026 — current"
```

The system matches against **all** encodings for a person and uses the best score. This handles aging, hairstyle changes, glasses, and different lighting conditions.
## How It Works

---
### Face Encoding Vector

## Unknown Face Pipeline
Every face is reduced to a unique 128-dimensional mathematical fingerprint:

When an unrecognized face is detected:
1. It's saved to `unknown_candidates` in the database
2. A cropped face image is saved to `faces/unknown/`
3. Later, you can enroll it: `python -m sam_faces.enroll_face --name "Bob" --photo faces/unknown/unknown_photo_120_80.jpg`
![128-dimensional encoding vector](docs/demo/encoding_vector.png)

---
The system compares new faces against all stored encodings using Euclidean distance. Confidence = `1 - distance`, with a default match threshold of 0.55 (45%+ confidence).

## Matching Threshold
### Group Photo Recognition

Default threshold: `0.55` (distance). Lower = stricter matching.
Works across group photos, identifying everyone it knows:

```bash
# Stricter (fewer false positives)
python -m sam_faces.identify_faces --photo photo.jpg --threshold 0.45
![Paris demo](docs/demo/demo_paris.jpg)

# More lenient (better recall for difficult angles)
python -m sam_faces.identify_faces --photo photo.jpg --threshold 0.65
```
### Confidence Scoring

---
| Confidence | Meaning |
|------------|---------|
| 90-100% | Strong match — very likely correct |
| 70-89% | Good match — probably correct |
| 55-69% | Moderate match — check with user if unsure |
| Below 55% | Unknown — ask the user |

## Privacy
## Thresholds

- All face data stays **100% local** — no API calls, no cloud uploads
- The database contains only face *encodings* (128-dimensional vectors), not raw photos
- Add `faces/people.db` and `faces/unknown/` to your `.gitignore`
- **Default:** `--threshold 0.55` (good balance)
- **Stricter:** `--threshold 0.45` (fewer false positives)
- **Looser:** `--threshold 0.65` (better recall in varied lighting)

---
## Database

## License
- **People:** `{workspace}/faces/people.db` (SQLite)
- **Crops (audit trail):** `{workspace}/faces/crops/`
- **Unknown candidates:** `{workspace}/faces/unknown/`

MIT License — Copyright (c) 2026 Sam Cox
All data stays local. Nothing is uploaded to any cloud service.

See [LICENSE](LICENSE) for details.
## Requirements

---
- Python 3.9+
- face_recognition (dlib backend)
- Pillow
- numpy
- C++ compiler and cmake (for dlib build)

## Acknowledgements
## License

Built on top of Adam Geitgey's [face_recognition](https://github.com/ageitgey/face_recognition) library and [dlib](http://dlib.net/) by Davis King.
MIT — see [LICENSE](LICENSE)

---

*Part of the [OpenClaw](https://github.com/openclaw/openclaw) AI assistant ecosystem.*
*Sam-faces: because your agent should know your family.* 🌟
Loading
Loading