🦉 Walnut Creek Critters — wildlife pipeline

A multi-worker pipeline that turns raw trail-cam clips into a curated wildlife livestream. It filters footage for real animals (and never humans), cleans the audio, identifies species, writes timed captions, and produces highlights, compilations, and an optional RTMP stream — running on a single RTX 4090, with a web dashboard to watch it all.

Personal/hobby project, tuned for one GPU host (Ubuntu + RTX 4090). It's shared as a reference, not a turnkey product — paths, models, and camera details are deployment-specific.

---

How it works

Clips are dropped into a per-area inbox (an area is a stream location fed by one or more cameras) and flow through independent worker stages, each watching a directory and handing off to the next:

inbox/<area>/  ─ingest─▶  01_gating  ─▶  02_audio  ─▶  03a_species_visual ─▶ 03b_species_audio
                          (MegaDetector:    (YAMNet:        (BioCLIP:            (BirdNET,
                           keep animals,     mute unwanted   per-segment          optional)
                           cut humans +      sounds)         species timeline)
                           shaky sections)                        │
                                                                  ▼
   archive/ ◀─ 05_ready ◀──────────────── 04_captions  ◀─────────┘
   used/         │                         (timed SRT/VTT)
   (compilations)└─▶ 04b_highlights ─▶ highlights/

• Human exclusion is a hard rule — the gate cuts any segment with a person (aggressive person_conf), so people never reach the output.
• Quality gating — shaky handheld sections are cut (per-frame camera-jerk), not whole clips; static tripod footage passes untouched.
• Species ID — BioCLIP constrained to a local candidate list (Central Texas), written as a per-segment timeline so captions change as animals come and go.
• Highlights — each notable moment (allowlisted or first-seen species) becomes its own short clip + a 9:16 vertical crop centered on the subject.
• Compilations — crossfaded montages rendered on demand; used clips move to used/ so each compilation pulls fresh footage.
• Streaming — optional RTMP push (YouTube/Twitch); archive-only by default.

GPU offload

Video decode runs on NVDEC (explicit *_cuvid decoders) and encode on NVENC, with automatic CPU fallback — freeing the CPU that used to be the bottleneck. MegaDetector runs batched. Clips wider than the 4096px H.264 GPU cap (e.g. 5K) are downscaled to 4K so they stay on the GPU. See DESIGN.md §8.1.

Dashboard

A FastAPI + HTMX web UI (http://<host>:8080/):

Tab	What
Stats	Lifetime totals — footage kept/cut, species counts, highlights, compilations (auto-refreshing).
Status	Live per-area / per-stage queue depths, failed clips (retrigger), recent output.
Videos	Browse + play everything; Generate compilation button; Rejects tagged with why.
Camera	Embedded AXIS Q1659 controls (exposure / focus / white balance + live preview).
Config	Edit config.yaml as a form or raw YAML.
Fresh Start	Reset & re-run (reprocess footage) or nuclear wipe, both behind a confirm.

Requirements

• Linux + NVIDIA GPU (built for an RTX 4090; NVIDIA driver ≥ CUDA 12.1).
• Python ≥ 3.12, uv, ffmpeg (with NVDEC/NVENC), supervisor.

Setup

# 1. Install (host, with GPU/ML deps — torch is pinned to a CUDA 12.1 build)
uv sync --extra ml

# 2. Models — place weights in models/ (paths set in config.yaml `models:`):
#    MegaDetector v5, BioCLIP (auto-downloaded), YAMNet (+ class map), BirdNET tflite

# 3. Config — edit config.yaml (paths, areas, species candidates, gate tuning).

# 4. Secrets — copy .env.example to .env for stream keys; the AXIS camera
#    password comes from the AXIS_PASS env var. Neither is committed.
cp .env.example .env

Run (supervisord)

The 10 workers and the dashboard run under supervisord:

bash deploy/supervisor/install-supervisor.sh     # install + deploy + start

sudo supervisorctl status                          # workers + dashboard
sudo supervisorctl restart wildlife:*              # all workers
sudo supervisorctl restart wildlife:animal-gate    # one worker
sudo supervisorctl restart dashboard               # the web UI

• Dashboard: http://<host>:8080/
• Supervisor web UI: http://<host>:9999/ (change the default auth in deploy/supervisor/web.conf)

See deploy/supervisor/README.md for details. A Docker Compose setup for the dashboard is also included as an alternative.

Repository layout

src/wildlife/
  cli.py            # `wildlife run <worker>` / `wildlife dashboard` / `wildlife render-file`
  workers/          # ingest, animal_gate, audio_cleanup, species_visual/_audio,
                    # caption_assembler, highlight_curator, stream_compiler, sd_import, resmon
  common/           # pipeline_fs (stage plumbing), ffmpeg_ops (GPU offload), config
  dashboard/        # FastAPI app, templates, axis.py (camera proxy)
deploy/supervisor/  # supervisord configs + install script
tools/              # standalone AXIS camera control GUI
config.yaml         # all tunables (paths, areas, gate, species, gpu, stream, …)
DESIGN.md           # full architecture / design document

More info

DESIGN.md is the detailed design doc (per-stage behavior, GPU offload, failure handling, dashboard internals) — also viewable in the dashboard's More info tab.