Skip to content

ARCHITECTURE.md

Latest commit

 

History

History
217 lines (161 loc) · 12.8 KB

ARCHITECTURE.md

File metadata and controls

217 lines (161 loc) · 12.8 KB

GeoReel Architecture

Pipeline Diagram

flowchart TD
    %% Inputs
    GPX([GPX File])
    PHOTOS([Photo Files])

    %% Stage 1 – Parsing
    GPX --> GPXParser[GPX Parser]

    %% Stage 2 – Photo matching
    GPXParser -->|trackpoints| PhotoMatcher[Photo Matcher\ntimestamp / GPS / both]
    PHOTOS -->|EXIF on demand| PhotoMatcher

    %% Stage 3 – External data fetching
    GPXParser -->|bounding box| DEMFetcher[DEM Fetcher\nSRTM]
    GPXParser -->|bounding box| ImgFetcher[Satellite Imagery Fetcher\nESRI / MapTiler / custom XYZ]

    %% Stage 4 – Scene construction
    DEMFetcher -->|elevation grid| SceneBuilder[3D Scene Builder\nterrain mesh + texture]
    ImgFetcher -->|satellite texture| SceneBuilder

    %% Stage 5 – Camera path
    GPXParser -->|trackpoints| CamPath[Camera Path Generator\nposition + orientation]
    PhotoMatcher -->|waypoint positions| CamPath

    %% Stage 6 – Rendering
    SceneBuilder -->|.blend scene| Renderer[Frame Renderer\nBlender headless]
    CamPath -->|camera keyframes| Renderer

    %% Stage 7 – Photo overlay
    PhotoMatcher -->|matched photos + positions| Compositor[Photo Overlay Compositor\nfull-screen inserts + carousel]
    Renderer -->|rendered frames| Compositor

    %% Stage 8 – Final output
    Compositor -->|frame sequence| VideoAssembler[Video Assembler\nFFmpeg]
    VideoAssembler --> OUTPUT([Output Video])
Loading

Architectural Components

1. GPX Parser (core/gpx_parser.py)

Reads the input .gpx file and extracts the ordered list of trackpoints (latitude, longitude, elevation, timestamp). Also derives the bounding box used downstream for DEM and imagery fetching.

2. Photo Matcher (core/photo_matcher.py)

Reads EXIF metadata from photo files on demand (GPS coordinates, capture timestamp) and resolves each photo to its position along the track using one of three strategies:

  • timestamp — closest trackpoint by time delta
  • gps — closest trackpoint by geographic distance
  • both (default) — GPS primary, timestamp fallback; warns when the two disagree beyond a configurable threshold

Photos that fall before or after the track time range are assigned a pre or post position and displayed as a slideshow before/after the fly-through. Outputs a list of MatchResult objects carrying photo_path, trackpoint_index, position (pre/track/post), and any warning or error.

3. DEM Fetcher (core/dem_fetcher.py)

Downloads SRTM elevation tiles (via the srtm-py library) for the expanded track bounding box. Tiles are cached locally (~/.cache/srtm.py/). Exposes a regular ElevationGrid (90 m resolution, float32, row-major) ready for mesh construction. Outlier cells and NoData voids are smoothed before the grid is handed to the scene builder.

4. Satellite Imagery Fetcher (core/satellite/)

Downloads XYZ/TMS tile imagery for the track bounding box at an automatically selected zoom level and stitches tiles into a single RGB texture. Supported providers:

Provider Key required
ESRI World Imagery (default) No
ESRI Clarity No
MapTiler Satellite Yes (free tier)
Custom XYZ URL —

5. 3D Scene Builder (core/scene_builder.py)

Launches Blender headlessly and runs blender_scripts/build_scene.py to:

  • Construct the terrain mesh from the elevation grid and apply the satellite texture
  • Build the animated track ribbon (colour-coded by slope gradient or GPS speed, with an optional self-lit emission mode) with a Build modifier for progressive reveal
  • Place the animated position marker and photo waypoint pins (billboard meshes with camera-facing constraints)
  • Set up sun lighting using computed azimuth/elevation for the track's location and time

The resulting .blend file references the texture tiles as external PNG files and is stored in a temporary directory managed by temp_manager. Stale temporary directories from previous runs are detected and pruned on startup via temp_manager.cleanup_stale(); a custom base directory can be configured in Pipeline Settings → Rendering.

6. Camera Path Generator (core/camera_path.py)

Fits a parametric cubic B-spline through the trackpoints and resamples it at equal arc-length intervals (one sample per frame at the configured speed). Computes per-frame camera position (behind and above the track at a configurable slant distance and tilt), look-at direction (tangent or next-waypoint), and inserts pause keyframes at photo waypoint positions. Outputs a list of CameraKeyframe objects.

Orientation smoothing is applied in two passes:

  1. Gaussian smoothing in unwrapped angle space (arctan2 → np.unwrap → Gaussian filter) to remove high-frequency heading noise from GPS jitter.
  2. Spike filter using a MAD (median absolute deviation) outlier detector: frame-to-frame heading deltas whose magnitude exceeds 6× the median delta are flagged, and the affected frames are replaced with np.interp linear interpolation over the surrounding good frames. This eliminates the abrupt single-frame heading reversals that survive Gaussian smoothing because they span only one or two frames.

7. Frame Renderer (core/frame_renderer.py)

Launches Blender headlessly and runs blender_scripts/render_frames.py, which positions the camera at each keyframe and renders a PNG. Three render engines are supported:

  • Viewport (draft) — EEVEE at 4 TAA samples, no shadow maps, no ambient occlusion, satellite textures downscaled to 50% resolution in VRAM before rendering. This is the primary performance lever for terrain scenes, which are texture-bandwidth-bound rather than compute-bound.
  • EEVEE — full rasterisation at the configured quality level (32/64/128 samples).
  • Cycles — physically-based path tracer at the configured quality level (64/128/256 samples), with automatic GPU detection.

When render/n_segments > 1, the render is split into N sequential Blender passes. Each pass loads only the terrain tiles whose world-space bounding boxes intersect the camera's AABB for that frame range, expanded by render/frustum_margin_km. This keeps per-pass VRAM proportional to the visible terrain fraction. Each Blender process exits completely between segments, fully releasing GPU memory before the next segment starts.

Output PNGs are stored in a temporary directory that is registered on Pipeline.temp_dirs for post-job cleanup.

8. Photo Overlay Compositor (core/photo_compositor.py)

Groups consecutive pause keyframes into blocks and renders them as a photo carousel:

  • Single photo at a waypoint: fade in from terrain → full-screen photo → fade out to terrain
  • Multiple photos at the same waypoint: terrain fade-in on the first, cross-fades between photos, terrain fade-out on the last
  • Letterboxing: blurred photo fill or black bars, preserving aspect ratio
  • Transition: fade (cross-dissolve) or cut (hard edit)

Supports all resolution presets (landscape 16:9, portrait 9:16, square 1:1). Output is stored in a temporary directory registered on Pipeline.temp_dirs for post-job cleanup.

9. Video Assembler (core/video_assembler.py)

Encodes the final frame sequence into a video file using FFmpeg. Configurable container (MKV/MP4), codec (H.264/H.265/AV1), and encoder with automatic detection of available hardware accelerators (NVIDIA NVENC, AMD AMF, Intel QSV, Apple VideoToolbox) and software fallbacks. For MKV output, the source GPX and render settings JSON are attached as named attachments.

Project File Format (.georeel)

A .georeel file is a standard ZIP archive. All saves are atomic: GeoReel writes to a .tmp sibling first, then renames it over the target so the original is never truncated mid-write.

ZIP entry Format Description
manifest.json JSON Format version and save timestamp
project.json JSON All project settings (match mode, output path, photo list, render settings, clip effects, locality names settings)
gpx/track.gpx GPX XML Embedded copy of the source GPX track
photos/*.jpg JPEG Embedded photos, stored under their original filenames
font/title.* TTF/OTF Embedded title font (only when a title overlay is configured)
music/*.mp3 audio Embedded audio files, stored under their original filenames
dem/data.bin raw float32, row-major Elevation grid binary (rows × cols × 4 bytes)
satellite/texture.png PNG, ZIP_STORED Satellite texture (not re-deflated — already compressed)
locality/timeline.json JSON Pre-computed Nominatim locality timeline; present only when a preview has been run and the result not invalidated

locality/timeline.json is an array of objects: [{"frame_start": N, "name": "…", "track_time_s": F}, …]. It is omitted from the archive when the cached timeline has been invalidated (e.g. after changing the GPX track or Nominatim settings).

autosave_tilde (path~) uses the same format and is built by clean rewrite from the base archive — never by ZIP append mode — to prevent duplicate central-directory entries.

Server Architecture

GeoReel uses a client-server architecture: the PySide6 GUI is a thin REST client; all pipeline logic runs inside a FastAPI service (georeel-server).

Startup

GUI starts
  ├─ GET http://localhost:8765/api/v1/health
  │    ├─ 200 OK → attach to existing server
  │    └─ refused → bind random OS port → spawn subprocess
  │         └─ poll /health until ready (10 s timeout)
  └─ at GUI exit: delete workspace → terminate subprocess (if owned)

Components

Component Path Role
server/app.py georeel.server.app FastAPI app factory + lifespan cleanup
server/jobs.py georeel.server.jobs In-memory job registry (UUID → JobRecord)
server/workspace.py georeel.server.workspace Per-session temp directories (WorkspaceManager)
server/routes/*.py — One module per resource group
ui/server_client.py georeel.ui.server_client Synchronous httpx wrapper (GUI side)
ui/server_manager.py georeel.ui.server_manager Subprocess lifecycle + port selection

REST API (all under /api/v1/)

Group Endpoints Type
Health GET /health sync
Workspaces POST /workspaces, DELETE /workspaces/{id} sync
GPX POST /gpx/parse, POST /gpx/clean sync
Photos POST /photos/upload, POST /photos/match sync
Camera POST /camera/keyframes sync
DEM POST /dem/fetch → job, GET /dem/{id}/result async job
Satellite POST /satellite/fetch → job, GET /satellite/{id}/result, GET /satellite/{id}/texture.png async job
Scene POST /scene/build → job async job
Render POST /render/frames → job async job
Compositor POST /compositor/run → job async job
Video POST /video/assemble → job, GET /video/{id}/download async job
Project POST /project/save, POST /project/load sync
Jobs GET /jobs/{id}, DELETE /jobs/{id}, GET /jobs/{id}/events polling / SSE

Job lifecycle

POST /resource/action  →  {"job_id": "..."}
  └─ GET /jobs/{id}   →  {"status": "running", "progress": 42, "message": "..."}
  └─ GET /jobs/{id}   →  {"status": "done", "result_path": "/tmp/..."}

Long-running pipeline stages run in a thread pool (asyncio.to_thread). The job registry holds the cancel_event (threading.Event) checked by each stage at regular intervals. Completed jobs are kept for 30 minutes before eviction.

Workspace lifecycle

A workspace is a server-side temp directory that holds uploaded photos for a session. The GUI creates one workspace on startup and passes its workspace_id to endpoints that need photo files. On GUI exit (closeEvent), the workspace is explicitly deleted; any orphaned workspaces are swept by the lifespan handler on server shutdown.

Data Flow Summary

Stage Input Output
GPX Parser .gpx file trackpoints, bounding box
Photo Matcher trackpoints + photo EXIF (on demand) list[MatchResult]
DEM Fetcher bounding box ElevationGrid (90 m, float32)
Satellite Imagery Fetcher bounding box RGB texture (PNG)
3D Scene Builder elevation grid + texture + match results .blend scene file
Camera Path Generator trackpoints + match results list[CameraKeyframe]
Frame Renderer .blend scene + keyframes PNG frame sequence
Photo Overlay Compositor frames + match results + keyframes merged PNG frame sequence
Video Assembler merged frames .mp4 or .mkv

Temporary File Lifecycle

Directory prefix Contents Cleanup
georeel_scene_* DEM binary, texture PNG, .blend atexit (on app exit)
georeel_frames_* Blender-rendered PNGs Immediately after stage 9 (or on cancel)
georeel_comp_* Composited PNGs Immediately after stage 9 (or on cancel)