X-ray scattering analysis refactor

.env.example

@@ -1,4 +1,103 @@
-TILED_URI_MASK="http://host.docker.internal:8888/api/v1/metadata/raw/AgB_2024_03_25_10s_2m_mask"
-TILED_API_KEY_MASK=<your-unique-tiled-api-key-for-mask>
-TILED_URI_IMAGES="http://127.0.0.1:8888/api/v1/metadata/raw/"
-TILED_API_KEY_IMAGES=<your-unique-tiled-api-key-for-images>
+# =============================================================================
+# Required Environment Variables
+# =============================================================================
+# These must be set before starting the backend server.
+
+# Tiled Server URL
+# The base URL for the Tiled data server providing scan data.
+SCATTERING_TILED_URL="http://tiled:8888/api/v1"
+
+# Tiled API Key
+# Authentication token for accessing the Tiled server.
+SCATTERING_TILED_API_KEY="your_api_key_here"
+
+# =============================================================================
+# Optional Environment Variables (with defaults)
+# =============================================================================
+# These have sensible defaults but can be tuned for your environment.
+
+# -----------------------------------------------------------------------------
+# Save Results to Tiled (Optional Feature)
+# -----------------------------------------------------------------------------
+# When set, a "Save to Tiled" button appears in the UI, allowing users to
+# save linecut and batch processing results to a writable Tiled container.
+# If not set, the feature is disabled and no save buttons are shown.
+
+# URL of the writable Tiled container for saving results.
+# Use the metadata path to target a specific container, e.g.:
+#   http://tiled:8888/api/v1/metadata/results
+# Or use the bare API URL to write to the root:
+#   http://tiled:8888/api/v1
+# SCATTERING_TILED_RESULTS_URL="http://tiled:8888/api/v1/metadata/results"
+
+# API key for the results Tiled server.
+# SCATTERING_TILED_RESULTS_API_KEY="your_results_api_key_here"
+
+# -----------------------------------------------------------------------------
+# Calibration Tiled Server (Optional Feature)
+# -----------------------------------------------------------------------------
+# When set, calibration parameters and masks can be loaded from a dedicated
+# Tiled server. If not set, calibration loading from Tiled is disabled and
+# users must enter calibration parameters manually or upload mask files.
+
+# URL of the Tiled server containing calibration data (PONI files and masks).
+# SCATTERING_TILED_CALIBRATION_URL="http://tiled-calibration:8888/api/v1"
+
+# API key for the calibration Tiled server.
+# SCATTERING_TILED_CALIBRATION_API_KEY="your_calibration_api_key_here"
+
+# -----------------------------------------------------------------------------
+# Server Configuration
+# -----------------------------------------------------------------------------
+
+# Development mode enables hot reload (default: false)
+# Set to true for local development, false for production/Docker.
+# SCATTERING_BACKEND_DEVELOPMENT=false
+
+# Backend server host address (default: 0.0.0.0)
+# Use 0.0.0.0 to accept connections from any interface.
+# SCATTERING_BACKEND_HOST=0.0.0.0
+
+# Backend server port (default: 8000)
+# SCATTERING_BACKEND_PORT=8000
+
+# -----------------------------------------------------------------------------
+# Cache Configuration
+# -----------------------------------------------------------------------------
+# Larger cache sizes improve performance but use more memory.
+# Each cached image is ~10-50MB depending on detector size.
+
+# Maximum number of processed images to cache (default: 50)
+# SCATTERING_BACKEND_CACHE_IMAGE_SIZE=50
+
+# Maximum number of Q-space matrices to cache (default: 20)
+# Q-matrices are reused when only linecut parameters change.
+# SCATTERING_BACKEND_CACHE_QSPACE_SIZE=20
+
+# Maximum number of GISAXS transforms to cache (default: 20)
+# SCATTERING_BACKEND_CACHE_GISAXS_SIZE=20
+
+# Maximum number of masks to cache per cache type (default: 50)
+# Applies to both Tiled masks and uploaded masks separately.
+# SCATTERING_BACKEND_CACHE_MASK_SIZE=50
+
+# Maximum number of Tiled URI client connections to cache (default: 4)
+# SCATTERING_BACKEND_CACHE_TILED_URIS=4
+
+# -----------------------------------------------------------------------------
+# Batch Processing Configuration
+# -----------------------------------------------------------------------------
+
+# Maximum number of parallel workers for batch processing (default: 16)
+# Higher values improve throughput but increase memory usage.
+# Recommended: Set to number of CPU cores or slightly higher for I/O-bound work.
+# SCATTERING_BACKEND_BATCH_MAX_WORKERS=16
+
+# -----------------------------------------------------------------------------
+# Logging Configuration
+# -----------------------------------------------------------------------------
+
+# Log level for application logging (default: INFO)
+# Valid values: DEBUG, INFO, WARNING, ERROR, CRITICAL
+# Use DEBUG for development, INFO or WARNING for production.
+# SCATTERING_BACKEND_LOG_LEVEL=INFO

.pre-commit-config.yaml

@@ -2,7 +2,7 @@ default_language_version:
     python: python3
 repos:
   - repo: https://github.com/pre-commit/pre-commit-hooks
-    rev: v4.5.0
+    rev: v6.0.0
     hooks:
       - id: trailing-whitespace
       - id: end-of-file-fixer
@@ -13,11 +13,11 @@ repos:
       - id: check-yaml
       - id: debug-statements
   - repo: https://github.com/gitguardian/ggshield
-    rev: v1.25.0
+    rev: v1.46.0
     hooks:
       - id: ggshield
         language_version: python3
-        stages: [commit]
+        stages: [pre-commit]
   # Using this mirror lets us use mypyc-compiled black, which is about 2x faster
   - repo: https://github.com/psf/black-pre-commit-mirror
     rev: 24.2.0
@@ -28,7 +28,19 @@ repos:
     hooks:
       - id: flake8
   - repo: https://github.com/pycqa/isort
-    rev: 5.13.2
+    rev: 7.0.0
     hooks:
       - id: isort
         args: ["--profile", "black"]
+  - repo: local
+    hooks:
+      - id: prettier
+        name: prettier
+        entry: npx --prefix frontend prettier --write
+        language: system
+        files: ^frontend/.*\.(js|jsx|ts|tsx|css|scss|json)$
+      - id: eslint
+        name: eslint
+        entry: npx --prefix frontend eslint --fix --config frontend/eslint.config.js
+        language: system
+        files: ^frontend/.*\.(js|jsx|ts|tsx)$

README.md

@@ -0,0 +1,81 @@
+# XScattering - GISAXS/SAXS Data Analysis
+
+Full-stack scientific application for X-ray scattering data visualization and analysis.
+
+**Stack:** React 18 + TypeScript + Vite | FastAPI + Python 3.13 | Docker Compose
+
+## Features
+
+- **Dual image comparison** with synchronized panning/zooming
+- **Linecut extraction** (horizontal, vertical, inclined) in Q-space
+- **Azimuthal integration** using pyFAI
+- **GISAXS and SAXS modes** with proper Q-space transformations
+- **Batch processing** across multiple scans with real-time progress
+- **Detector mask support** (upload or load from Tiled)
+- **Session persistence** for analysis state
+
+## Quick Start (Docker)
+
+```bash
+# 1. Set environment variables
+export SCATTERING_TILED_URL="http://your-tiled-server:8000/api/v1"
+export SCATTERING_TILED_API_KEY="your-api-key"
+
+# 2. Build and run
+docker-compose up --build
+
+# 3. Access at http://localhost:3000
+```
+
+## Environment Variables
+
+| Variable | Required | Description |
+|----------|----------|-------------|
+| `SCATTERING_TILED_URL` | Yes | Tiled server URL (e.g., `http://localhost:8000/api/v1`) |
+| `SCATTERING_TILED_API_KEY` | Yes | Tiled API authentication token |
+| `SCATTERING_TILED_RESULTS_URL` | No | Writable Tiled URL for saving results (enables "Save to Tiled" buttons) |
+| `SCATTERING_TILED_RESULTS_API_KEY` | No | API key for the results Tiled server |
+| `SCATTERING_BACKEND_DEVELOPMENT` | No | Enable hot reload (default: `false`) |
+| `SCATTERING_BACKEND_HOST` | No | Server bind address (default: `0.0.0.0`) |
+| `SCATTERING_BACKEND_PORT` | No | Server port (default: `8000`) |
+| `SCATTERING_BACKEND_LOG_LEVEL` | No | Logging level (default: `INFO`) |
+
+See `.env.example` for all available configuration options.
+
+## Development (without Docker)
+
+### Backend
+
+```bash
+cd backend
+uv sync                                                  # Install dependencies
+SCATTERING_BACKEND_DEVELOPMENT=true uv run xscattering-backend  # Dev server with hot reload
+```
+
+### Frontend
+
+```bash
+cd frontend
+npm install          # Install dependencies
+npm run dev          # Runs on http://localhost:4000
+```
+
+## Project Structure
+
+```
+├── frontend/           # React app (Vite + TypeScript)
+│   └── src/
+│       └── components/
+│           └── Scattering/  # Main analysis component
+├── backend/            # FastAPI server
+│   └── src/
+│       └── xscattering_backend/
+│           ├── routers/     # API endpoints
+│           ├── cache/       # LRU caches
+│           ├── config/      # Settings and models
+│           └── utils/       # Q-space, linecut extraction
+├── docker-compose.yml
+└── .env.example
+```
+
+## License

backend/.dockerignore

@@ -8,3 +8,4 @@ venv/
 ENV/
 .git
 .gitignore
+*.ipynb

backend/.python-version

@@ -0,0 +1 @@
+3.13

backend/Dockerfile

@@ -12,25 +12,37 @@ FROM python:3.13-slim
 # All subsequent commands will be run from this directory
 WORKDIR /app
 
-# Copy only the requirements file first
+# Install build dependencies
+RUN apt-get update && apt-get install -y --no-install-recommends \
+    build-essential \
+    && rm -rf /var/lib/apt/lists/*
+
+# Install uv
+COPY --from=ghcr.io/astral-sh/uv:latest /uv /uvx /bin/
+
+# Copy dependency files first for better layer caching
 # This takes advantage of Docker's layer caching
-# If requirements.txt doesn't change, Docker will reuse the cached dependencies
-COPY requirements.txt .
-# Install Python dependencies specified in requirements.txt
-# --no-cache-dir reduces the image size by not storing the pip cache
-RUN pip install --no-cache-dir -r requirements.txt
+# If pyproject.toml and uv.lock don't change, Docker will reuse the cached dependencies
+COPY pyproject.toml uv.lock ./
+
+# Install Python dependencies using uv
+# --system installs packages into the system Python rather than creating a virtual environment
+# --locked ensures the exact versions from uv.lock are used
+# --no-cache reduces the image size by not storing the uv cache
+RUN uv sync --no-cache --no-dev
 
 # Copy the rest of the application code
 # This step is done after installing dependencies for better caching
-# When you change your code but not requirements.txt, only this layer is rebuilt
+# When you change your code but not pyproject.toml/uv.lock, only this layer is rebuilt
 COPY . .
 
-# Inform Docker that the container will listen on port 8000
+# Inform Docker/Podman that the container will listen on port 8000
 # This is for documentation - it doesn't actually publish the port
 EXPOSE 8000
 
 # Define the command to run the FastAPI application
+# uv run executes the command in the environment with installed dependencies
 # uvicorn is the ASGI server that will run your FastAPI app
 # --host 0.0.0.0 makes the server available to all network interfaces
 # --port 8000 specifies the port the server will listen on
-CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]
+CMD ["uv", "run", "uvicorn", "xscattering_backend.main:app", "--host", "0.0.0.0", "--port", "8000"]

backend/README.md

@@ -0,0 +1,95 @@
+# XScattering Backend
+
+FastAPI backend for SAXS/GISAXS X-ray scattering analysis.
+
+## Requirements
+
+- Python 3.13+
+- [uv](https://github.com/astral-sh/uv) package manager
+
+## Setup
+
+```bash
+cd backend
+uv sync
+```
+
+## Configuration
+
+Required environment variables:
+
+| Variable | Description |
+|----------|-------------|
+| `SCATTERING_TILED_URL` | Tiled data server URL |
+| `SCATTERING_TILED_API_KEY` | Tiled API key |
+
+Optional (Tiled services):
+
+| Variable | Description |
+|----------|-------------|
+| `SCATTERING_TILED_RESULTS_URL` | Writable Tiled container URL for saving analysis results. Feature disabled if unset. |
+| `SCATTERING_TILED_RESULTS_API_KEY` | API key for the results Tiled server |
+| `SCATTERING_TILED_CALIBRATION_URL` | Tiled server URL for calibration data (PONI files and masks). Feature disabled if unset. |
+| `SCATTERING_TILED_CALIBRATION_API_KEY` | API key for the calibration Tiled server |
+
+Optional (server):
+
+| Variable | Default | Description |
+|----------|---------|-------------|
+| `SCATTERING_BACKEND_DEVELOPMENT` | false | Enable development mode with hot reload |
+| `SCATTERING_BACKEND_HOST` | 0.0.0.0 | Server host |
+| `SCATTERING_BACKEND_PORT` | 8000 | Server port |
+| `SCATTERING_BACKEND_LOG_LEVEL` | INFO | DEBUG/INFO/WARNING/ERROR/CRITICAL |
+| `SCATTERING_BACKEND_BATCH_MAX_WORKERS` | 16 | Thread pool size for batch processing |
+
+Optional (cache sizes):
+
+| Variable | Default | Description |
+|----------|---------|-------------|
+| `SCATTERING_BACKEND_CACHE_IMAGE_SIZE` | 50 | Image cache entries |
+| `SCATTERING_BACKEND_CACHE_QSPACE_SIZE` | 20 | Q-matrix cache entries |
+| `SCATTERING_BACKEND_CACHE_GISAXS_SIZE` | 20 | GISAXS transform cache entries |
+| `SCATTERING_BACKEND_CACHE_MASK_SIZE` | 50 | Mask cache entries (per type) |
+| `SCATTERING_BACKEND_CACHE_TILED_URIS` | 4 | Tiled URI client connection cache entries |
+
+## Run
+
+```bash
+uv run xscattering-backend
+```
+
+Or with uvicorn directly:
+
+```bash
+uv run uvicorn xscattering_backend.main:app --reload
+```
+
+## Docker
+
+```bash
+docker build -t xscattering-backend .
+docker run -p 8000:8000 \
+  -e SCATTERING_TILED_URL=... \
+  -e SCATTERING_TILED_API_KEY=... \
+  xscattering-backend
+```
+
+## API Endpoints
+
+| Endpoint | Method | Description |
+|----------|--------|-------------|
+| `/api/health` | GET | Unified health status for all services |
+| `/api/summary` | GET | Scan metadata for folder |
+| `/api/fetch-scan-image` | GET | Detector image with optional GISAXS transform |
+| `/api/q-space` | GET | Q-matrix computation (SAXS) |
+| `/api/extract-linecut` | POST | Single linecut extraction |
+| `/api/azimuthal-integrator` | GET | 1D azimuthal integration |
+| `/api/batch-all` | POST | Parallel batch processing across scans |
+| `/api/batch-cancel/{batch_id}` | POST | Cancel an active batch job |
+| `/api/get-mask` | GET | Retrieve cached mask data |
+| `/api/upload-mask` | POST | Upload a detector mask file |
+| `/api/load-mask-from-tiled` | GET | Load a mask from Tiled |
+| `/api/load-calibration` | GET | Load calibration parameters and mask from a PONI file in Tiled |
+| `/api/save-linecuts` | POST | Save linecut results to Tiled |
+| `/api/save-batch-results` | POST | Save batch results to Tiled |
+| `/ws/progress` | WS | Real-time progress updates |