feat: add Datomic Free→Pro migration tooling

Bare-metal script (scripts/migrate-db.sh) wraps bin/datomic CLI for
backup/restore/verify/list with auto-detection of Free vs Pro
distributions. For backup, searches lib/ for an extracted Free
distribution, prompts interactively if not found, extracts the bundled
tarball (lib/datomic-free-0.9.5703.tar.gz) as last resort.

Docker wrapper (docker-migrate.sh) runs the same operations in
temporary containers, auto-detecting images and Compose networks.

Includes full migration docs (runbook + in-depth explanation),
devcontainer Dockerfile simplification (pre-cache zip only, let
post-create.sh handle install), and datomic container unzip fix.

Author: codeGlaze

.devcontainer/Dockerfile

@@ -16,62 +16,15 @@ RUN curl -fsSL https://raw.githubusercontent.com/technomancy/leiningen/stable/bi
 
 WORKDIR /workspace
 
-# Copy project.clj first (needed to know version)
-COPY project.clj /workspace/
-
-# Copy existing lib/ directory first (preserves pdfbox and other vendor deps)
-# Create lib/ directory - COPY will handle copying if lib/ exists in build context
-RUN mkdir -p /workspace/lib/
-COPY lib/ /workspace/lib/
+# Copy project source (lib/ includes pdfbox vendor deps from host)
+COPY . /workspace
 
-# Download and install Datomic Pro to lib/ using existing file:lib repository pattern
-# Latest version: https://docs.datomic.com/releases-pro.html
+# Pre-cache Datomic Pro zip so post-create.sh skips the download.
+# The actual install (extract, flatten, maven-install) is done by post-create.sh
+# because the volume mount overwrites /workspace during container creation.
 ARG DATOMIC_VERSION=1.0.7482
-RUN echo "[DOCKER BUILD] Starting Datomic Pro ${DATOMIC_VERSION} installation..." && \
-    mkdir -p lib/com/datomic/datomic-pro/${DATOMIC_VERSION} && \
-    echo "[DOCKER BUILD] Downloading Datomic Pro zip..." && \
-    curl -L -o /tmp/datomic-pro-${DATOMIC_VERSION}.zip \
-    "https://datomic-pro-downloads.s3.amazonaws.com/${DATOMIC_VERSION}/datomic-pro-${DATOMIC_VERSION}.zip" && \
-    echo "[DOCKER BUILD] Extracting zip..." && \
-    unzip -q /tmp/datomic-pro-${DATOMIC_VERSION}.zip -d /tmp/datomic-extract && \
-    echo "[DOCKER BUILD] Listing extracted contents..." && \
-    find /tmp/datomic-extract -type f -name "*.jar" | head -5 && \
-    # Copy entire distribution into vendor layout so POMs and support files are available
-    EXTRACTED_DIR=$(find /tmp/datomic-extract -maxdepth 1 -type d ! -name . | head -1) && \
-    mkdir -p lib/com/datomic/datomic-pro/${DATOMIC_VERSION} && \
-    if [ -n "${EXTRACTED_DIR}" ]; then \
-      echo "[DOCKER BUILD] Copying extracted distribution from ${EXTRACTED_DIR} -> lib/com/datomic/datomic-pro/${DATOMIC_VERSION}" && \
-      cp -r "${EXTRACTED_DIR}"/* "lib/com/datomic/datomic-pro/${DATOMIC_VERSION}/" && \
-      # If a peer jar exists in the distribution, also ensure datomic-pro-<version>.jar exists for vendor layout
-      if [ -f "lib/com/datomic/datomic-pro/${DATOMIC_VERSION}/peer-${DATOMIC_VERSION}.jar" ]; then \
-        cp "lib/com/datomic/datomic-pro/${DATOMIC_VERSION}/peer-${DATOMIC_VERSION}.jar" "lib/com/datomic/datomic-pro/${DATOMIC_VERSION}/datomic-pro-${DATOMIC_VERSION}.jar"; \
-      fi; \
-    else \
-      # Fallback to copying a single jar if extraction produced unexpected layout
-      DATOMIC_POTENTIAL_JAR=$(find /tmp/datomic-extract -type f -name 'peer*.jar' -print -quit || true) && \
-      if [ -z "${DATOMIC_POTENTIAL_JAR}" ]; then DATOMIC_POTENTIAL_JAR=$(find /tmp/datomic-extract -type f -name 'datomic-pro-*.jar' -print -quit || true); fi && \
-      if [ -z "${DATOMIC_POTENTIAL_JAR}" ]; then echo "[DOCKER BUILD] ERROR: No Datomic JAR found in extracted zip" && find /tmp/datomic-extract -type f | head -20 && exit 1; fi && \
-      cp "${DATOMIC_POTENTIAL_JAR}" "lib/com/datomic/datomic-pro/${DATOMIC_VERSION}/datomic-pro-${DATOMIC_VERSION}.jar"; \
-    fi && \
-    echo "[DOCKER BUILD] Verifying vendor path contents..." && \
-    ls -lh "lib/com/datomic/datomic-pro/${DATOMIC_VERSION}/" && \
-    rm -rf /tmp/datomic-pro-${DATOMIC_VERSION}.zip /tmp/datomic-extract && \
-    echo "[DOCKER BUILD] ✅ Datomic Pro installed to lib/ (file:lib repository pattern)"
-
-# Copy the rest of the source (exclude lib/ since we've already set it up)
-# Use .dockerignore or copy selectively to avoid overwriting lib/
-COPY . /workspace
-# Ensure Datomic Pro JAR is still present after COPY
-RUN test -f lib/com/datomic/datomic-pro/${DATOMIC_VERSION}/datomic-pro-${DATOMIC_VERSION}.jar || \
-    (echo "[DOCKER BUILD] ⚠️  Datomic Pro JAR missing after COPY, re-downloading..." && \
-     mkdir -p lib/com/datomic/datomic-pro/${DATOMIC_VERSION} && \
-     curl -L -o /tmp/datomic-pro-${DATOMIC_VERSION}.zip \
-     "https://datomic-pro-downloads.s3.amazonaws.com/${DATOMIC_VERSION}/datomic-pro-${DATOMIC_VERSION}.zip" && \
-     unzip -q /tmp/datomic-pro-${DATOMIC_VERSION}.zip -d /tmp/datomic-extract && \
-     DATOMIC_JAR=$(find /tmp/datomic-extract -name "datomic-pro-*.jar" -type f | head -1) && \
-     cp "${DATOMIC_JAR}" "lib/com/datomic/datomic-pro/${DATOMIC_VERSION}/datomic-pro-${DATOMIC_VERSION}.jar" && \
-     rm -rf /tmp/datomic-pro-${DATOMIC_VERSION}.zip /tmp/datomic-extract && \
-     echo "[DOCKER BUILD] ✅ Datomic Pro JAR restored")
+RUN curl --fail -L -o "/tmp/datomic-pro-${DATOMIC_VERSION}.zip" \
+      "https://datomic-pro-downloads.s3.amazonaws.com/${DATOMIC_VERSION}/datomic-pro-${DATOMIC_VERSION}.zip"
 
 # Default shell to bash for VS Code terminal
 SHELL ["/bin/bash", "-c"]

.dockerignore

@@ -8,4 +8,8 @@ test/*
 *.md
 data/*
 log/*
-backups/*
+backups/*
+backup/*
+
+# Datomic Pro distribution (~280MB) — Docker images download their own from S3
+lib/com/datomic/

README.md

@@ -73,15 +73,16 @@ For the full walkthrough (including manual setup without scripts), see **[docs/G
 
 ### Self-hosting (Docker)
 
-For running your own production instance from pre-built images:
+For running your own production instance:
 
-1. Clone the repository
-2. Copy `.env.example` to `.env` and edit the values (see [docs/ENVIRONMENT.md](docs/ENVIRONMENT.md))
-3. Create SSL certificates: `./deploy/snakeoil.sh`
-4. Run: `docker-compose pull && docker-compose up`
-5. Visit `https://localhost`
+```bash
+git clone https://github.com/orcpub/orcpub.git && cd orcpub
+./docker-setup.sh           # generates .env, SSL certs, directories
+docker compose up -d        # pull images and start
+./docker-user.sh init       # create admin from .env settings
+```
 
-See the [Docker deployment section](#docker-deployment) for full details.
+Visit `https://localhost`. See the [Docker deployment section](#docker-deployment) for full details including migration from older versions.
 
 ---
 
@@ -293,36 +294,84 @@ For self-hosting a production instance.
 
 ### Containers
 
-The Docker setup runs three containers:
-
 | Container | Purpose |
 |-----------|---------|
-| `datomic` | Datomic database transactor |
-| `orcpub` | JVM application server |
-| `web` | nginx reverse proxy with SSL |
+| `datomic` | Datomic Pro database transactor |
+| `orcpub` | JVM application server (Java 21) |
+| `web` | nginx reverse proxy with SSL termination |
+
+### Fresh Install
+
+```bash
+git clone https://github.com/orcpub/orcpub.git && cd orcpub
+
+# Interactive setup — generates .env, SSL certs, and directories
+./docker-setup.sh
+
+# Pull pre-built images and start
+docker compose up -d
+
+# Create your first user (once containers are healthy)
+./docker-user.sh init                                   # from .env settings
+./docker-user.sh create <username> <email> <password>   # or directly
+```
+
+Visit `https://localhost` when running.
+
+To build from source instead of pulling images:
+
+```bash
+docker compose -f docker-compose-build.yaml build
+docker compose -f docker-compose-build.yaml up -d
+```
+
+For environment variable details, see [docs/ENVIRONMENT.md](docs/ENVIRONMENT.md).
+
+### Upgrading from Datomic Free (pre-2026)
 
-### Setup
+If you have an existing deployment using the old Java 8 / Datomic Free stack,
+your `./data` directory is **not compatible** with the new Datomic Pro transactor.
+The storage protocols (`datomic:free://` vs `datomic:dev://`) use different formats.
 
-1. Clone the repository
-2. Copy `.env.example` to `.env` and configure (see [docs/ENVIRONMENT.md](docs/ENVIRONMENT.md))
-3. Create SSL certificates:
-   - Quick self-signed: `./deploy/snakeoil.sh` (unix) or `./deploy/snakeoil.bat` (windows)
-   - Or use your own certificate
-4. Launch:
+**You must migrate your database before upgrading.** The migration tool handles this:
+
+**Bare metal** — use `scripts/migrate-db.sh` which wraps the `bin/datomic` CLI:
 
 ```bash
-docker-compose pull    # use pre-built images
-docker-compose up -d   # start in background
+./scripts/migrate-db.sh backup                    # With old (Free) transactor running
+# ... stop Free transactor, move ./data aside, start Pro transactor ...
+./scripts/migrate-db.sh restore "datomic:dev://localhost:4334/orcpub?password=..."
+./scripts/migrate-db.sh verify
 ```
 
-To build from source instead:
+**Docker** — use `docker-migrate.sh` which runs `bin/datomic` inside containers:
 
 ```bash
-docker-compose -f docker-compose-build.yaml build
-docker-compose -f docker-compose-build.yaml up -d
+./docker-migrate.sh backup        # With old stack running
+docker compose down
+docker compose -f docker-compose-build.yaml build
+docker compose -f docker-compose-build.yaml up -d
+./docker-migrate.sh restore       # After new stack is healthy
+./docker-migrate.sh verify
 ```
 
-Visit `https://localhost` when running.
+Or run `./docker-migrate.sh full` for a guided migration.
+
+The backup is storage-protocol-independent and writes to `./backup/`, so databases
+of any size (including 20GB+) are handled. See [docs/migration/datomic-data-migration.md](docs/migration/datomic-data-migration.md)
+for the full guide including disk space planning and troubleshooting.
+
+### User Management
+
+```bash
+./docker-user.sh create <user> <email> <password>   # Create a verified user
+./docker-user.sh batch users.txt                     # Bulk create from file
+./docker-user.sh list                                # List all users
+./docker-user.sh check <user>                        # Check user status
+./docker-user.sh verify <user>                       # Verify unverified user
+```
+
+See [docs/docker-user-management.md](docs/docker-user-management.md) for details.
 
 ### Importing Homebrew Content
 
@@ -333,6 +382,15 @@ Place your `.orcbrew` file at `./deploy/homebrew/homebrew.orcbrew` — it loads
 - **Database**: Stored in `./data/`. Back up this directory when Datomic is stopped. Delete it to start fresh.
 - **Logs**: Stored in `./logs/`. Safe to clean up; does not affect character data. Set up log rotation for production.
 
+### Scripts Reference
+
+| Script | Purpose |
+|--------|---------|
+| `scripts/migrate-db.sh` | Migrate data from Datomic Free to Pro (bare metal) |
+| `docker-migrate.sh` | Migrate data from Datomic Free to Pro (Docker) |
+| `docker-setup.sh` | Generate `.env`, SSL certs, and directories |
+| `docker-user.sh` | Create, verify, and list users in the database |
+
 ---
 
 ## Architecture