Automated Generation of Functionally Complete Assurance Suites for COLREGS-Compliance of Autonomous Surface Vehicles

Research software and MODELS26 artifact evaluation package for generating and evaluating initial scenes for autonomous surface vehicle (ASV) scenario-based assurance. The project implements genetic algorithms, rejection sampling, and evolutionary computation for scene generation using the MSR and DC approaches, with detailed visualization and statistical evaluation tooling.

This repository is supplementary material for the paper Automated Generation of Functionally Complete Assurance Suites for COLREGS-Compliance of Autonomous Surface Vehicles.

Artifact reviewers: see RESPONSE_TO_REVIEWERS.md for a summary of changes made after the initial evaluation.

Image is available at assets/images/msr_only_overview.png

Zenodo archives

Published MODELS26 artifacts on Zenodo (for reviewers and offline use):

Artifact	Zenodo record	Version	License
Software code	10.5281/zenodo.19246756	Version 5.0	MIT
Measurement data	10.5281/zenodo.20792733	Version 1.0	CC-BY 4.0

Download MSR_ASV_SceneGeneration-main.zip from the software record, or clone from GitHub (see Installation). Load measurement files from the data record via Data Manager -> Download from Zenodo, or upload a downloaded .pkl.gz manually.

Response to reviewers

A brief summary of changes made after the MODELS26 artifact evaluation is in RESPONSE_TO_REVIEWERS.md. The same text is linked from the Home page in the Streamlit demonstrator.

Overview

All artifact evaluation workflows are exposed through a browser-based Streamlit demonstrator (src/artifact_ui/). You do not need to run individual Python scripts from the terminal to reproduce the paper experiments: scene generation, hyperparameter tuning, trajectory synthesis, data preparation, plotting, and scenario browsing are available as named pages in the web application.

Long-running operations (scene generation, hyperparameter tuning, trajectory generation, Zenodo download, data compression) run in isolated background worker processes. The UI remains responsive, shows live logs, supports cancellation, and offers download buttons for completed or partially completed results directly in the browser.

Getting started without the paper

You do not need to read the paper first to explore this artifact. The Streamlit UI reproduces the paper's evaluation workflows end-to-end.

What MSR does: MSR (multi-step refinement) generates concrete assurance scenarios from graph-based functional models of COLREGS abstractions. Each scenario is a set of vessels with geometric properties suitable for assurance testing of autonomous surface vehicles (ASVs).

The diagram shows how MSR refines functional equivalence classes (FEC) into functional scenario models (FSM), logical scenario models (LSM), initial scenes (IS), and trajectories via search-based (SB) or CDRS(+PS) constraint satisfaction.

What the UI is for: Browse published measurement data from the paper, visualize COLREGS scenes, run statistical evaluation plots, optionally generate new scenes, and synthesize trajectories: all without editing source code or having to run command line scripts.

Key terms:

Term	Meaning
COLREGS scene	Concrete vessel properties, initial placements, headings, and speeds encoded into a JSON file. The scene represents the first frame of a challenging COLREGS assurance scenario.
FEC	functional equivalence class: a graph-shape category of scenarios relevant for assurance coverage
Approach	A scene-generation algorithm configuration (MSR, DC, genetic search, rejection sampling, etc.)
Measurement record	One evaluation run: approach, seed, vessel count, validity flags, and the best generated scene

Recommended first path (~30 minutes):

Start the demonstrator (Quick start).
Data Manager -> Load: download from Zenodo -> select msr_measurements_for_full_coverage.pkl.gz -> Load Zenodo dataset (plotting and visualization only; 6-vessel cases).
Scenario Browser: inspect the table; render one COLREGS scene.
Functional Models: browse one .problem file for a 2-vessel category (inputs to MSR).
Evaluation Plots: generate one statistical figure.
(Optional) Scene Generation: 1 seed, 1 approach, 2–3 vessels, 1 core -> Compress -> Load.
(Optional) Trajectories: on a 2–3 vessel record from step 6 (not from the MSR Zenodo file).

See Kick-the-tires walkthrough for the full ordered path and RAM and dataset guidance before loading large archives.

Quick start (Docker)

If Docker is installed, the fastest path to the demonstrator is:

Clone the repository, or unzip a downloaded source archive (from GitHub or Zenodo), then from the project root:

git clone https://github.com/PELAB-LiU/MSR_ASV_SceneGeneration.git
cd MSR_ASV_SceneGeneration
docker compose up --build

If you used a zip archive instead of git clone, skip the clone step and cd into the extracted folder before running docker compose up --build.

Open http://localhost:8501 in a web browser.

The first build downloads Python dependencies and may take several minutes. Subsequent starts are much faster.

Installation

Docker (recommended)

Docker provides a reproducible environment matching the MODELS26 artifact evaluation setup: Python 3.12, pinned dependencies, headless plotting (MPLBACKEND=Agg), and preconfigured paths for data and job output.

Prerequisites

Docker Engine 24 or newer
Docker Compose v2 (docker compose, not legacy docker-compose)
4+ CPU cores and 8+ GB RAM recommended (32 GB for full paper-scale runs)
~10 GB free disk for the image, uploaded data, job output, and optional Zenodo download
A modern web browser (Chrome, Firefox, Edge, Safari)

Step-by-step

Obtain the source Clone the repository, or unzip a downloaded source archive (from GitHub or Zenodo), and open a terminal in the project root. With git: If you extracted a zip archive, cd into the extracted folder instead.
Review docker-compose.yml (optional) Default host port mapping is 8501:8501. Data and output are persisted on the host:

Host path	Container path	Purpose
`./data`	`/data`	Uploaded datasets, Zenodo download cache
`./output`	`/output`	Job logs, downloadable archives, job state
`./assets/gen_data`	`/app/assets/gen_data`	Scene generation, hyperparam, and trajectory outputs

Build and start

 docker compose up --build

To run detached in the background: 4. Open the UI Navigate to http://localhost:8501. 5. Stopping Press Ctrl+C in the terminal, or:

Zenodo dataset

The default measurement-data DOI from Zenodo archives is hardcoded in src/utils/artifact_config.py and used by Data Manager -> Download from Zenodo.

Docker troubleshooting

Issue	Suggestion
Port 8501 already in use	Change the port mapping in `docker-compose.yml`, e.g. `"8502:8501"`, then open `http://localhost:8502`.
Build fails on dependency install	Ensure network access; retry `docker compose build --no-cache`.
Upload rejected	Default max upload is 300 MB per file (`STREAMLIT_SERVER_MAX_UPLOAD_SIZE`).
RRT / multiprocessing errors	`shm_size: "2gb"` is set in `docker-compose.yml`; increase if needed on very large trajectory jobs.
Permission errors on `./data` or `./output`	Ensure the Docker user can write to these host directories.

Local installation without Docker

Use this path when Docker is unavailable or when developing changes to the UI. The same Streamlit application runs on the host Python interpreter.

Prerequisites

Python 3.12 (paper measurements used Python 3.12 on Ubuntu 24.04)
pip (bundled with Python)
4+ CPU cores and 8+ GB RAM recommended
A modern web browser

Linux system packages (non-Docker only)

On Debian/Ubuntu Linux, install native libraries required by scipy, pygame, and matplotlib before running the Python dependency installer. The Docker image installs these automatically via apt-get; Windows and macOS pip wheels usually bundle what they need.

sudo apt-get update && sudo apt-get install -y --no-install-recommends \
    libgl1 \
    libglib2.0-0 \
    libgomp1 \
    libsm6 \
    libxext6 \
    libxrender1

A full desktop Ubuntu install may already include these packages. If python scripts/install_dependencies.py succeeds but imports fail with a missing shared-library error (for example libgomp.so or libGL.so), run the command above.

Step-by-step

Obtain the source Clone the repository, or unzip a downloaded source archive (from GitHub or Zenodo), and open a terminal in the project root. With git: If you extracted a zip archive, cd into the extracted folder instead.
Create and activate a virtual environment Linux / macOS: Windows (PowerShell or Command Prompt):
Install pinned dependencies Dependencies are declared in package.json under pythonDependencies. Platform-specific packages (e.g. pywin32 on Windows) are selected automatically. Or, if Node.js is available: Streamlit is installed by the dependency script or can be added explicitly:
Set environment variables Linux / macOS: Windows (Command Prompt): Windows (PowerShell): Windows (Git Bash): use the same export syntax as Linux/macOS: PYTHONPATH is optional for the Streamlit entry point (app.py adds src/ automatically), but other CLI scripts still need it. Create data directories:
Start the demonstrator

 streamlit run src/artifact_ui/app.py

Open the URL printed in the terminal (default http://localhost:8501).

Local upload size limit

Browser uploads are limited to 300 MB per file, configured in .streamlit/config.toml (server.maxUploadSize and server.maxMessageSize). For Docker, the same limits are passed on the Streamlit command line in the Dockerfile.

Running the web demonstrator

Regardless of installation method, the entry point is always:

streamlit run src/artifact_ui/app.py

(In Docker this command is the container CMD.)

Navigation is organized into pages via the sidebar:

Page	Purpose
Home	Artifact introduction, data workflow diagram, time budget table
Functional Models	Browse Refinery `.problem` COLREGS functional model inputs
Scene Generation	Run constraint-satisfaction scene generation (paper approaches)
Data Manager	Load, compress, annotate, unzip evaluation datasets
Trajectories	Headless RRT* trajectory generation for a selected scene
Scenario Browser	Tabular browse + COLREGS scene visualization
Evaluation Plots	Paper-style statistical figures from loaded data
Hyperparam Tuning	NSGA-III hyperparameter search
Hyperparam Evaluation	Rank tuning results from uploaded JSON measurements

The sidebar shows whether a background job is running and provides a Cancel button. The job panel at the bottom of every page streams logs, exposes download buttons when results (or partial results) are available, and lists on-disk output locations (with host paths such as ./output/... when running under Docker Compose).

How the UI works

Background jobs

When you start scene generation, hyperparameter tuning, trajectories, compression, annotation, unzip, or Zenodo download, the UI:

Creates a unique job directory under ARTIFACT_OUTPUT_DIR/jobs/<job_id>/.
Spawns a separate Python subprocess (artifact_ui.job_entry) so Streamlit never blocks.
Writes run.log incrementally and result.json when the job finishes or is interrupted.
Records on-disk output locations immediately when a job finishes (fast feedback), including Docker host paths (./output/jobs/<job_id>/, ./assets/gen_data/..., ./data/full/, etc.). Browser zip downloads are created only on request from the job panel (.pkl.gz files from compress/annotate are available for direct download).

Only one job runs at a time. Controls on other pages are disabled while a job is active.

Cancellation and partial results

Cancelling a running scene generation, hyperparameter tuning, or trajectory job packages whatever measurement data was already written to disk and enables a partial results download. This allows you to inspect intermediate output without waiting for full completion.

Active dataset

Many pages require a loaded dataset (a .pkl.gz file selected in Data Manager -> Load). The UI stores only the file name in session state; the file itself remains on the server under ARTIFACT_DATA_DIR/uploads or the Zenodo cache.

Plots in the browser

Scenario Browser and Evaluation Plots render both a static PNG preview and an interactive Plotly chart. You can download PNG or HTML exports from the plot panel.

UI pages and functionality

Home

Introduces the MODELS26 artifact goals and badge targets (Artifact Evaluated: Reusable, Artifact Available).
Explains the recommended data workflow with a diagram (assets/images/usage.png): generate -> compress -> annotate -> load, or load from Zenodo directly, then analyze.
Lists a kick-the-tires checklist and a global time budget table (worst-case formulas for scene generation, hyperparameter tuning, trajectories, and data utilities).

Functional Models

Browse the bundled Refinery functional models under assets/functional_models/ (no dataset load required):

MSR pipeline diagram (assets/images/msr_only_overview.png): shows where functional scenario models (FSM) sit between equivalence classes and logical models.
Catalog overview: table of vessel/obstacle categories and model counts (7 to 462 files per category).
Category and model selectors: pick a functional equivalence class by folder and filename.
Parsed scenario summary: own ship, target ships, obstacles, and active COLREGS relations extracted from the .problem file.
Declare block and full Refinery source: textual view of the instance specification.
Download selected model (single .problem file), Download category zip, or Open in system editor (local installs; use download in Docker).

MSR scene generation loads these models automatically for the selected vessel count. Use this page to inspect inputs before or after running Scene Generation.

Scene Generation

Runs the core constraint satisfaction evaluation pipeline (formerly evaluation_main.py) with user-selected parameters:

Inputs: Refinery functional models (.problem files under assets/functional_models/). Browse them on the Functional Models page. MSR approaches load one model per functional equivalence class for the selected vessel count.
Outputs: JSON evaluation data (measurement records) under assets/gen_data/, one file per scheduler run. Compress and annotate in Data Manager before loading for analysis.
Approaches: multiselect of paper configurations (MSR/CDRS, search-based NSGA-II/III, rejection sampling, etc.). Each option maps to a row of Table: Configurations of compared approaches in the paper. Non-paper configurations are omitted.
Vessel counts: 2 through 6 (obstacle count fixed at 0 in the artifact UI).
Random seeds: repeat the full batch with different pseudo-random initializations for statistical significance (paper uses 30; use 1 for exploration).
CPU cores: parallel scheduler workers.
Verbose logging: per-scenario FEC progress and solver diagnostics in the job log.
Worst-case / typical time estimates displayed before launch, derived from paper measurement medians (time per newly covered relevant FEC). When no seed among 30 paper runs reached 100% coverage for an approach and vessel count, the full-suite worst case is shown as unbounded with an explanation. Rejection-sampling approaches are also flagged as having no global scheduler timeout.

Example input and output

Concrete example from a MSRRSP (rs-msr) run with 3 vessels, 0 obstacles, and random seed 1234:

Input (functional model): assets/functional_models/all/3vessel_0obstacle_scenarios/functional_model0.problem

Declare block (instance specification):

declare os, ts1, ts2.
!exists(OS::new).
!exists(TS::new).
!exists(StaticObstacle::new).
OS(os).
TS(ts1).
TS(ts2).
default !overtakingToPort(*, *).
default !overtakingToStarboard(*, *).
overtakingToStarboard(ts1, os).
default !crossingFromPort(*, *).
crossingFromPort(os, ts2).
default !vesselType(*, *).
vesselType(os, OSPassengerShip).
default !atDangerousHeadOnSectorOf(*, *).
default !headOn(*, *).

Output (evaluation data): assets/gen_data/MSR_evaluation_3_vessel_0_obstacle_scenarios/RS-MSR/Two_Step_CD_Rejection_Sampling_all/1234/3vessel_0obstacle_2026-07-16T09-05-35.485778.json

A copy is bundled at artifact_sample_data/scene_generation_example/3vessel_0obstacle_2026-07-16T09-05-35.485778.json. The record includes run metadata (algorithm_desc, config_group, evaluation_time, random_seed) and a best_scene field with concrete vessel geometry (positions, headings, speeds, types). Open the file in the repository for the full JSON.

On completion (or cancellation), measurement JSON trees are written under assets/gen_data/ and listed in the job panel. Create a download zip from the job panel only if you need a browser copy.

Separate runs: each job appends to the same folder tree when approach and vessel count match a previous run. Clear assets/gen_data/ (or ./assets/gen_data/ in Docker) before the next generation if you do not want results from separate jobs merged into one compress/download step.

Data Manager

Four tabs cover all dataset I/O:

Compress

Upload measurement files from your computer: individual .json files, .pkl.gz archives, or a .zip of a measurement tree.
Merges many JSON records into a single compressed .pkl.gz suitable for fast loading in the UI.
Download the result as compressed.pkl.gz when the job completes.

Load

Upload a .pkl.gz from your computer for browsing and plotting.
Download from Zenodo: fetches the full published dataset into data/full/ (configurable DOI).
Load Zenodo dataset: registers the downloaded archive as the active dataset. Two archives are available in the Measurement dataset dropdown (each option includes a short description):
- msr_measurements_for_full_coverage.pkl.gz: recommended for plotting and visualization (~26 MB); 6-vessel cases only, not for Trajectories.
- main_measurements.pkl.gz: full paper archive (~241 MB); 16+ GB free RAM recommended; pick 2–3 vessel records for Trajectories.
Shows the active dataset name used by other pages.

Annotate Hash

Adds functional equivalence-class hash fields to each measurement record. Hashing requires a unified pass over the full dataset and is intended to run after compression.
Accepts an uploaded .pkl.gz or the active loaded dataset.
Download the result as annotated.pkl.gz.

Unzip

Expands a compressed .pkl.gz back into individual JSON files under the job folder (useful for external tools or hyperparameter evaluation uploads).
Optional Create unzipped_eval_data.zip for download in the job panel (large archives may take a long time to zip).

Trajectories

Requires an active dataset path (registered in Data Manager -> Load). Supports 2–3 vessel records only in the current artifact version.

Select a record index whose best_scene will be used as the initial configuration. Use output from Scene Generation (2–3 vessels). The Zenodo MSR file (msr_measurements_for_full_coverage.pkl.gz) contains 6-vessel cases and is intended for Scenario Browser and Evaluation Plots, not trajectories.
Runs headless bidirectional RRT (ENABLE_RRT_ANIMATION=false) in a background job.
Verbose logging prints RRT iteration details (closest node, distance to goal).
Download trajectory outputs as a zip archive when the job completes or is cancelled with partial progress.

Scenario Browser

Requires a loaded dataset.

Displays an interactive dataframe of all evaluation records (metrics, approach, vessel count, validity flags, etc.).
Select a row index and click Render COLREGS scene to visualize the best_scene for that record.
Shows a matplotlib top-down COLREGS scene plot plus an interactive Plotly view.
Download PNG or HTML exports of the rendered scene.

Evaluation Plots

Requires a loaded dataset.

Choose one of four paper-style figures (see descriptions below), then click Generate plot.
Displays static matplotlib PNG and an interactive Plotly version; supports PNG/HTML download.

Plot	Description
Relevant Coverage Evolution	Cumulative relevant FEC coverage (%) over wall-clock evaluation time for each paper approach, vessel count, and random seed. Shows how quickly each configuration discovers new functionally relevant equivalence classes.
Relevant Coverage	Final relevant FEC coverage (%) per approach and vessel count after all scheduled scenarios finish. Bars aggregate seeds; asterisks mark statistically significant pairwise differences (Mann–Whitney U with effect size).
Time to 100% Coverage	Wall-clock time until 100% of relevant FECs are covered for each approach and vessel count. Only runs that reach full coverage contribute; compares how long complete assurance-suite generation takes across configurations.
Time Per Eqv Class	Mean evaluation time per newly covered relevant equivalence class (graph-shape hash), averaged over seeds. Lower values indicate faster discovery of distinct functional scenarios; includes statistical comparison markers.

Hyperparam Tuning

Explores NSGA-III hyperparameters (population size, mutation/crossover settings) for DC or MSR search-based configurations:

Max combinations: how many hyperparameter tuples to evaluate. Each combination uses population sizes {2, 4, 8, 15, 30} in order with fixed mutation/crossover parameters, running a full measurement batch across all logical scenarios for the selected vessel count.
Planned combinations expander lists exact settings before you run.
Verbose logging runs combinations sequentially and logs start/finish with remaining count.
Download results zip from the job panel.

Hyperparam Evaluation

Ranks hyperparameter tuning output (formerly evaluate_hyperparameters.py):

Upload JSON measurement files or a zip of a tuning result tree.
Produces a summary dataframe of best configurations.
Download results as hyperparam_evaluation.zip containing a CSV.

This page runs synchronously in the Streamlit process (fast analysis, not a background job).

Recommended data workflow

Scene generation produces many separate JSON files on the server. For practical analysis inside the UI, use a single compressed, annotated .pkl.gz archive:

Usage workflow

Generate your own measurements

Scene Generation: run a job and download the result zip.
Data Manager -> Compress: merge JSON into one .pkl.gz and download.
Data Manager -> Annotate hash: add graph-shape hash fields for functional equivalence-class analysis (must run on the full compressed archive); download when done.
Data Manager -> Load: activate the annotated .pkl.gz as the active dataset.

Use the published Zenodo dataset

Data Manager -> Load: download from Zenodo and load a pre-annotated .pkl.gz (skips compress and annotate).

Analyze the active dataset

Use Scenario Browser, Evaluation Plots, Trajectories, and Hyperparam Evaluation on the loaded data.

Optional utility

Data Manager -> Unzip: export human-readable JSON from a loaded archive.

Kick-the-tires walkthrough (~30 minutes)

Designed to complete on a commodity laptop without full paper-scale runtime:

Start the demonstrator (docker compose up --build or local Streamlit).

Explore published data (fastest)

Data Manager -> Load: download from Zenodo; select msr_measurements_for_full_coverage.pkl.gz for Scenario Browser and Evaluation Plots (6-vessel cases: not for trajectories).
Scenario Browser: inspect the table; render one COLREGS scene.
Functional Models: browse one 2-vessel .problem file (MSR inputs to scene generation).
Evaluation Plots: generate one plot type from the loaded dataset.

Try the generation pipeline (optional)

Scene Generation: minimal run: 1 seed, 1 approach, 2 or 3 vessels, 1 core; download the result zip when the job finishes.
Data Manager -> Compress, then Annotate hash, then Load the processed archive (see recommended data workflow).

Optional

Trajectories on a 2–3 vessel record from your own Scene Generation output (not from the MSR Zenodo file); Hyperparam Evaluation on uploaded tuning JSON; Unzip for human-readable JSON exports.

The Home page lists worst-case time formulas so you can verify estimates before starting long jobs.

Full paper reproduction

Use Scene Generation with paper parameters:

Parameter	Paper value
Seeds	30
Approaches	All paper configurations (`sb-msr2` through `ts-rs` as listed in the UI)
Vessel counts	2, 3, 4, 5, 6
CPU cores	All available

The UI displays worst-case duration before starting (median paper time per FEC × scenarios × seeds / cores, or unbounded when paper runs never reached 100% relevant FEC coverage for that configuration). Rejection-sampling approaches (rs, cd-rs, ts-rs) may also run without a global scheduler timeout and can dominate total wall time.

Paper measurements were taken on Ubuntu 24.04, Python 3.12, 32 GB RAM, Intel Haswell-class CPU.

After generation, follow the recommended data workflow to compress, annotate, and analyze results in Evaluation Plots.

Datasets and Zenodo

Licenses and DOIs are listed under Zenodo archives. The artifact UI does not bundle the full dataset inside the Docker image. Acquire it by:

Data Manager -> Download from Zenodo, or
Uploading a .pkl.gz or JSON measurements downloaded from the measurement-data record.

Hardware and software requirements

RAM and dataset guidance

Scenario	RAM guidance
Plotting & visualization (`msr_measurements_for_full_coverage.pkl.gz`)	8–12 GB free
Full archive browsing (`main_measurements.pkl.gz`)	16+ GB free recommended
Trajectories	2–3 vessel records only; generate via Scene Generation (MSR Zenodo file has 6-vessel cases). Unload dataset from memory before starting the job on large archives.
Full paper reproduction (all approaches, 30 seeds)	32 GB

The UI loads entire .pkl.gz archives into memory for browsing and plots. Trajectory jobs run in a separate worker but still decompress the archive on disk: unloading the in-memory copy before a trajectory job avoids doubling RAM use.

Environment variables

Variable	Default (Docker)	Description
`PYTHONPATH`	`/app/src`	Must include `src/` so imports resolve
`ARTIFACT_DATA_DIR`	`/data`	Uploads and Zenodo cache
`ARTIFACT_OUTPUT_DIR`	`/output`	Job directories and downloadable artifacts
`MPLBACKEND`	`Agg`	Headless matplotlib
`ENABLE_RRT_ANIMATION`	`false`	Disable pygame RRT animation in artifact mode
`STREAMLIT_SERVER_MAX_UPLOAD_SIZE`	`300`	Max browser upload size (MB)
`STREAMLIT_SERVER_MAX_MESSAGE_SIZE`	`300`	Max WebSocket message size (MB)

Output and job artifacts

Location	Contents
`ARTIFACT_OUTPUT_DIR/jobs/<job_id>/run.log`	Timestamped worker log
`ARTIFACT_OUTPUT_DIR/jobs/<job_id>/result.json`	Status, error message, paths to downloadable files
`ARTIFACT_OUTPUT_DIR/jobs/<job_id>/.zip` / `.pkl.gz`	Optional packaged downloads (zip created on demand from the job panel; `.pkl.gz` from compress/annotate available immediately)
`assets/gen_data/`	Generated measurement JSON trees (scene gen, hyperparam, trajectories). In Docker, mounted at `./assets/gen_data` on the host. Clear this folder before a new scene generation if you do not want separate runs to merge.

In Docker with default volumes, host paths are ./output/, ./data/, and ./assets/gen_data/. After each job, the job panel lists the relevant folders.

Project structure

High-level layout of the research codebase (the UI orchestrates these modules; you do not need to invoke them manually):

├── assets/                    # Images, Scenic scenarios, functional models
├── refinery_functional_models/ # Refinery .problem COLREGS specifications
├── src/
│   ├── artifact_ui/           # Streamlit MODELS26 demonstrator (entry: app.py)
│   ├── concrete_level/        # Concrete scenes, trajectories, RRT
│   ├── logical_level/         # Logical scenarios, CSP, evolutionary algorithms
│   ├── functional_level/      # Refinery functional model parsing
│   ├── evaluation/            # Statistical tests (Mann-Whitney, Fisher, etc.)
│   ├── visualization/         # Evaluation plots, COLREGS scene figures
│   ├── scripts/               # Worker backends invoked by artifact_ui jobs
│   └── utils/                 # Configuration, archiving, vessel types
├── docker-compose.yml
├── Dockerfile
├── package.json               # Pinned Python dependencies

Research components (brief)

logical_level/: Scenario synthesis via constraint satisfaction; NSGA-II/III, GA, PSO, differential evolution; rejection sampling with Scenic; aggregate fitness strategies.
functional_level/: Parses Refinery .problem files into functional scenario models used by MSR approaches.
concrete_level/: Concrete scene representation, abstraction to logical/functional levels, trajectory data models, RRT*.
evaluation/: Risk vectors, equivalence classes, Mann-Whitney, Fisher's exact, chi-square / KL divergence, permutation evenness tests.
visualization/: Matplotlib/Seaborn evaluation plots and COLREGS scenario rendering (static + Plotly in the UI).

Badges and licensing

Target badges (see STATUS.md):

Artifact Evaluated: Reusable: Dockerized Streamlit UI; all workflows without editing source; subprocess isolation; worst-case time estimates; Zenodo dataset integration.
Artifact Available: MIT license; archived on Zenodo.

Component	License
Software	MIT (`LICENSE`)
Dataset	CC-BY 4.0 on Zenodo

Citation

When using this artifact, cite the paper and the Zenodo records listed under Zenodo archives:

Paper: Automated Generation of Functionally Complete Assurance Suites for COLREGS-Compliance of Autonomous Surface Vehicles

Resource	Minimum	Recommended (full reproduction)
CPU	4 cores	8+ cores
RAM	8 GB	32 GB
Disk	10 GB free	50+ GB if storing full Zenodo download and generation output
Display	Web browser only	Same
Docker	Engine 24+, Compose v2	Same
Python (non-Docker)	3.12	3.12

Name		Name	Last commit message	Last commit date
Latest commit History 16 Commits
.github/workflows		.github/workflows
.streamlit		.streamlit
artifact_sample_data		artifact_sample_data
assets		assets
refinery_functional_models		refinery_functional_models
scripts		scripts
src		src
tests		tests
.dockerignore		.dockerignore
.flake8		.flake8
.gitignore		.gitignore
AUTHORS.md		AUTHORS.md
CITATION.cff		CITATION.cff
Dockerfile		Dockerfile
LICENSE		LICENSE
README.md		README.md
REQUIREMENTS.md		REQUIREMENTS.md
RESPONSE_TO_REVIEWERS.md		RESPONSE_TO_REVIEWERS.md
STATUS.md		STATUS.md
docker-compose.yml		docker-compose.yml
package.json		package.json
pytest.ini		pytest.ini
requirements-dev.txt		requirements-dev.txt

Uh oh!

Folders and files

Latest commit

History

Repository files navigation

Automated Generation of Functionally Complete Assurance Suites for COLREGS-Compliance of Autonomous Surface Vehicles

Zenodo archives

Response to reviewers

Table of contents

Overview

Getting started without the paper

Quick start (Docker)

Installation

Docker (recommended)

Prerequisites

Step-by-step

Zenodo dataset

Docker troubleshooting

Local installation without Docker

Prerequisites

Linux system packages (non-Docker only)

Step-by-step

Local upload size limit

Running the web demonstrator

How the UI works

Background jobs

Cancellation and partial results

Active dataset

Plots in the browser

UI pages and functionality

Home

Functional Models

Scene Generation

Example input and output

Data Manager

Compress

Load

Annotate Hash

Unzip

Trajectories

Scenario Browser

Evaluation Plots

Hyperparam Tuning

Hyperparam Evaluation

Recommended data workflow

Usage workflow

Generate your own measurements

Use the published Zenodo dataset

Analyze the active dataset

Optional utility

Kick-the-tires walkthrough (~30 minutes)

Full paper reproduction

Datasets and Zenodo

Hardware and software requirements

RAM and dataset guidance

Environment variables

Output and job artifacts

Project structure

Research components (brief)

Badges and licensing

Citation

About

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages