Skip to content

CHANGELOG.md

Latest commit

 

History

History
145 lines (118 loc) · 6.44 KB

CHANGELOG.md

File metadata and controls

145 lines (118 loc) · 6.44 KB
Raw

Changelog

All notable changes to this benchmark are listed here. Dates are UTC.

v2 final — 2026-06-17 (HF simplification)

Following user feedback for an open-source-friendly layout, the HF dataset was simplified to one directory per (subset, track):

  • The 4 stage-17 OSM-temporal-consistency flag columns (osm_temporal_consistent, osm_max_inland_depth_m, osm_n_inland_points, osm_max_consec_inland_run) are merged inline into <subset>_track_v1/{train,val,test}/part-000.csv.gz.

  • For Piraeus, the inline flag uses the 2020-01-01 OSM snapshot (matches the 2019 AIS year). The 2026-OSM flag remains in the side-car osm_temporal_consistency/ for ablation.

  • The 16 redundant sibling directories (<subset>_track_v1_all/, <subset>_track_v1_filtered/ × 4 subsets × 2 tracks) were removed from HF. The same row sets are recoverable in one line of pandas:

    df = pd.read_csv("<subset>_track_v1/train/part-000.csv.gz")
df_filtered = df[df["osm_temporal_consistent"] == "true"]

  • scripts/extras/publish_dual_subset.py is kept for historical reproducibility but marked deprecated.

v2 — 2026-06-14

Notice to downstream authors using the Norway subset

The Norway environmental context (context_v1/environment/) has been substantively updated. If your trained model relies on the per-sample SDF / raster / scene-type fields for Norway, re-evaluate against the new context. The corresponding tensors at rasters/{split}/{masks,signed_dist_shore,signed_dist_nav}.npy and all_environment_descriptors.csv have all changed for ~85 % of Norway samples. Pre-v2 numbers for Norway env-aware models are no longer comparable to v2 numbers. The Track A trajectory CSVs and the env-free models are unaffected.

Corrections to documentation

  • Time spans in the original 2026-05-21 REPORT.md were inaccurate. Verified time spans (from train/part-000.csv.gz first-row hist_end_ts):
    • DMA = 2025-09-01 ~ 2025-09-30 (was claimed: 2019)
    • NOAA = 2025-03-01 ~ 2025-03-31 (was claimed: 2015–2019)
    • Piraeus = 2019-01-01 ~ 2019-12-26 (was partly claimed: 2019-12 only)
    • Norway = 2025-08-01 ~ 2025-09-30 (unchanged)
  • Piraeus upstream license corrected: CC BY 4.0, not CC BY-NC-SA 4.0 (verified from Zenodo 10.5281/zenodo.6323416 metadata).

New files

  • LICENSE, NOTICE.md, DATA_CARD.md, CITATION.cff, SUMMARY_v2.md
  • Piraeus_ship_trajectory_datasets/LICENSE (CC BY 4.0)
  • norway_ship_trajectory_datasets/LICENSE (NLOD 2.0)
  • NOAA_ship_trajectory_datasets/LICENSE (US public domain notice)
  • scripts/extras/stage_17_osm_temporal_consistency.py
  • scripts/extras/pbf_to_tile_cache.py
  • scripts/extras/publish_dual_subset.py
  • scripts/extras/post_rebuild_refresh.sh
  • scripts/extras/verify_v2_artifacts.py

New data artefacts per subset

  • multi_type_mini_bench_build/standard_track_v1/osm_temporal_consistency/ — Stage-17 per-sample flag CSVs.
  • multi_type_mini_bench_build/standard_track_v1_all/ — Original benchmark rows with 4 extra flag columns.
  • multi_type_mini_bench_build/standard_track_v1_filtered/ — Subset where osm_temporal_consistent = true. Recommended default for paper main tables.

Piraeus-only

  • data_raw/historical_osm/greece-200101.osm.pbf (185 MB, Geofabrik archive).
  • Track A: multi_type_mini_bench_build/standard_track_v1/context_v1_2019osm/ — Env / SDF / raster stack rebuilt against the 2020-01-01 OSM snapshot for temporal alignment with the 2019 AIS year.
  • Track B: track_b/Piraeus/multi_type_mini_bench_build/standard_track_v1/context_v1_2019osm/ — Same rebuild applied to the long-horizon track (1,154 samples). Build wall time: 10 min.
  • Stage-17 flag CSVs against 2019-OSM at osm_temporal_consistency_2019osm/ for both Track A and Track B. These drive Piraeus's default standard_track_v1_filtered/.
  • Per-sample comparison of the 2026-OSM vs 2019-OSM consistency flags:
    • Track A: train 8 / val 0 / test 4 samples flipped consistent under 2019 OSM (i.e. the 2026 OSM incorrectly flagged them as inland due to post-2020 port construction).
    • Track B: 0 samples changed (Track B trajectories happen to stay clear of the contested coastal cells; we ship the historical context anyway for ablation symmetry).

DMA-only

  • data_raw/dma/historical_osm/denmark-260101.osm.pbf (459 MB).
  • 188 → 2 failed OSM tiles after Geofabrik refill + env rebuild.
  • Top-level context_v1/summary.json now correctly serialises the environment section (previously empty {} due to a cosmetic bug in the summary writer — the underlying data was always at environment/summary.json and was correct).

Norway-only

  • data_raw/historical_osm/norway-260101.osm.pbf (1.36 GB).
  • 184 → 2 failed OSM tiles after refill + env rebuild.
  • Scene-counts on train split rebalanced from open_water 47,695 / nearshore 168 / harbor 0 / constrained 137 to open_water 34,888 / nearshore 10,113 / harbor 1,320 / constrained 1,679, reflecting the true fjord / coastal distribution that had been hidden by the silently-failing OSM tiles.
  • The pre-rebuild context is no longer kept on disk for Norway (overwritten by the v2 rebuild). The pre-rebuild failed-tile list is preserved at failed_tiles.pre_refill.json.

Known limitations carried into v2

  • 2 DMA + 2 Norway OSM tiles still fail Overpass (429/504 throttle). The affected anchors are all in the open North Sea or Baltic with no expected coastline features; their SDFs read as uniformly +5000 m (open water), which is the correct outcome for the anchor locations.
  • The Track B contexts were built later when Overpass was stable (failed_tiles = 0 for DMA / Piraeus / Norway, = 17 for NOAA). They were NOT rebuilt as part of v2; their existing SDFs and stage-17 flags are kept as-is.

Build wall times

Job Elapsed wall
DMA env rebuild (150 K samples) 6.5 h
Piraeus 2020-01 OSM rebuild (60 K) 8.6 h
Norway env rebuild (60 K) 10.3 h
Stage 17 + dual-subset publish < 5 min

Total wall time (3-way parallel + final stage): ~10.5 h.

v1.x — earlier history

See REPORT.md and PIPELINE_NOTES.md for the pre-v2 history. Note that several specific numeric / license claims in those documents are superseded by v2 — REPORT.md carries an inline v2-update banner at the top that lists the corrections.