Promote reach-end gridpoints to breakpoints, remove boundary

[jpalm3r]

Important

Stacked on #685 (network-loading) — merge that first. This diff will include #685's changes until then; the description below covers only what this branch adds on top of network-loading.

Summary

Reach-end gridpoint data (e.g. EPANET's Flow/Velocity, or MIKE's reach-end WaterLevel, which is a distinct series from the node's own) was previously held in a boundary dict on NetworkNode that nothing ever read. This PR removes boundary and instead promotes a MIKE reach's first/last gridpoint to an ordinary breakpoint, so that data is reachable through find()/recall()/to_dataframe() like any other break point.

• Res1DReach now includes a reach's own start/end gridpoint in its breakpoints (previously only the intermediate ones), gated by populate_gridpoints like every other breakpoint — reaches=[] now consistently means "no gridpoint data at all," including these.
• Since a reach-end breakpoint sits at the same location as its node, _generate_graph connects them with a zero-length edge, tagged boundary=True and clamped to exactly 0.0 (the raw value is a difference between two independently-sourced numbers, so floating-point noise could otherwise leave a tiny negative weight and break shortest-path algorithms).
• NetworkNode.boundary (and BasicNode/Res1DNode's boundary param) are removed — a breaking change for any existing NetworkNode subclass.
• Reverts an in-progress, broken rewrite of _build_dataframe (25 failed / 29 errors on this branch beforehand) — it turned out not to be needed, since breakpoints already flow into the dataframe through the same mechanism as any other graph node.
• Docs and the one notebook referencing boundary are updated accordingly.

Deliberately out of scope: EPANET. Its reaches have at most one synthetic gridpoint belonging to neither end, so the existing gate (len(reach.gridpoints) > 2) already excludes them — EPANET keeps today's behavior (no breakpoints, length=None) unchanged.

Closes #599.

Related to #680, which stays open: this only reaches MIKE's reach-end data. EPANET's single-gridpoint case needs its own placement rule and is left for a follow-up.

Test plan

• uv run pytest tests/ — 814 passed, 6 skipped
• uv run ruff check / uv run mypy on all touched files
• Manually verified boundary-edge tagging and the floating-point clamp against the real network.res1d/network_cali.res11 fixtures

[Copilot]

Pull request overview

This pull request updates ModelSkill’s network loading and time-series handling to (1) make MIKE reach-end gridpoint data discoverable via standard breakpoint mechanisms (removing the unused NetworkNode.boundary concept), (2) add a quantities filter to Network.from_mike()/from_epanet() for more selective reads, and (3) standardize timezone-aware time series inputs by converting them to UTC timezone-naive datetime64[ns].

Changes:

• Promote MIKE reach start/end gridpoints to ordinary breakpoints and tag node↔breakpoint connections as zero-length boundary=True edges.
• Add quantities filtering to network constructors and propagate it through the Res1D adapter read path.
• Normalize tz-aware time to UTC-naive and extend persistence support to include node-geometry comparers’ raw model data; update docs and tests accordingly.

Reviewed changes

Copilot reviewed 13 out of 13 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
tests/test_timeseries.py	Adds coverage for tz-aware inputs being converted to UTC-naive.
tests/test_network.py	Updates node-count expectations; adds tests for boundary edges, reach-end breakpoint behavior, quantity filtering, and filtered-load correctness.
tests/test_match.py	Adds matching tests for tz-aware vs tz-naive combinations and different timezones.
tests/test_comparercollection.py	Adds regression test for save/load round-trip of node-geometry comparer raw data.
tests/test_comparer.py	Adds regression test for tz-aware from_matched() normalization.
src/modelskill/timeseries/_timeseries.py	Replaces ns-only normalization with timezone + ns normalization and applies it during dataset validation.
src/modelskill/timeseries/_point.py	Switches to the new _normalize_time() helper.
src/modelskill/network.py	Removes NetworkNode.boundary, adds quantities filtering, shared chainage tolerance, and boundary-edge tagging/clamping in graph generation.
src/modelskill/model/adapters/_res1d.py	Implements quantities filtering and shared empty-frame usage for topology-only locations (with one inconsistency flagged in review).
src/modelskill/comparison/_comparison.py	Uses _normalize_time() and enables save/load of raw model data for node-geometry comparers.
docs/user-guide/network.qmd	Updates docs to remove boundary, explain boundary edges, and document quantities.
docs/user-guide/matching.qmd	Documents timezone handling and UTC-naive conversion behavior.

Suppressed comments (1)

src/modelskill/model/adapters/_res1d.py:28

• _simplify_colnames() returns a new empty DataFrame() when node.quantities is empty (e.g. MIKE 11 nodes). This defeats the new “share one empty frame” optimization and causes many unnecessary allocations; it also makes topology-only empties inconsistent with the _EMPTY_DATA sentinel used elsewhere.

    # Some formats keep no timeseries at all on some locations - MIKE 11, for instance,
    # stores everything on reach gridpoints, leaving the nodes empty. Asking mikeio1d
    # for a dataframe there raises, so return an empty one instead.
    if not node.quantities:
        return pd.DataFrame()

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[ecomodeller]

Stepping back from this PR specifically: I'd like to open a discussion about whether the network topology layer belongs in modelskill at all, or whether it should live in mikeio1d.

What prompted it is this PR. The work here is careful and correct — boundary edges, clamping the end segment to an exact zero, promoting reach-end gridpoints to real breakpoints. But nothing in modelskill consumes any of it. NetworkModelResult needs exactly four things from Network: to_dataset(), find(), _reaches and _alias_map. It never reads .graph, never reads an edge length, never reads a boundary tag — grep for .graph outside network.py comes back empty. So we are specifying, testing and maintaining a contract for graph traversal that we never perform. Those semantics only pay off for someone running shortest-path or upstream/downstream tracing, and that someone isn't us.

A few other things point the same way:

• mikeio1d already has experimental.to_networkx() doing the same Res1D → networkx conversion, in 51 lines against our 1255.
• NetworkNode / NetworkReach / ReachBreakPoint have exactly one real adapter (_res1d.py), since MIKE and EPANET both arrive through the same Res1D class. The plug points are there for a second backend that doesn't exist.
• We now hand-parse EPANET .inp files, and ADR-012 accepts that as ours to maintain.
• test_every_mikeio1d_extension_is_accounted_for means a mikeio1d release adding a tenth format breaks our CI. We've taken on their format matrix.

Compare how we treat mikeio (ADR-001): we call mikeio.read() and stop. We don't model dfsu geometry or police its extension list. The precedent argues for a thin adapter over mikeio1d, and this has grown past that.

One possible seam:

• modelskill keeps NodeObservation, ReachObservation, NetworkModelResult, the matching path, and BasicNode/BasicReach for hand-built networks — written against a small protocol of what the comparer actually needs.
• mikeio1d takes the file constructors, the .resx/.inp companions, the extension policy tables and the graph construction with its length and boundary semantics — where the fixtures, the format knowledge and to_networkx already are.

To be clear, this is not a verdict on the code and I don't think it should block this PR. It's a question about the feature as a whole, and it is much cheaper to answer now than after the API is public. Interested in hearing where others land.

/Claude & Henrik😉

[jpalm3r]

Thanks for writing this up — it's a fair point.

The network module has grown significantly, and it makes sense to consider a migration. We actually discussed exactly this at the start of the project with @ryan-kipawa, and I want to involve him to reopen this conversation.

However, I want to contextualize the 51-vs-1255 line comparison, because it isn't comparing the same thing: roughly half (630) of our 1255 lines are code, the rest are docstrings, comments and blank lines. to_networkx is well documented too but, if we compare code to code we get somewhere around: 630 against ~25.

Still a huge difference, but the reason is simply because the current network module in modelskill does more.

to_networkx's actual conversion is four lines. Our equivalent is _generate_graph lines 1071–1083 plus the return — 12 lines. Our twelve do a bit more per line (they deduplicate a node shared by several reaches, attach its timeseries as a node attribute, and carry the reach length onto the edge).

The other ~37 lines of _generate_graph are breakpoint handling. to_networkx's own docstring says "Gridpoints ignored", so that part isn't duplicated work — it's work the other function declines to do.

What the remaining ~580 code lines are:

	what it gives the user	code lines
Protocol + BasicNode/BasicReach	a network that exists without an open Res1D — hand-built from arrays, and the seam for any non-Res1D backend	93
Selective loading (nodes= / reaches= / quantities=)	full topology always, timeseries only where the observations are	105
find() / recall()	(reach, chainage 231.4) ↔ integer alias, with tolerance and "start"/"end"	135
to_dataframe / to_dataset	the xr.Dataset that NetworkModelResult consumes	44
Companion files (.resx, .inp)	merged quantities with same-run validation; reach lengths	93
Format policy	which extensions we read, and why not the others	58
Res1D adapter	column simplification, quantity filter, gridpoint → breakpoint, length resolution	126

Where that leaves the proposal

The pieces you name — the file constructors, the companions, the extension policy, the graph's length and boundary semantics — come to about 470 code lines. Items 1 through 4 above, about 325 lines plus NetworkModelResult, would stay with modelskill.

I am happy to have the discussion now and find the best solution before the release.