Skip to content

extract: scan way nodes once across all extracts in complete_ways pass 1#1

Closed
yuiseki wants to merge 1 commit intomasterfrom
feat/extract-complete-ways-eway-all
Closed

extract: scan way nodes once across all extracts in complete_ways pass 1#1
yuiseki wants to merge 1 commit intomasterfrom
feat/extract-complete-ways-eway-all

Conversation

@yuiseki
Copy link
Copy Markdown
Owner

@yuiseki yuiseki commented Apr 29, 2026

See osmcode#312.

In Pass 1 of --strategy=complete_ways, eway() is called independently for
each extract on every way. This means way.nodes() is scanned up to twice per
extract, giving up to 2N scans total for N extracts.

This adds an eway_all() override in strategy_complete_ways that scans
way.nodes() at most twice regardless of N. A uint64_t bitmask tracks which
extracts have claimed the way in the first pass; the second pass then records
all node refs into extra_node_ids for matched extracts only.

When there are more than 64 extracts the method falls back to the original
per-extract eway() loop.

Benchmark

japan-260423.osm.pbf (~1 GB), 8-tile extraction:

Version Elapsed vs baseline
upstream 1m 29s
this PR 1m 25s -4%

planet-260413.osm.pbf (~86 GB), 16-tile extraction:

Version Elapsed vs baseline
upstream 51m 36s
this PR 49m 27s -4%

Output verified to be identical to the upstream result for all tiles.

@yuiseki
Copy link
Copy Markdown
Owner Author

yuiseki commented Apr 30, 2026

Opened by mistake against my own fork. The actual PR is osmcode#314.

@yuiseki yuiseki closed this Apr 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant