All notable changes to this project will be documented in this file.
The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.
- GitHub actions config
- Feature parsing in
AugmentedDiffSourceusesvectorpipe.util.RobustFeatureto tolerate bad geometries in the stream #148. - Receive GPG key while publishing artifacts #138
Pipeline#finalize(vectorTiles, zoom)method to receive the final RDD of generated vector tiles for a zoom levelPipeline.Outputmixin trait that overridesfinalizewith default implementation usingsaveVectorTiles(vectorTiles, zoom, pipeline.baseOutputURI)
VectorPipe.Optionsto support for any square layout level (not just from ZoomedLayoutScheme)Pipeline#baseOutputURImoved toPipeline.Output#baseOutputURI- Updated Geotrellis dependency to 3.5.1
- Improve robustness of functions in
vectorpipe.sources.ChangesetSource
- Catch 403 S3Exceptions when checking minutely diffs in AugmentedDiffSource
- Catch proper AWS SDK v2 NoSuchKeyException when checking minutely diffs in AugmentedDiffSource
- AugmentedDiffSource failed to properly decode from JSON sources
- MicroBatchReader null pointer exception when reading baseURI from DataSourceOptions
vectorpipe.examples: VectorPipe examples moved from https://github.com/azavea/osmesaVectorPipe.defaultSparkSessionWithJTSmethod to construct a VectorPipe tailoredSparkSession. Users with more complicated use cases will still want to manually construct their own session.
This is the first release to depend on GeoTrellis 3.0.
- Streaming sources now fallback to the current remote sequence if no database checkpoint or option can be found
- Depend on Spark 2.4.4
- Depend on GeoTrellis 3.1.0
useCachingoption to VectorPipe.Options allows for persisting to disk. Helps avoid repeated computations.- Functions for converting sequence numbers to timestamps and back for both
changeset replications and augmented diff replications. See
ChangesetSourceandAugmentedDiffSourceinvectorpipe.sources.
- Improved empty geometry handling in UDFs
- RELEASING.md - Instructions for releasing new versions of this project
- Support for semicolon-delimited tag values in UDFs, e.g.
shop=bakery;dairy - Support for
ndsin augmented diff GeoJSON (matchingosm-replication-streams@^0.7.0output) - "Uninteresting" tags are dropped when processing OSM inputs; this will result in fewer point features being generated (as those nodes previously had tags applied).
- Sync with id-area-keys@2.13.0 for determining area-ness of a way.
- Fetch gzipped augmented diff JSON (produced by overpass-diff-publisher)
- Preserve the last-known coordinates of deleted nodes
- Better handling of falsy boolean values in tag UDFs
- Adds
riverbank,stream_end,dam,weir,waterfall, andpressurisedto the list of waterway features - Populates
ndsandmembersfor deleted elements from the previous version
- Resolve commons-io deprecation warnings
- Convert coordinates to Doubles (expected by VP internals) when pre-processing
- Mark all logger vals and some UDF vals as @transient lazy to avoid Spark serialization issues
- Properly strip leading and trailing slashes from S3 URIs when exporting vector tiles