OpenStreetMap PBF history parser

This Rust program:

• parses OSM history files from Geofabrik
• then creates a postgres {schema}.{country_code}_history table with the following fields:
  • id (negative IDs are for relations)
  • timestamps
  • changesets
  • first_timestamps
  • last_timestamps
  • users_number
  • versions_number

All countries share a single schema huwise_osm, with tables prefixed by country code (e.g. fr_history, au_history).

---

Usage

./pbf_history_reader HISTORY_OSH_PBF_FILE TAG_LIST_FILE [COUNTRYCODE] [SCHEMA]
# to get the version number
./pbf_history_reader --version

The COUNTRYCODE must be only two letters, such as an iso-code. For instance en, fr, jp, ...

Basic usage (France, default)

./pbf_history_reader /path/to/history.osh.pbf /path/to/tag_list.txt

Writes to schema huwise_osm, table fr_history.

With country code (multi-country support)

./pbf_history_reader /path/to/history.osh.pbf /path/to/tag_list.txt nz

Writes to schema huwise_osm, table nz_history.

./pbf_history_reader /path/to/history.osh.pbf /path/to/tag_list.txt au

Writes to schema huwise_osm, table au_history.

With explicit schema (optional)

./pbf_history_reader /path/to/history.osh.pbf /path/to/tag_list.txt fr huwise_osm

To know the version

./pbf_history_reader --version

---

Configure your rust environment

Tools

Install rust and cargo with:

curl https://sh.rustup.rs -sSf | sh

Useful commands

Documentation

Building the executable file

from the osm/pbf_history_reader subfolder

cargo build

It creates an executable file in ./target/debug/ directory (by default, cargo builds in debug mode)

Building & running the executable file

cargo run

This command builds then run the executable file.

Building for release

cargo build --release

It creates an executable file in ./target/release/ directory. With this option, the program is optimized to run faster.

---

Cross-compilation

Method 1 (recommended): GCC toolchain

• Install a cross toolchain:

On mac:

brew install messense/macos-cross-toolchains/x86_64-unknown-linux-gnu  # for x86_64
brew install messense/macos-cross-toolchains/aarch64-unknown-linux-gnu  # for arm64

Then specify it in cargo config file .cargo/config.toml:

# .cargo/config.toml
[target.x86_64-unknown-linux-gnu]
linker = "/usr/local/bin/x86_64-unknown-linux-gnu-gcc"

[target.aarch64-unknown-linux-gnu]
linker = "/usr/local/bin/aarch64-unknown-linux-gnu-gcc"

• Specify the compilation target:

rustup target add x86_64-unknown-linux-gnu   # for x86_64
rustup target add aarch64-unknown-linux-gnu  # for arm64

• Compile (in release):

cargo build --target x86_64-unknown-linux-gnu --release   # for x86_64
cargo build --target aarch64-unknown-linux-gnu --release  # for arm64

---

Method 2 (fallback): cargo-zigbuild

If the GCC toolchain installation fails (e.g. Python dependency issues), use Zig as a linker via cargo-zigbuild. Zig natively supports all cross-compilation targets without requiring large GCC toolchains.

brew install zig
cargo install cargo-zigbuild@0.21.8

• Specify the compilation target:

rustup target add aarch64-unknown-linux-musl  # for arm64
rustup target add x86_64-unknown-linux-musl  # for x86_64

• Compile (in release):

cargo zigbuild --target aarch64-unknown-linux-musl --release  # for arm64
cargo zigbuild --target x86_64-unknown-linux-musl --release  # for x86_64

---

Environment variables

pbf_history_reader needs several environment variables to work:

• DB_HOST (required)
• DB_PASSWORD (required)
• DB_NAME
• DB_PORT (default: 5432)
• DB_USER
• OSM_ACCOUNT_USER (for history files)
• OSM_ACCOUNT_PASSWORD (for history files)

They can be provided with env.local or .envrc files (don't forget to pass them to the container in docker-compose.yml file)

---

Release

How to release a new version

1. Update the version tag in .github/workflows/rust.yml and in Cargo.toml

   Edit the tag_name and release_name fields:

   - name: Create Release
     uses: softprops/action-gh-release@v1
     with:
       files: target/x86_64-unknown-linux-musl/release/*
       tag_name: v1.2.0        # ← update this
       release_name: Release v1.2.0  # ← and this

Edit the version fields:

   version = "1.2.0". # <- update this

2. Commit and push to a branch for PR

   git add .github/workflows/rust.yml
   git commit -m "chore: bump version to v1.2.0"
   git push origin my_branch

3. GitHub Actions will automatically:
   • Build the binary for x86_64-unknown-linux-musl
   • Create a GitHub Release v1.3.0 with the binary attached
   • The binary will be available at:

     https://github.com/opendatasoft/osm-pbf-history-reader/releases/download/v1.2.0/pbf_history_reader