Skip to content

Consider batched LatLngToCells API to lower cgo transition overhead #113

Description

@zoobst

Type: feature request
TL;DR: each call to LatLng.Cell(res) crosses cgo, and in tight loops the transition cost dominates the actual H3 work by ~5×. A single batched entry point that runs the loop on the C side recovers most of that overhead for any Go caller doing per-row lookups.

Background

I appreciate that h3-go is intentionally a thin Go binding around the C library, the contribution guide is explicit about that, and the policy keeps the package small and easy to reason about. The proposal below is a single small entry point rather than a broader API expansion. I'd be happy to refile against uber/h3 instead if the maintainers see this as belonging in the C library. The cgo-overhead concern is a Go-binding-specific motivation, which is why I'm starting here.

Problem

For workloads that compute cells over millions of independent (lat, lng) rows, the per-call overhead of cgo (goroutine state save/restore, P-binding, stack growth checks) is the dominant cost. Concrete numbers from a recent profile of a Go ETL pipeline computing H3 resolutions per row:

Component % of total CPU
runtime.cgocall (transition itself) ~21%
Actual H3 computation inside C ~4%

A ~5× ratio of transition overhead to useful work. The pipeline processes millions of rows per cycle; the cgo calls alone are the single biggest hot spot in the CPU profile.

Proposed API

// Compute cells for n points at a single resolution. One cgo call.
func LatLngToCellsBatch(lls []LatLng, res int) ([]Cell, error)

Wraps a tight C loop that calls latLngToCell per point. Implementation is small: ~15 lines of cgo + ~30 lines of Go (input validation, error mapping, slice prep).

Measurements

Local benchmarks comparing the proposed batched call against the current per-call API at a single H3 resolution (6):

n rows Batched Per-call (current API) Speedup
64 11.8 µs 16.0 µs 1.35×
1,024 224 µs 287 µs 1.28×
16,384 5.14 ms 5.99 ms 1.17×
65,536 21.5 ms 24.7 ms 1.15×

The win narrows at very large batches because the C-side work starts to dominate the fixed cgo cost — but it's a real per-row saving that adds up to seconds of wall time per million rows for an ETL workload.

Offer

If a batched API is of interest, I'm happy to send a PR with the implementation + regression tests + benchmarks.

If this falls outside what the maintainers want h3-go to take on, no worries, totally understand and will refile against uber/h3 with the C-library framing (vectorization argument for restrict-qualified inputs).

Thanks for the great library either way.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions