Type: feature request
TL;DR: each call to LatLng.Cell(res) crosses cgo, and in tight loops the transition cost dominates the actual H3 work by ~5×. A single batched entry point that runs the loop on the C side recovers most of that overhead for any Go caller doing per-row lookups.
Background
I appreciate that h3-go is intentionally a thin Go binding around the C library, the contribution guide is explicit about that, and the policy keeps the package small and easy to reason about. The proposal below is a single small entry point rather than a broader API expansion. I'd be happy to refile against uber/h3 instead if the maintainers see this as belonging in the C library. The cgo-overhead concern is a Go-binding-specific motivation, which is why I'm starting here.
Problem
For workloads that compute cells over millions of independent (lat, lng) rows, the per-call overhead of cgo (goroutine state save/restore, P-binding, stack growth checks) is the dominant cost. Concrete numbers from a recent profile of a Go ETL pipeline computing H3 resolutions per row:
| Component |
% of total CPU |
runtime.cgocall (transition itself) |
~21% |
| Actual H3 computation inside C |
~4% |
A ~5× ratio of transition overhead to useful work. The pipeline processes millions of rows per cycle; the cgo calls alone are the single biggest hot spot in the CPU profile.
Proposed API
// Compute cells for n points at a single resolution. One cgo call.
func LatLngToCellsBatch(lls []LatLng, res int) ([]Cell, error)
Wraps a tight C loop that calls latLngToCell per point. Implementation is small: ~15 lines of cgo + ~30 lines of Go (input validation, error mapping, slice prep).
Measurements
Local benchmarks comparing the proposed batched call against the current per-call API at a single H3 resolution (6):
| n rows |
Batched |
Per-call (current API) |
Speedup |
| 64 |
11.8 µs |
16.0 µs |
1.35× |
| 1,024 |
224 µs |
287 µs |
1.28× |
| 16,384 |
5.14 ms |
5.99 ms |
1.17× |
| 65,536 |
21.5 ms |
24.7 ms |
1.15× |
The win narrows at very large batches because the C-side work starts to dominate the fixed cgo cost — but it's a real per-row saving that adds up to seconds of wall time per million rows for an ETL workload.
Offer
If a batched API is of interest, I'm happy to send a PR with the implementation + regression tests + benchmarks.
If this falls outside what the maintainers want h3-go to take on, no worries, totally understand and will refile against uber/h3 with the C-library framing (vectorization argument for restrict-qualified inputs).
Thanks for the great library either way.
Type: feature request
TL;DR: each call to
LatLng.Cell(res)crosses cgo, and in tight loops the transition cost dominates the actual H3 work by ~5×. A single batched entry point that runs the loop on the C side recovers most of that overhead for any Go caller doing per-row lookups.Background
I appreciate that h3-go is intentionally a thin Go binding around the C library, the contribution guide is explicit about that, and the policy keeps the package small and easy to reason about. The proposal below is a single small entry point rather than a broader API expansion. I'd be happy to refile against
uber/h3instead if the maintainers see this as belonging in the C library. The cgo-overhead concern is a Go-binding-specific motivation, which is why I'm starting here.Problem
For workloads that compute cells over millions of independent
(lat, lng)rows, the per-call overhead of cgo (goroutine state save/restore, P-binding, stack growth checks) is the dominant cost. Concrete numbers from a recent profile of a Go ETL pipeline computing H3 resolutions per row:runtime.cgocall(transition itself)A ~5× ratio of transition overhead to useful work. The pipeline processes millions of rows per cycle; the cgo calls alone are the single biggest hot spot in the CPU profile.
Proposed API
Wraps a tight C loop that calls
latLngToCellper point. Implementation is small: ~15 lines of cgo + ~30 lines of Go (input validation, error mapping, slice prep).Measurements
Local benchmarks comparing the proposed batched call against the current per-call API at a single H3 resolution (6):
The win narrows at very large batches because the C-side work starts to dominate the fixed cgo cost — but it's a real per-row saving that adds up to seconds of wall time per million rows for an ETL workload.
Offer
If a batched API is of interest, I'm happy to send a PR with the implementation + regression tests + benchmarks.
If this falls outside what the maintainers want h3-go to take on, no worries, totally understand and will refile against
uber/h3with the C-library framing (vectorization argument forrestrict-qualified inputs).Thanks for the great library either way.