Skip to content

Draft: Experiments using Rapidhash #544

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 7 commits into
base: master
Choose a base branch
from
Open

Conversation

hoxxep
Copy link
Contributor

@hoxxep hoxxep commented Mar 20, 2025

On the back of #532, this is a PR with further benchmarks for replacing fnv with rapidhash.

Full disclosure, I'm the author of the rapidhash crate and was checking to see if it could give some easy wins.

Summary of findings:

  • at small field/value sizes, FnvHasher is very marginally faster.
  • at longer field/value sizes, RapidHasher can be significantly faster (my benchmarking suggests the cutoff is likely at around 8-12 chars).
  • this assumes a user is using the standard HashMap for labels — rapidhash and fnv hashmaps benefit from using rapidhash in the vec.rs at most lengths, and I haven't got a good explanation for why a default HashMap doesn't see the same improvement.
  • I've only been able to benchmark this on an M1 Max chip, I would be curious to see benchmarks on x86, other Arm architectures, and particularly server platforms.

The benchmarks are written as:

  • counter_with_mapped_labels the original tests with very short field names around 4 characters (zwei).
  • counter_with_mapped_mid_labels a mid-size label of around 12 chars (midsize_zwei).
  • counter_with_mapped_long_labels a longer label of around 24 chars (longer_field_name_zwei).

Benchmarking fnv vs. rapidhash actually entails commenting and uncommenting vec.rs lines 9 and 10.

I would love to hear thoughts or see other benchmarks, otherwise if this is of interest I can benchmark on some AWS instances with different architectures.

use fnv::FnvHasher;
// TODO: experimenting, alternating while benchmarking
// use fnv::FnvHasher;
use rapidhash::RapidHasher as FnvHasher;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The key lines to change to compare benchmarks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant