Skip to content

Commit f6a2c5b

Browse files
committed
Add reproducible Simhash benchmark
1 parent e20bcc1 commit f6a2c5b

5 files changed

Lines changed: 53 additions & 21 deletions

File tree

.formatter.exs

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
11
# Used by "mix format"
22
[
3-
inputs: ["{mix,.formatter}.exs", "{config,lib,test}/**/*.{ex,exs}"]
3+
inputs: ["{mix,.formatter}.exs", "{bench,config,lib,test}/**/*.{ex,exs}"]
44
]

CHANGELOG.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,11 @@ Release dates are based on the [Hex.pm release history](https://hex.pm/packages/
88

99
## [Unreleased]
1010

11+
### Changed
12+
13+
- Replaced the historical Simhash performance claim with a packaged, reproducible
14+
benchmark script using pinned Similarity, simhash-ex, and Benchee versions.
15+
1116
## [0.5.1] - 2026-08-23
1217

1318
### Changed

README.md

Lines changed: 17 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -88,28 +88,25 @@ Similarity.sorensen_dice("this that", "just that")
8888
```
8989

9090
## Performance
91-
Similarity.simhash is 2x faster than simhash-ex v1.1.0 package.
9291

92+
The reproducible [Simhash benchmark](bench/simhash.exs) uses `Mix.install/1` to
93+
install pinned versions of Benchee, Similarity, and
94+
[simhash-ex](https://github.com/UniversalAvenue/simhash-ex). It verifies that both
95+
implementations return the same score before benchmarking them:
96+
97+
```console
98+
$ elixir bench/simhash.exs
9399
```
94-
Benchmark suite executing with the following configuration:
95-
warmup: 2 s
96-
time: 5 s
97-
memory time: 0 ns
98-
parallel: 1
99-
inputs: none specified
100-
Estimated total run time: 14 s
101-
102-
Benchmarking simhash-ex...
103-
Benchmarking similarity.simhash...
104-
105-
Name ips average deviation median 99th %
106-
similarity.simhash 3.67 K 272.69 μs ±6.50% 267.84 μs 353.05 μs
107-
simhash-ex 1.75 K 572.14 μs ±12.31% 552.22 μs 781.02 μs
108-
109-
Comparison:
110-
similarity.simhash 3.67 K
111-
simhash-ex 1.75 K - 2.10x slower +299.46 μs
112-
```
100+
101+
On Linux with an AMD Ryzen 7 8845HS, Elixir 1.20.3, and Erlang/OTP 29.0.5:
102+
103+
| Implementation | Average time | Throughput | Memory |
104+
| --- | ---: | ---: | ---: |
105+
| Similarity 0.5.1 | 55.80 μs | 17.92 K ips | 126.82 KB |
106+
| simhash-ex `e04aa01` | 143.60 μs | 6.96 K ips | 262.54 KB |
107+
108+
For this input, Similarity was 2.57× faster and used 52% less memory. Results vary
109+
by hardware and runtime version.
113110

114111
## License
115112

bench/simhash.exs

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
Mix.install([
2+
{:benchee, "== 1.5.1"},
3+
{:similarity, "== 0.5.1"},
4+
# simhash-ex is published as :simhash; this revision supports current Elixir.
5+
{:simhash,
6+
git: "https://github.com/UniversalAvenue/simhash-ex.git",
7+
ref: "e04aa014e39ea2a7f53baf30f9d23c1e7cf6cd21"}
8+
])
9+
10+
left = "pork belly jerky brisket tenderloin shank kevin spare ribs"
11+
right = "porchetta pork loin. Leberkas ball tip biltong, beef ribs"
12+
13+
similarity_score = Similarity.simhash(left, right)
14+
simhash_ex_score = Simhash.similarity(left, right)
15+
16+
unless similarity_score == simhash_ex_score do
17+
raise "implementations returned different scores: " <>
18+
"#{similarity_score} != #{simhash_ex_score}"
19+
end
20+
21+
Benchee.run(
22+
%{
23+
"similarity 0.5.1" => fn -> Similarity.simhash(left, right) end,
24+
"simhash-ex e04aa01" => fn -> Simhash.similarity(left, right) end
25+
},
26+
warmup: 2,
27+
time: 5,
28+
memory_time: 2
29+
)

mix.exs

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,7 @@ defmodule Similarity.MixProject do
3737

3838
defp package do
3939
[
40+
files: ~w(lib bench .formatter.exs mix.exs README.md LICENSE CHANGELOG.md),
4041
maintainers: ["Barna Kovacs"],
4142
licenses: ["MIT"],
4243
links: %{"GitHub" => "https://github.com/preciz/similarity"}

0 commit comments

Comments
 (0)