You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.md
+17-7Lines changed: 17 additions & 7 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -5,19 +5,22 @@
5
5
6
6
A fast bloom filter with a real bitset, JSON serialization, and thread-safe variants.
7
7
8
-
Forked from [`AndreasBriese/bbloom`](https://github.com/AndreasBriese/bbloom). Uses an inlined SipHash-2-4 for hashing.
8
+
## Why this fork
9
9
10
-
## Install
10
+
Forked from [`AndreasBriese/bbloom`](https://github.com/AndreasBriese/bbloom) in 2019 after the upstream became unmaintained. The fork fixes safety and correctness issues, and adds features needed for production use:
11
11
12
-
```sh
13
-
go get github.com/ipfs/bbloom
14
-
```
12
+
- Caller-provided SipHash keys (`NewWithKeys`) to prevent hash-flooding with untrusted input
13
+
- Fixed double-hash step to always be odd, avoiding degenerate probe sequences
14
+
- SipHash keys preserved across JSON serialization round-trips
15
+
- Proper error handling in deserialization
16
+
17
+
The library may contain IPFS-specific optimizations but works as a general-purpose bloom filter.
15
18
16
19
## Usage
17
20
18
21
```go
19
-
// create a bloom filter for 65536 items and 1% false-positive rate
20
-
bf, _:= bbloom.New(float64(1<<16), float64(0.01))
22
+
// create a bloom filter for 65536 items and 0.1% false-positive rate
[Kubo](https://github.com/ipfs/kubo) and [Boxo](https://github.com/ipfs/boxo) use this library where CID deduplication or tracking is needed but the number of CIDs is too large to keep in memory as a map. Two main use cases:
52
+
53
+
-**Blockstore bloom cache**: answers `Has()` checks without hitting the datastore, filtering out the majority of negative lookups.
54
+
-**DAG walker dedup**: tracks visited CIDs during DAG traversal in the provider/reprovide system, keeping memory usage proportional to the bloom filter size rather than the number of blocks walked.
55
+
46
56
## Benchmarks
47
57
48
58
See [BENCHMARKS.md](BENCHMARKS.md) for comparison against other bloom filter libraries.
0 commit comments