File tree Expand file tree Collapse file tree
Expand file tree Collapse file tree Original file line number Diff line number Diff line change @@ -138,12 +138,14 @@ and co-rater decay.
138138Thanks also to ** [ Graze] ( https://github.com/graze-social/personalization ) ** , whose Rust implementation
139139of the LinkLonk algorithm for Bluesky feeds was a helpful reference.
140140
141- Where this implementation differs is ** memory efficiency** . It holds the entire like graph in RAM as a
142- compressed-sparse-row (CSR) layout of typed arrays with an arena-based string interner — DIDs and URIs
143- are mapped to integer IDs in a chunked byte arena + an open-addressed hash, instead of JavaScript
144- Maps/objects. That cuts graph memory roughly 2.5–3× versus a naive in-memory representation, so the
145- full 90-day like graph fits in ~ 47 GB and the traversal runs in-process (tens of milliseconds) on a
146- single commodity server.
141+ What this implementation focuses on is ** fitting the graph in RAM** . It holds the entire like graph in
142+ memory as a compressed-sparse-row (CSR) layout of typed arrays with an arena-based string interner —
143+ DIDs and URIs are mapped to integer IDs in a chunked byte arena + an open-addressed hash, rather than
144+ JavaScript Maps/objects. That's ~ 2.5–3× leaner than the same graph built with ordinary JS Maps (the
145+ fallback ` arrays ` layout), so the full 90-day like graph fits in ~ 47 GB and the traversal runs
146+ in-process (tens of milliseconds) on a single commodity server. (Graze takes a different tack — the
147+ graph lives in Redis as day-tranched sorted sets, shared across feeds; this keeps it in-process for
148+ single-feed traversal latency.)
147149
148150## License
149151
You can’t perform that action at this time.
0 commit comments