chore: make core kv-router its own crate

[PeaBrane]

Motivation:

• Faster compilation for benching
• Some core indexing functionalties can be reused without the rest of dynamo-llm

Summary by CodeRabbit

• New Features

  • KV router functionality is now available as a dedicated, separately maintained component.
  • Token block types are now publicly exposed and accessible.

• Refactor

  • Reorganized workspace structure to separate concerns across multiple crates for improved maintainability and modularity.

_{✏️ Tip: You can customize this high-level summary in your review settings.}

[coderabbitai]

Walkthrough

This PR extracts the kv-router functionality from lib/llm into a standalone lib/kv-router crate. Import paths are updated across multiple files to reference the new dynamo_kv_router and dynamo_tokens crates. Module visibility in lib/tokens is adjusted to support the extraction.

Changes

Cohort / File(s)	Summary
Workspace configuration Cargo.toml	Adds lib/kv-router workspace member and declares dynamo-kv-router workspace dependency with version 0.9.0
New kv-router crate lib/kv-router/Cargo.toml, lib/kv-router/src/lib.rs, lib/kv-router/src/approx.rs, lib/kv-router/src/indexer.rs, lib/kv-router/src/protocols.rs, lib/kv-router/benches/radix_tree_microbench.rs	Establishes new crate with public modules (approx, indexer, protocols), public re-exports (RadixTree, RouterEvent, compute_block_hash_for_seq, LocalBlockHash, WorkerId). Updates internal imports from crate::kv_router::* to crate::* and external SequenceHash imports to dynamo_tokens::SequenceHash
lib/llm workspace integration lib/llm/Cargo.toml, lib/llm/src/kv_router.rs	Adds dynamo-tokens and dynamo-kv-router dependencies. Replaces local module declarations (approx, indexer, protocols) with public re-exports from dynamo_kv_router
lib/llm import path updates lib/llm/src/kv_router/scheduler.rs, lib/llm/src/kv_router/sequence.rs, lib/llm/src/mocker/kv_manager.rs, lib/llm/src/mocker/protocols.rs, lib/llm/src/mocker/scheduler.rs, lib/llm/src/mocker/sequence.rs	Updates SequenceHash and UniqueBlock imports from crate::tokens::* to dynamo_tokens::*
lib/llm tokens module visibility lib/llm/src/tokens.rs	Removes public re-export of blocks module
lib/tokens module exposure lib/tokens/src/lib.rs, lib/tokens/src/blocks.rs, lib/tokens/Cargo.toml	Makes blocks module publicly exported; adds GlobalHash = u64 type alias and expanded documentation; adds uuid workspace dependency

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Poem

🐰 A crate extracts with care and grace,
From llm's nest to find its place,
With imports reorganized clean,
The tokens module now is seen,
New kv-router hops with glee!

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Description check	⚠️ Warning	The description provides motivation but lacks structure and completeness compared to the template. Missing Overview, Details, and Where should the reviewer start sections.	Expand the description to follow the template structure: add an Overview section, provide detailed change descriptions organized by section, and specify which files reviewers should examine first.

✅ Passed checks (2 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title accurately summarizes the main change: extracting the kv-router code into its own standalone crate, making it a distinct module.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[biswapanda]

lgtm