[optimizer] add LiteHit: single-pass multi-capacity LRU prefix hit-rate analyzer (#242)

* [optimizer] add LiteHit multi-capacity LRU prefix hit-rate analyzer core

* [optimizer] move OnlineOptimizerManager under manager/online_runtime

* [optimizer] switch online TraceQuery to LiteHit token-based prefix hit rate

* [optimizer] add LiteHit offline runner with streaming trace loader and lite_hit_main

* [docs] document LiteHit algorithm and refresh optimizer architecture

* [optimizer] commit LiteHit requests tail-to-head to match leaf-first eviction

* [optimizer] add LiteHit hit-curve facts primitives

Introduce the capacity-independent building blocks of the LiteHit facts
refactor: HitCurveSegment/RequestFact with a stateless HitCurveProjector,
shared request preprocessing (authoritative input length validation plus a
rolling Jenkins prefix hash bit-compatible with the Python trace producer),
and the facts CSV row codec and post-hoc capacity query used by offline
replay.

* [optimizer] rework LiteHit to capacity-independent facts

* [optimizer] move enable_prefix_hash from instance to instance group

* [optimizer] support LiteHit trace re-blocking and multi-instance fanout

* [optimizer] clean up online trace query response semantics

Remove redundant slot-0 scalars from TraceQueryResult, unify the -1
sentinel for disabled theoretical max cache across TraceQuery and
ListInstances, rename avg_bytes_per_block to bytes_per_block, and report
full-attention kv_cache_usage_bytes under infinite-capacity semantics.

* [docs] consolidate LiteHit docs into the README

Document why request commit is hit/miss-agnostic (writes also touch) and
the read/write-separation path to decode support, retire REQUIREMENTS.md
in favor of the README, and index LiteHit in the optimizer README.

* [optimizer] address LiteHit PR review feedback

- keep TraceQueryResponse wire numbers (capacity_results=3,
  theoretical_result=4) and add input_token_len as new tag 5
- reject duplicate instance_id in offline lane setup so fanout cannot
  route one request twice into the same lane
- guard trace block_size==0 on the programmatic-setter path (modulo
  divisor in re-blocking check)
- distinguish I/O errors from EOF (in.bad()) in trace loader and facts
  query instead of silently treating truncated input as complete
- docs: rewrite the stale LiteHit section in optimizer_architecture.md
  (drop nonexistent assume_time_sorted and the old single-CSV schema,
  describe the two-step facts flow) and state that write trace events
  are recognized and ignored by the facts replay

Author: Tyndalllll

docs/design/module_architecture.md

@@ -16,7 +16,7 @@
 |---|---|---|
 | **KVCache Manager** | `kv_cache_manager/` | 核心系统：全局 KVCache 元数据管理服务，以及配套的客户端 SDK 与推理框架连接器。本文档的主体。 |
 | **HiSim** | `hisim/` | 独立的 LLM 推理仿真系统，通过回放 trace 预测 TTFT/TPOT/吞吐等指标，不依赖 Manager 运行时。 |
-| **Optimizer** | `kv_cache_manager/optimizer/` | 缓存仿真与优化：回放 KVCache 访问 trace，模拟命中率与容量消耗，指导逐出策略与容量参数调优。在线化能力正在开发中，后续会与现有 optimizer 合并。 |
+| **Optimizer** | `kv_cache_manager/optimizer/` | 缓存仿真与优化：既支持离线回放 KVCache 访问 trace，也提供在线 TraceQuery 服务。完整 optimizer 用于策略/容量分析；LiteHit 用最小状态精确计算多容量 full-attention LRU 命中率。 |
 
 KVCache Manager 采用中心化部署，负责 KVCache 的全局元数据管理（查询、写入、容量管理），推理引擎通过 Client/Connector 接入。
 
@@ -97,7 +97,7 @@ service → manager → meta → config → data_storage → common
 ### 3.3 客户端与 Optimizer
 
 - **client** 是独立的对外分支，仅共享 `common`、`config`、`data_storage`、`protocol` 以及 `service/util:manager_message_proto_util`；**py_connector** 通过 pybind 位于 client 之上。核心服务端不依赖 client。运行时，元数据面经 gRPC（C++ `MetaClient`）或 HTTP（py_connector 的 Python `KvCacheManagerClient`）调用 KVCM 服务，数据面经 C++ `TransferClient` 直接读写存储后端——这几条链路是理解端到端流程的关键（见第 4 节）。
-- **optimizer** 负责 KVCache 访问 trace 的仿真与优化（命中率/容量分析、逐出与容量参数调优）。目前通过 `meta:cache_location` 类型与 `event` 的 optimizer 事件与核心关联；在线化能力正在开发中，后续会与现有 optimizer 合并。
+- **optimizer** 负责 KVCache 访问 trace 的仿真与优化（命中率/容量分析、逐出与容量参数调优），并通过独立 online runtime/service 提供实时 TraceQuery。full-attention LRU 的在线多容量统计复用 LiteHit；它通过 `meta:cache_location` 类型与 `event` 的 optimizer 事件与核心关联。
 
 ### 3.4 模块关系图
 

kv_cache_manager/optimizer/BUILD

@@ -13,32 +13,38 @@ cc_binary(
         "//kv_cache_manager/common:logger"
     ],
 )
-cc_library(
-    name = "config",
-    hdrs = [
-        "advanced_cache_reclaimer.h",
-        "advanced_data_storage_selector.h",
-    ],
-    srcs = [
 
+cc_binary(
+    name = "lite_hit_main",
+    srcs = [
+        "lite_hit_main.cc",
+    ],
+    deps = [
+        "//kv_cache_manager/optimizer/manager:lite_hit_offline_runner",
+        "//kv_cache_manager/optimizer/config:optimizer_lite_hit_config",
+        "//kv_cache_manager/common:logger",
     ],
 )
 
-cc_library(
-    name = "online_optimizer_manager",
+cc_binary(
+    name = "lite_hit_facts_query_main",
     srcs = [
-        "online_runtime/online_optimizer_manager.cc",
+        "lite_hit_facts_query_main.cc",
     ],
+    deps = [
+        "//kv_cache_manager/optimizer/liteHit:facts_query",
+        "//kv_cache_manager/common:logger",
+    ],
+)
+
+cc_library(
+    name = "config",
     hdrs = [
-        "online_runtime/online_optimizer_manager.h",
+        "advanced_cache_reclaimer.h",
+        "advanced_data_storage_selector.h",
     ],
-    deps = [
-        "//kv_cache_manager/common",
-        "//kv_cache_manager/optimizer/config:online_optimizer_config",
-        "//kv_cache_manager/optimizer/config:optimizer_registry_manager",
-        "//kv_cache_manager/optimizer/index:cache_indexer",
-        "//kv_cache_manager/optimizer/index:cache_indexer_factory",
-        "//kv_cache_manager/optimizer/index:ttl_cache_indexer_wrapper",
+    srcs = [
+
     ],
 )
 
@@ -65,7 +71,7 @@ cc_library(
         "service/metrics/optimizer_metrics_reporter.h",
     ],
     deps = [
-        ":online_optimizer_manager",
+        "//kv_cache_manager/optimizer/manager:online_optimizer_manager",
         ":optimizer_metrics_collector",
         "//kv_cache_manager/common",
         "//kv_cache_manager/common:common_util",
@@ -100,7 +106,7 @@ cc_library(
         "service/optimizer_service_impl.h",
     ],
     deps = [
-        ":online_optimizer_manager",
+        "//kv_cache_manager/optimizer/manager:online_optimizer_manager",
         ":optimizer_call_guard",
         ":optimizer_metrics_collector",
         ":optimizer_metrics_reporter",
@@ -181,7 +187,7 @@ cc_library(
         "-fcoroutines",
     ],
     deps = [
-        ":online_optimizer_manager",
+        "//kv_cache_manager/optimizer/manager:online_optimizer_manager",
         ":online_optimizer_server_config",
         ":optimizer_metrics_reporter",
         ":optimizer_service_grpc",

kv_cache_manager/optimizer/README.md

@@ -9,6 +9,7 @@ KVCacheManager Optimizer 是一个独立的缓存优化分析模块，通过回
 - 多种驱逐策略的模拟和对比
 - 缓存命中率的实时统计和分析
 - Radix Tree 索引结构的可视化
+- LiteHit：full-attention 场景的容量无关命中率分析（一次回放产出事实，任意 LRU 容量事后投影，支持重分块与多 block size fanout），见 [liteHit/README.md](liteHit/README.md)
 
 ## 动机
 
@@ -31,7 +32,7 @@ KVCacheManager Optimizer 是一个独立的缓存优化分析模块，通过回
 - **灵活配置**：通过 JSON 配置文件灵活配置实例、存储和策略
 - **可视化分析**：支持 Radix Tree 可视化和命中率图表生成
 
-标准策略配置、multi-instance replay、trace schema 和命中率口径见 [docs/strategy_config.md](docs/strategy_config.md)。标准版中 `HitRate` 统一表示整体 token hit rate，即 `HitTokens / InputTokens`；local/remote 只作为 trace `block_mask` 与 optimizer 模拟命中的诊断拆分，不作为标准结论维度。传入 optimizer config 的 Python 入口统一使用配置中的 `output_result_path`；`multi_instance_replay` 不读取完整 config，使用显式 `--output-dir`。标准 `get` trace 必须包含 `input_len`；外部只有请求级日志时可使用 `type=request`，optimizer 会按 `trace_replay.write_delay_ns` 在内部调度 delayed write；已经拆分好的 `get` / `write` trace 仍然支持。
+标准策略配置、multi-instance replay、trace schema 和命中率口径见 [docs/strategy_config.md](docs/strategy_config.md)。标准版中 `HitRate` 统一表示整体 token hit rate，即 `HitTokens / InputTokens`；local/remote 只作为 trace `block_mask` 与 optimizer 模拟命中的诊断拆分，不作为标准结论维度。传入 optimizer config 的 Python 入口统一使用配置中的 `output_result_path`；`multi_instance_replay` 不读取完整 config，使用显式 `--output-dir`。标准 `get` trace 必须包含 `input_len`；外部只有请求级日志时可使用 `type=request`，optimizer 会按 `trace_replay.write_delay_ns` 在内部调度 delayed write；已经拆分好的 `get` / `write` trace 仍然支持（仅限 replay 路径；LiteHit facts 回放会识别并忽略 `write` 事件，`get` 提交即视为写回完成）。
 
 ### 架构设计
 

kv_cache_manager/optimizer/config/BUILD

@@ -45,6 +45,21 @@ cc_library(
     ],
 )
 
+cc_library(
+    name = "optimizer_lite_hit_config",
+    srcs = [
+        "optimizer_lite_hit_config.cc",
+    ],
+    hdrs = [
+        "optimizer_lite_hit_config.h",
+    ],
+    deps = [
+        ":online_optimizer_config",
+        "//kv_cache_manager/common",
+        "//kv_cache_manager/common:jsonizable",
+    ],
+)
+
 cc_library(
     name = "config",
     deps = [

kv_cache_manager/optimizer/config/optimizer_instance_group.cc

@@ -19,6 +19,7 @@ bool OptimizerInstanceGroup::FromRapidValue(const rapidjson::Value &rapid_value)
     KVCM_JSON_GET_DEFAULT_MACRO(rapid_value, "shared_group_quota", shared_group_quota_, false);
     KVCM_JSON_GET_DEFAULT_MACRO(rapid_value, "enable_theoretical_max_cache", enable_theoretical_max_cache_, false);
     KVCM_JSON_GET_DEFAULT_MACRO(rapid_value, "ttl_seconds", ttl_seconds_, int64_t(0));
+    KVCM_JSON_GET_DEFAULT_MACRO(rapid_value, "enable_prefix_hash", enable_prefix_hash_, false);
     SortCapacities();
     return true;
 }
@@ -32,6 +33,7 @@ void OptimizerInstanceGroup::ToRapidWriter(rapidjson::Writer<rapidjson::StringBu
     Put(writer, "shared_group_quota", shared_group_quota_);
     Put(writer, "enable_theoretical_max_cache", enable_theoretical_max_cache_);
     Put(writer, "ttl_seconds", ttl_seconds_);
+    Put(writer, "enable_prefix_hash", enable_prefix_hash_);
 }
 
 bool OptimizerInstanceGroup::ValidateRequiredFields(std::string &invalid_fields) const {

kv_cache_manager/optimizer/config/optimizer_instance_group.h

@@ -25,6 +25,11 @@ class OptimizerInstanceGroup : public Jsonizable {
     bool shared_group_quota() const { return shared_group_quota_; }
     bool enable_theoretical_max_cache() const { return enable_theoretical_max_cache_; }
     int64_t ttl_seconds() const { return ttl_seconds_; }
+    // When true, incoming block keys of every instance in this group are
+    // per-block raw hashes and are converted to rolling prefix-chained keys
+    // before analysis. Key shape follows the model deployment, so it lives at
+    // group granularity for both online and offline.
+    bool enable_prefix_hash() const { return enable_prefix_hash_; }
 
     void set_name(const std::string &v) { name_ = v; }
     void set_capacity_gb(const std::vector<double> &v) {
@@ -35,6 +40,7 @@ class OptimizerInstanceGroup : public Jsonizable {
     void set_shared_group_quota(bool v) { shared_group_quota_ = v; }
     void set_enable_theoretical_max_cache(bool v) { enable_theoretical_max_cache_ = v; }
     void set_ttl_seconds(int64_t v) { ttl_seconds_ = v; }
+    void set_enable_prefix_hash(bool v) { enable_prefix_hash_ = v; }
 
 private:
     void SortCapacities();
@@ -45,6 +51,7 @@ class OptimizerInstanceGroup : public Jsonizable {
     bool shared_group_quota_ = false;
     bool enable_theoretical_max_cache_ = false;
     int64_t ttl_seconds_ = 0;
+    bool enable_prefix_hash_ = false;
 };
 
 } // namespace kv_cache_manager

kv_cache_manager/optimizer/config/optimizer_lite_hit_config.cc

@@ -0,0 +1,55 @@
+#include "kv_cache_manager/optimizer/config/optimizer_lite_hit_config.h"
+
+#include "kv_cache_manager/common/logger.h"
+
+namespace kv_cache_manager {
+
+bool OptimizerLiteHitConfig::FromRapidValue(const rapidjson::Value &rapid_value) {
+    KVCM_JSON_GET_MACRO(rapid_value, "trace_file_path", trace_file_path_);
+    KVCM_JSON_GET_MACRO(rapid_value, "output_result_path", output_result_path_);
+    KVCM_JSON_GET_MACRO(rapid_value, "instance_groups", instance_groups_);
+    KVCM_JSON_GET_MACRO(rapid_value, "instances", instances_);
+    KVCM_JSON_GET_DEFAULT_MACRO(rapid_value, "override_instance_id", override_instance_id_, std::string());
+    KVCM_JSON_GET_DEFAULT_MACRO(rapid_value, "block_size", block_size_, uint64_t(256));
+    KVCM_JSON_GET_DEFAULT_MACRO(rapid_value, "fanout_all_instances", fanout_all_instances_, false);
+    KVCM_JSON_GET_DEFAULT_MACRO(rapid_value, "pipeline_worker_count", pipeline_worker_count_, int32_t(1));
+
+    if (trace_file_path_.empty()) {
+        KVCM_LOG_ERROR("lite_hit config: trace_file_path must not be empty");
+        return false;
+    }
+    if (output_result_path_.empty()) {
+        KVCM_LOG_ERROR("lite_hit config: output_result_path must not be empty");
+        return false;
+    }
+    if (instance_groups_.empty()) {
+        KVCM_LOG_ERROR("lite_hit config: instance_groups must not be empty");
+        return false;
+    }
+    if (instances_.empty()) {
+        KVCM_LOG_ERROR("lite_hit config: instances must not be empty");
+        return false;
+    }
+    if (block_size_ == 0) {
+        KVCM_LOG_ERROR("lite_hit config: block_size (trace granularity) must be positive");
+        return false;
+    }
+    if (fanout_all_instances_ && !override_instance_id_.empty()) {
+        KVCM_LOG_ERROR("lite_hit config: fanout_all_instances and override_instance_id are mutually exclusive");
+        return false;
+    }
+    return true;
+}
+
+void OptimizerLiteHitConfig::ToRapidWriter(rapidjson::Writer<rapidjson::StringBuffer> &writer) const noexcept {
+    Put(writer, "trace_file_path", trace_file_path_);
+    Put(writer, "output_result_path", output_result_path_);
+    Put(writer, "instance_groups", instance_groups_);
+    Put(writer, "instances", instances_);
+    Put(writer, "override_instance_id", override_instance_id_);
+    Put(writer, "block_size", block_size_);
+    Put(writer, "fanout_all_instances", fanout_all_instances_);
+    Put(writer, "pipeline_worker_count", pipeline_worker_count_);
+}
+
+} // namespace kv_cache_manager

kv_cache_manager/optimizer/config/optimizer_lite_hit_config.h

@@ -0,0 +1,73 @@
+#pragma once
+
+#include <string>
+#include <vector>
+
+#include "kv_cache_manager/common/jsonizable.h"
+#include "kv_cache_manager/optimizer/config/optimizer_instance_group.h"
+#include "kv_cache_manager/optimizer/config/optimizer_instance_info.h"
+
+namespace kv_cache_manager {
+
+// Configuration for the offline LiteHit entry (lite_hit_main / LiteHitOfflineRunner).
+//
+// It deliberately reuses the SAME config objects as the online service
+// (OptimizerInstanceGroup + OptimizerInstanceInfo): instance registration is
+// validated by OnlineOptimizerManager exactly like online. The replay itself
+// produces one capacity-independent facts CSV
+// (${output_result_path}/litehit_facts.csv); capacities are applied
+// afterwards by the facts query tool.
+class OptimizerLiteHitConfig : public Jsonizable {
+public:
+    OptimizerLiteHitConfig() = default;
+    ~OptimizerLiteHitConfig() override = default;
+
+    bool FromRapidValue(const rapidjson::Value &rapid_value) override;
+    void ToRapidWriter(rapidjson::Writer<rapidjson::StringBuffer> &writer) const noexcept override;
+
+    [[nodiscard]] const std::string &trace_file_path() const { return trace_file_path_; }
+    // Output DIRECTORY. The replay atomically publishes
+    // ${output_result_path}/litehit_facts.csv on success.
+    [[nodiscard]] const std::string &output_result_path() const { return output_result_path_; }
+    [[nodiscard]] const std::vector<OptimizerInstanceGroup> &instance_groups() const { return instance_groups_; }
+    [[nodiscard]] const std::vector<OptimizerInstanceInfo> &instances() const { return instances_; }
+    // When non-empty, every replayed request is attributed to this single
+    // instance_id (the trace's own instance_id is ignored). Use it to pool a
+    // whole per-service trace into ONE global cache, e.g. instance_id=pod files
+    // replayed as a single service-wide cache. The instances() list must define
+    // exactly this id.
+    [[nodiscard]] const std::string &override_instance_id() const { return override_instance_id_; }
+    // Native granularity (tokens per block) the trace keys were produced at.
+    // Every instance's block_size must be an exact multiple of it; requests
+    // are re-blocked (coarsening only) per lane during preprocessing.
+    [[nodiscard]] uint64_t block_size() const { return block_size_; }
+    // When true, every request is replayed into EVERY configured instance
+    // (the trace's own instance_id is ignored). Combined with instances of
+    // different block_size this sweeps several granularities over one trace
+    // in a single run. Mutually exclusive with override_instance_id.
+    [[nodiscard]] bool fanout_all_instances() const { return fanout_all_instances_; }
+    // The only public parallelism knob of the offline pipeline. Values < 1 are
+    // clamped to 1; queue/window sizes are derived internally.
+    [[nodiscard]] int32_t pipeline_worker_count() const { return pipeline_worker_count_; }
+
+    void set_trace_file_path(const std::string &path) { trace_file_path_ = path; }
+    void set_output_result_path(const std::string &path) { output_result_path_ = path; }
+    void set_instance_groups(const std::vector<OptimizerInstanceGroup> &groups) { instance_groups_ = groups; }
+    void set_instances(const std::vector<OptimizerInstanceInfo> &instances) { instances_ = instances; }
+    void set_override_instance_id(const std::string &id) { override_instance_id_ = id; }
+    void set_block_size(uint64_t block_size) { block_size_ = block_size; }
+    void set_fanout_all_instances(bool fanout) { fanout_all_instances_ = fanout; }
+    void set_pipeline_worker_count(int32_t count) { pipeline_worker_count_ = count; }
+
+private:
+    std::string trace_file_path_;
+    std::string output_result_path_;
+    std::vector<OptimizerInstanceGroup> instance_groups_;
+    std::vector<OptimizerInstanceInfo> instances_;
+    std::string override_instance_id_;
+    uint64_t block_size_ = 256;
+    bool fanout_all_instances_ = false;
+    int32_t pipeline_worker_count_ = 1;
+};
+
+} // namespace kv_cache_manager