Skip to content

logprob.md

Latest commit

 

History

History
31 lines (23 loc) · 2.24 KB

logprob.md

File metadata and controls

31 lines (23 loc) · 2.24 KB

Reference: Logprobs in Distributed Systems & Verification

Focus: Infrastructure-Level Integrity for Lanka KV-Cache

“Can chosen-token logprob divergence across replicas predict later output divergence or latent KV/state inconsistency early enough to support selective recomputation?”

1. Core Concept

A Logprob is the natural logarithm of a probability $P$.

  • Equation: $L = \ln(P)$
  • Range: Since $0 \le P \le 1$, the Logprob is always $\le 0$ (e.g., $P=1 \rightarrow L=0$; $P=0.01 \rightarrow L \approx -4.6$).

Why Systems Engineers Should Care:

  • Numerical Stability: Multiplying thousands of small probabilities across LLM layers leads to floating-point underflow. Adding Logprobs is mathematically identical but stays within the range of f32/f64.
  • Information Density: Logprobs provide more "precision room" to distinguish between very small probabilities (e.g., $10^{-7}$ vs $10^{-8}$) which represent the "tails" of a model's reasoning.

2. The TRS (Differential) Analogy

Treat the Logprob as the Signal and hardware-induced divergence as Noise.

  • Common-Mode (The Ground Truth): Node A and Node B both output Token X with Logprob Y. The signal is clean; the state is synchronized.
  • Differential (The Hallucination/Drift): Node A and Node B output the same token, but with different Logprobs. This indicates Numerical Jitter or a latent divergence in the KV-cache state.

3. Lanka Verification Protocol

In a distributed verifier, the Logprob is used as a Semantic Checksum.

  1. Generation: Runtimes produce a token and its associated Logprob.
  2. Comparison: Lanka compares the Logprobs from multiple nodes.
  3. Thresholding ($\epsilon$): Because GPUs are inherently non-deterministic (e.g., atomic addition order), Lanka must allow for a tiny difference: $|L_a - L_b| < \epsilon$.
  4. Consensus: If the difference exceeds $\epsilon$, Lanka rejects the commit and triggers a high-precision recomputation (The "Isolated Transaction").

4. Financial & SRE Implications

  • Auditability: Storing Logprobs in Lanka creates a mathematical audit trail of certainty.
  • Safety: Prevents "Silent Hallucinations" where a node drifts numerically but still outputs a plausible-looking token.