Per-language token estimation (replace global AVG_CHARS_PER_LINE=35)

[egorfedorov]

Problem

Token estimation uses a single global constant:

// src/utils.js:108
const AVG_CHARS_PER_LINE = 35;
estimateTokens = (lineCount * 35) / ratio

Real chars/line vary a lot by language: .json ≈ 18–22, .md ≈ 45–60, .py ≈ 35–40, minified JS ≈ 200+. A flat 35 over/under-estimates per file, and that number feeds the headline "tokens saved" figure the product is built around.

Why it matters

v4.1 is explicitly about honest NET savings. A biased per-line constant biases the central metric.

Proposal

• Replace the scalar with a per-extension map of avg chars/line (data-backed defaults), falling back to 35 for unknown types.
• Keep it overridable in config.json.

Affected

src/utils.js (estimateTokens, AVG_CHARS_PER_LINE)

Acceptance

• estimateTokens uses extension-aware ratios.
• Unit tests assert .json/.md/.py estimates land within tolerance of a tokenizer baseline.