Skip to content

version_pinning_and_model_lock.md

Latest commit

 

History

History
187 lines (148 loc) · 9.12 KB

version_pinning_and_model_lock.md

File metadata and controls

187 lines (148 loc) · 9.12 KB

Version Pinning & Model Lock — OpsDeploy

🧭 Quick Return to Map

You are in a sub-page of OpsDeploy.
To reorient, go back here:

Think of this page as a desk within a ward.
If you need the full triage and all prescriptions, return to the Emergency Room lobby.

Lock models, prompts, embeddings, rerankers, tokenizers, and analyzers so prod can’t drift.
Use this page to make “what ran” exactly the same as “what you tested,” and to detect any silent shift before a rollout.

Open these first

What to pin (and where)

Layer Pin fields Why it matters
LLM model MODEL_VER, provider model ID, decoding params Minor provider updates shift ΔS/λ; force exact version and params.
Prompt pack PROMPT_VER, header checksum Header order changes λ; lock and audit.
Embedding EMBED_MODEL_VER, EMBED_DIM, NORM, metric Any mismatch breaks neighborhood semantics.
Reranker RERANK_CONF, model/ver, cutoff Top-k order flips alter ΔS and coverage.
Tokenizer/Analyzer TOK_VER, ANALYZER_CONF CJK/diacritics/full/half width change token boundaries.
Chunker CHUNK_SCHEMA_VER, overlap, window Alters anchor alignment and traceability.
Index INDEX_HASH, build time, doc schema Guarantees the retriever sees the tested graph.
Tools/Functions TOOL_SCHEMA_VER JSON looseness raises ΔS and loop risk.

Acceptance targets

  • Prod = Staging for: MODEL_VER, PROMPT_VER, EMBED_MODEL_VER, EMBED_DIM, NORM, metric, RERANK_CONF, TOK_VER, ANALYZER_CONF, CHUNK_SCHEMA_VER, INDEX_HASH.
  • ΔS(question, retrieved) ≤ 0.45 and coverage ≥ 0.70 on gold set after pinning.
  • λ convergent across 3 paraphrases and 2 seeds with the pinned headers.
  • First call cold-start passes in ≤ 5 minutes.

60-second lock checklist

  1. Freeze versions in config
    Single source of truth (env or KV) for all fields above; deny deploy if any are unset.
  2. Emit version headers on every request
    Log and return: all pins + BUILD_ID and GIT_SHA. Refuse if missing.
  3. Index integrity
    Retriever must assert INDEX_HASH matches; otherwise hard-fail with fix tip.
  4. Semantic sanity
    Run 20–40 gold items; verify ΔS/coverage/λ targets before canary.

CI/CD gate (paste-ready)

# opsdeploy/version_lock.yml
checks:
  require_fields:
    - MODEL_VER
    - PROMPT_VER
    - EMBED_MODEL_VER
    - EMBED_DIM
    - NORM
    - metric
    - RERANK_CONF
    - TOK_VER
    - ANALYZER_CONF
    - CHUNK_SCHEMA_VER
    - INDEX_HASH
  assert_equal_envs:
    - { field: MODEL_VER, envs: [staging, prod] }
    - { field: PROMPT_VER, envs: [staging, prod] }
    - { field: EMBED_MODEL_VER, envs: [staging, prod] }
    - { field: INDEX_HASH, envs: [staging, prod] }
  deny_if_missing: true
  deny_if_changed_without_alias_swap: [INDEX_HASH]
decision:
  on_fail: block_rollout
  on_pass: proceed
artifacts:
  - logs/version_manifest.json
  - logs/index_hash.txt

Runtime header contract (client/server)

Client must send X-WFGY: MODEL_VER,PROMPT_VER,EMBED_MODEL_VER,EMBED_DIM,NORM,metric,RERANK_CONF,TOK_VER,ANALYZER_CONF,CHUNK_SCHEMA_VER,INDEX_HASH,BUILD_ID,GIT_SHA

Server must validate

  • All fields present.
  • Exact match to server’s pins.
  • If mismatch: return 409 + a JSON diff and links to fix pages.

Typical failure → exact fix

Version manifest example

{
  "MODEL_VER": "gpt-4o-2025-07-15",
  "PROMPT_VER": "p-2025.08.31-h1",
  "EMBED_MODEL_VER": "text-embedding-3-large",
  "EMBED_DIM": 3072,
  "NORM": "l2",
  "metric": "cosine",
  "RERANK_CONF": "bge-rerank-v2@top32",
  "TOK_VER": "tiktoken-2025.07",
  "ANALYZER_CONF": "icu-cjk-folding-v3",
  "CHUNK_SCHEMA_VER": "s128/o32",
  "INDEX_HASH": "a1d9f0...e7",
  "BUILD_ID": "2025.08.31.01",
  "GIT_SHA": "abcdef1"
}

Rollback note

If any pin drifts in prod, treat as outage. Block writes, flip to Blue/previous alias, then reopen with: blue_green_switchovers.md

🔗 Quick-Start Downloads (60 sec)

Tool Link 3-Step Setup
WFGY 1.0 PDF Engine Paper 1️⃣ Download · 2️⃣ Upload to your LLM · 3️⃣ Ask “Answer using WFGY + <your question>”
TXT OS (plain-text OS) TXTOS.txt 1️⃣ Download · 2️⃣ Paste into any LLM chat · 3️⃣ Type “hello world” — OS boots instantly

Explore More

Layer Page What it’s for
⭐ Proof WFGY Recognition Map External citations, integrations, and ecosystem proof
⚙️ Engine WFGY 1.0 Original PDF tension engine and early logic sketch (legacy reference)
⚙️ Engine WFGY 2.0 Production tension kernel for RAG and agent systems
⚙️ Engine WFGY 3.0 TXT based Singularity tension engine (131 S class set)
🗺️ Map Problem Map 1.0 Flagship 16 problem RAG failure taxonomy and fix map
🗺️ Map Problem Map 2.0 Global Debug Card for RAG and agent pipeline diagnosis
🗺️ Map Problem Map 3.0 Global AI troubleshooting atlas and failure pattern map
🧰 App TXT OS .txt semantic OS with fast bootstrap
🧰 App Blah Blah Blah Abstract and paradox Q&A built on TXT OS
🧰 App Blur Blur Blur Text to image generation with semantic control
🏡 Onboarding Starter Village Guided entry point for new users

If this repository helped, starring it improves discovery so more builders can find the docs and tools.
GitHub Repo stars