chore(specify): add C11-C14 constitution rules + V4.7/V4.8/V4.9 verification gates#49
Open
fangxiu-wf wants to merge 3 commits into
Open
chore(specify): add C11-C14 constitution rules + V4.7/V4.8/V4.9 verification gates#49fangxiu-wf wants to merge 3 commits into
fangxiu-wf wants to merge 3 commits into
Conversation
…cation gates New constitutional rules from claude plugin bug fixes: - C11: Anthropic provider input_tokens must be full value (api + cache_read + cache_creation) - C12: JSONL records must include trace_id/span_id/parent_span_id (W3C format) - C13: CONFIG_DIR env var compatibility for containerized deployments Verification checklist updates: - V4.7: JSONL span_id completeness with detailed test methods - V4.8: input_token full value assertion for Anthropic provider - V3: additional failure points for anchor steal and cache tokens Reverse impact: openclaw plugin has the same C11 bug (line 967) — tracked separately. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
This PR updates the Spec-Kit “constitution” and verification checklist templates to codify three cross-plugin rules (C11–C13) derived from fixes in opentelemetry-instrumentation-claude, and extends the V4 E2E gate with two additional verification items plus detailed procedures (V4.7/V4.8).
Changes:
- Add constitution clauses C11 (Anthropic full input_tokens), C12 (JSONL trace/span linkage IDs), and C13 (CONFIG_DIR env compatibility).
- Expand V4 E2E checklist from 6 to 8 assertions (adds JSONL span linkage integrity + Anthropic cache token accounting).
- Add detailed test methodology sections for V4.7 (JSONL IDs) and V4.8 (Anthropic input_tokens aggregation), plus V3 common failure points.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
.specify/templates/otel-plugin/verification-checklist.md |
Extends V3 failure points and V4 verification gates; adds detailed V4.7/V4.8 procedures. |
.specify/memory/constitution.md |
Adds C11–C13 rules and updates LLM required fields note to reference C11. |
Comments suppressed due to low confidence (1)
.specify/templates/otel-plugin/verification-checklist.md:99
- 标题写的是“input_token 全量值”，但文中/宪法条款使用的是
input_tokens（复数）字段名。建议统一为input_tokens，避免读者误以为是不同字段或写错断言路径。
### V4.8 详细测试方法 — input_token 全量值(Anthropic provider)
1. 构造含 cache token 的 mock LLM 事件:
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| **通过条件**:6/6 PASS。 | ||
| | **7** | **JSONL 每条 record 含非 null 的 `trace_id`(32位hex)、`span_id`(16位hex)、`parent_span_id`(16位hex);同一 turn 内 trace_id 一致** | **C12** | | ||
| | **8** | **含 cache token 的 LLM 事件,`usage.input_tokens` = api_input + cache_read + cache_creation(不是仅 api_input)** | **C11** | | ||
|
|
Comment on lines
+244
to
+248
| 当目标 agent 支持自定义配置目录的环境变量时(如 `CLAUDE_CONFIG_DIR`、`CODEX_CONFIG_DIR`),插件的配置文件路径必须跟随: | ||
|
|
||
| ```js | ||
| const configDir = process.env.CLAUDE_CONFIG_DIR || path.join(os.homedir(), ".claude"); | ||
| const CONFIG_PATH = path.join(configDir, "otel-config.json"); |
Adds a new constitutional rule covering a class of bugs hit twice now (claude alignWithHookEvents history-transcript ghost spans, codex multi-turn token usage stuck on turn-1 values). The shared shape is: a plugin consumes hook event stream + transcript file, but tries to align them by event-stream order — which silently breaks the moment state lifecycles diverge (per-turn Stop hook + session-cumulative transcript). C14 names the rule explicitly: when both data sources are in play, prefer explicit correlation keys over implicit positional alignment. Priority spelled out — turn_id > message_id > byteOffset > wall-clock window. Fallback paths must be documented and regression-tested rather than emergent. This is "preferred", not "must" — some agent transcripts genuinely have no explicit key and only byteOffset / time windows work. Companion changes: - verification-checklist V4.9: a multi-turn alignment assertion that each turn's LLM span token attrs match the fixture 1:1 across N>=3 turns, with the explicit anti-regression bound that turn 2 / turn 3 values must NOT equal turn 1's. Single-turn-equality assertions alone don't catch this class of bug. - plan-template §2.5: forces plan stage to answer three questions — which correlation key, how state crosses hook boundaries, and how duplicate / heartbeat events are deduped. Constitution version bump 1.0.0 → 1.1.0. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
fangxiu-wf
changed the title
Three small consistency fixes flagged by the Copilot reviewer:
- verification-checklist V4 header: "6 项必须断言" → "9 项必须断言"
(table grew from 6 to 9 with V4.7/V4.8/V4.9 additions)
- V4.8 method title: "input_token" → "input_tokens" to match the
field name used in the constitution and assertions
- constitution C13 sample code: replace hardcoded CLAUDE_CONFIG_DIR
/ ".claude" with <AGENT>_CONFIG_DIR / ".<agent>" placeholders, so
the example does not contradict the very rule it illustrates
("不能硬编码 ~/.<agent>/")
No semantic changes — purely doc consistency.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
从
opentelemetry-instrumentation-claude与opentelemetry-instrumentation-codex插件的真实 bug 修复中提炼出的通用规范,固化为宪法条款和验证准出条件。新增宪法条款
input_tokens必须为全量值(api + cache_read + cache_creation)trace_id/span_id/parent_span_id(W3C 格式)CLAUDE_CONFIG_DIR/CODEX_CONFIG_DIR环境变量clearState,改为清空 events + 保留水位线字段(2026-05-26 新增)验证 Checklist 更新
模板更新
plan-template.md§2.5 新增"Hook ↔ Transcript 对齐策略"小节(C14 配套):plan 阶段必须回答三个问题——关联键选择 / 跨 hook 边界状态持久化 / fallback 与去重反向影响
opentelemetry-instrumentation-claude: C11-C13 已在 PR feat(claude): v0.2.0-beta — transcript tracing, per-turn traces, config file, JSONL logging #40 中修复;C14 早期已通过 byteOffset 增量(Issue 7.5)/ 时间锚点(7.6)/ 后向配对(7.9)实现,符合规范opentelemetry-instrumentation-codex: C14 在 PR feat(codex): add OpenTelemetry instrumentation plugin for OpenAI Codex CLI #41 commita89392b中已实施(byteOffset + turn_id 双重对齐 + state 不删)opentelemetry-instrumentation-openclaw: 存在 C11 同类 bug(index.ts:967),需后续单独修复Test plan
🤖 Generated with Claude Code