Skip to content

Commit b7adc04

Browse files
Roy tongRoy tong
authored andcommitted
Draft 0.4.1 Consistency & Productization Pass:COMMERCIAL 关系模型(Offering 商业对象、Event/Unit/Quantity 三分离、Consumer Account、Quote/Pricing Policy、Metering Ledger、独立毕业);Whitepaper 校准(Capability 定义、thesis & assumptions、Emerging Evidence、Value Evidence Ladder、Distribution Events、措辞降确定性);DATA Verification 解耦 + raw/derived 分层;METRICS M3 grain 修正 + Active Clients 迁移;QUALITY provenance + coverage_basis;全仓 S0-S4/L2/E2/Receipt 术语迁移;product/ 六文档 + Caller Identity Confidence;README 瘦身 + claims 对齐;zh whitepaper 改名
1 parent 1081318 commit b7adc04

31 files changed

Lines changed: 1047 additions & 233 deletions

README.md

Lines changed: 29 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,8 @@ AgentMeasure defines a common language for measuring how AI agents discover, cho
1313
> AgentMeasure is **not** a payment protocol, marketplace, or universal ranking system.
1414
> It standardizes the facts and measurement semantics those systems can build on.
1515
16+
[Whitepaper](whitepaper/measuring-software-used-by-ai-agents.md) · [白皮书(中文)](whitepaper/measuring-software-used-by-ai-agents.zh-CN.md) · [Core Specification](standard/CORE.md) · [中文](README.zh-CN.md)
17+
1618
---
1719

1820
## Why capabilities need a new measurement layer
@@ -65,6 +67,10 @@ Measured Usage ≠ Billable Usage
6567

6668
The last inequality is why the Operation/Attempt model matters commercially: 3 attempts of one operation are not 3 billable operations — unless the metering policy says so.
6769

70+
> The extended thesis — economic units, scarcity, measurement before monetization,
71+
> metering semantics, and the evidence that commerce is arriving before measurement —
72+
> lives in [docs/CAPABILITY-ECONOMY.md](docs/CAPABILITY-ECONOMY.md).
73+
6874
## The measurement view: Reach → Choice → Use → Utility → Value
6975

7076
AgentMeasure defines **metric families**, not a universal KPI. A search capability, a booking API, and a compute job have different value structures.
@@ -123,29 +129,33 @@ Four things a real developer needs to know:
123129
>
124130
> AgentMeasure is **not** on the critical request path.
125131
132+
Product architecture (Provider SDK → local buffer → hosted ingestion → dashboard):
133+
[product/ARCHITECTURE.md](product/ARCHITECTURE.md).
134+
126135
## What AgentMeasure measures today
127136

128137
- **Decision Opportunity / Candidate Set / Presentation / Selection** — the four objects of choice; the Observed Selection Rate denominator is Presented, not Available
129138
- **Software Entity → Capability → Interaction Surface** — what exists, what it can do, and the observable interface; observation happens on surfaces, attribution resolves to entities through the machine-readable registry
130139
- **Operation / Attempt** — one logical use vs. one execution; **retries are multiple attempts of one operation**, not a validity class (they are kept as reliability signals, not counted as distinct logical uses)
131140
- **Qualified Usage** — production usage after excluding test, benchmark, synthetic, replay, duplicate and other invalid traffic according to policy
132-
- **Result Consumption / Effect Confirmation** — the result was used by the task, or the intended world-state change was confirmed (Interaction Classes: Information / Action / Transaction / Computation / Communication / …)
141+
- **Result Consumption***defined, reference partial*: the result was used by the task
142+
- **Effect Confirmation***domain model defined, metric planned for Draft 0.5*: the intended world-state change was confirmed
133143
- **Measurement Label** — a nutrition label for every public number (coverage / sampling / policy / method)
134144

135145
The full model lives in the [Core Specification](standard/CORE.md), [Metrics](standard/METRICS.md), [Entity](standard/ENTITY.md), and [Quality](standard/QUALITY.md). The README is not a spec summary.
136146

137147
## From measurement to CaaS
138148

139-
```text
140-
Capability
141-
142-
143-
Discover → Choose → Use → Deliver Value → Measure → Meter → Price / Pay / Settle
144-
```
145-
146-
AgentMeasure standardizes the data and semantics of the **first five steps** — discovery through measurement. Metering and commercial attribution are future extensions; payment rails can be provided by existing payment infrastructure.
149+
AgentMeasure is **progressively standardizing** the measurement chain from discovery
150+
and choice through execution, utility and value: core usage semantics are defined
151+
today; utility/value and commercial metering remain active drafts. Metering and
152+
commercial attribution are future extensions; payment rails can be provided by
153+
existing payment infrastructure.
147154

148155
> **AgentMeasure standardizes economic facts, not money movement.**
156+
>
157+
> Extended thesis: [docs/CAPABILITY-ECONOMY.md](docs/CAPABILITY-ECONOMY.md) ·
158+
> Economic semantics: [extensions/COMMERCIAL.md](extensions/COMMERCIAL.md)(Experimental)
149159
150160
## What AgentMeasure is / is not
151161

@@ -173,20 +183,28 @@ AgentMeasure standardizes the data and semantics of the **first five steps** —
173183
```bash
174184
git clone https://github.com/roy-tong/AgentMeasure && cd AgentMeasure
175185
python3 conformance/runners/run_metrics.py # metric vectors (M2.2 / M2.5 / M4.1)
176-
python3 verify_vectors.py # receipt / correlation / operation vectors
186+
python3 verify_vectors.py # verification / correlation / operation vectors
177187
python3 registry/validate_entities.py # validate the machine-readable registry
178188
```
179189

180190
## Product MVP — in development
181191

182-
The first product path is **Remote MCP / API Capability Measurement**: an AgentMeasure Provider SDK that emits observations from the provider side (no agent-side install), feeding a collector and hosted analytics. The SDK and hosted analytics are not implemented yet — the standard, reference collector, and conformance suite are.
192+
The first product path is **Remote MCP / API Capability Measurement**: an AgentMeasure
193+
Provider SDK that emits observations from the provider side (no agent-side install),
194+
feeding a collector and hosted analytics. The SDK and hosted analytics are not
195+
implemented yet — the standard, reference collector, and conformance suite are.
196+
197+
Scope and acceptance: [product/MVP.md](product/MVP.md) · SDK contract:
198+
[product/PROVIDER-SDK.md](product/PROVIDER-SDK.md) · Deployment:
199+
[product/DEPLOYMENT.md](product/DEPLOYMENT.md)
183200

184201
## Repository map
185202

186203
```text
187204
AgentMeasure/
188205
├── standard/ # the normative standard (CORE / METRICS / QUALITY / DATA / ...)
189206
├── extensions/ # experimental, non-normative profiles (COMMERCIAL.md)
207+
├── product/ # product architecture (SDK / hosted analytics, in development)
190208
├── whitepaper/ # methodology papers (EN/CN)
191209
├── conformance/ # language-neutral test vectors + runners
192210
├── reference/ # reference implementation (collector + adapters)

README.zh-CN.md

Lines changed: 22 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,8 @@ AgentMeasure 定义一套共同语言,衡量 AI Agent 如何发现、选择、
1313
> AgentMeasure **不是**支付协议、市场或通用排名系统。
1414
> 它标准化的是这些系统可以构建于其上的事实与测量语义。
1515
16+
[Whitepaper](whitepaper/measuring-software-used-by-ai-agents.md) · [中文白皮书](whitepaper/measuring-software-used-by-ai-agents.zh-CN.md) · [Core Specification](standard/CORE.md) · [English](README.md)
17+
1618
---
1719

1820
## 为什么 Capability 需要新的测量层
@@ -66,6 +68,9 @@ Agent → Capability → 执行 → 结果
6668
最后一个不等式正是 Operation/Attempt 模型的商业意义:一次 Operation 的 3 次
6769
Attempt ≠ 3 次可计费操作——除非计量策略如此规定。
6870

71+
> 完整论点——经济单元、稀缺性、先计量后变现、计量语义,以及"商业先于计量到来"
72+
> 的证据——见 [docs/CAPABILITY-ECONOMY.md](docs/CAPABILITY-ECONOMY.md)
73+
6974
## Measurement View:Reach → Choice → Use → Utility → Value
7075

7176
AgentMeasure 定义 **Metric Families**,不定义全局北极星。搜索能力、预订 API、计算任务的价值结构各不相同。
@@ -128,31 +133,32 @@ Claude / Codex
128133
>
129134
> AgentMeasure **不在请求关键路径上**
130135
136+
产品架构(Provider SDK → 本地缓冲 → 托管 ingestion → 面板):
137+
[product/ARCHITECTURE.md](product/ARCHITECTURE.md)
138+
131139
## 今天测量什么
132140

133141
- **Decision Opportunity / Candidate Set / Presentation / Selection** —— 选择的四个对象;Observed Selection Rate 的分母是 Presented,不是 Available
134142
- **Software Entity → Capability → Interaction Surface** —— 存在什么、能做什么、怎么交互;观察发生在 surface 层,归属经机器可读 registry 解析到 entity
135143
- **Operation / Attempt** —— 一次逻辑使用 vs 一次执行;**重试 = 同一 Operation 的多个 Attempt,不再是 validity 分类**(作为可靠性信号保留,不当作多次逻辑使用计数)
136144
- **Qualified Usage** —— 按策略排除 test / benchmark / synthetic / replay / duplicate 等无效流量后的生产使用
137-
- **Result Consumption / Effect Confirmation** —— 结果被任务使用,或预期的世界状态改变被确认(Interaction Classes:Information / Action / Transaction / Computation / Communication / …)
145+
- **Result Consumption** —— *已定义,参考实现部分*:结果被任务使用
146+
- **Effect Confirmation** —— *领域模型已定义,指标计划于 Draft 0.5*:预期的世界状态改变被确认
138147
- **Measurement Label** —— 每个公开数字的营养成分表(覆盖 / 采样 / 口径 / 方法)
139148

140149
完整模型见 [Core Specification](standard/CORE.md) / [Metrics](standard/METRICS.md) /
141150
[Entity](standard/ENTITY.md) / [Quality](standard/QUALITY.md)。README 不是 spec 摘要。
142151

143152
## 从 Measurement 到 CaaS
144153

145-
```text
146-
Capability
147-
148-
149-
Discover → Choose → Use → Deliver Value → Measure → Meter → Price / Pay / Settle
150-
```
151-
152-
AgentMeasure 标准化**前五步**——从发现到测量——产生的数据与语义。计量与商业归因
153-
是未来扩展;支付轨道可以由现有支付基础设施提供。
154+
AgentMeasure 正在**渐进标准化**从发现与选择、经执行到效用与价值的测量链:核心使用
155+
语义今天已定义;效用/价值与商业计量仍是活跃草案。计量与商业归因是未来扩展;支付
156+
轨道可以由现有支付基础设施提供。
154157

155158
> **AgentMeasure 标准化经济事实,不移动金钱。**
159+
>
160+
> 完整论点:[docs/CAPABILITY-ECONOMY.md](docs/CAPABILITY-ECONOMY.md) ·
161+
> 经济语义:[extensions/COMMERCIAL.md](extensions/COMMERCIAL.md)(Experimental)
156162
157163
## AgentMeasure 是 / 不是
158164

@@ -180,7 +186,7 @@ AgentMeasure 标准化**前五步**——从发现到测量——产生的数据
180186
```bash
181187
git clone https://github.com/roy-tong/AgentMeasure && cd AgentMeasure
182188
python3 conformance/runners/run_metrics.py # 指标 vectors(M2.2 / M2.5 / M4.1)
183-
python3 verify_vectors.py # receipt / correlation / operation vectors
189+
python3 verify_vectors.py # verification / correlation / operation vectors
184190
python3 registry/validate_entities.py # 校验机器可读 registry
185191
```
186192

@@ -190,12 +196,17 @@ python3 registry/validate_entities.py # 校验机器可读 registry
190196
SDK 在 Provider 侧产出 observations(无需 Agent 侧安装),接入 collector 与托管分析。
191197
SDK 与托管分析尚未实现——标准、参考 collector 与 conformance 套件已就绪。
192198

199+
范围与验收:[product/MVP.md](product/MVP.md) · SDK 契约:
200+
[product/PROVIDER-SDK.md](product/PROVIDER-SDK.md) · 部署:
201+
[product/DEPLOYMENT.md](product/DEPLOYMENT.md)
202+
193203
## 仓库结构
194204

195205
```text
196206
AgentMeasure/
197207
├── standard/ # 规范性标准本体(CORE / METRICS / QUALITY / DATA / ...)
198208
├── extensions/ # 实验性、非规范性 profile(COMMERCIAL.md)
209+
├── product/ # 产品架构(SDK / 托管分析,in development)
199210
├── whitepaper/ # 方法论论文(中英)
200211
├── conformance/ # 语言无关 test vectors + runners
201212
├── reference/ # 参考实现(collector + adapters)

docs/CAPABILITY-ECONOMY.md

Lines changed: 86 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,86 @@
1+
# The Agent Capability Economy — Extended Thesis
2+
3+
> 这是项目 Vision 的展开叙事(非规范性)。README 只保留摘要;
4+
> 规范定义见 `standard/`,经济语义见 `extensions/COMMERCIAL.md`(Experimental)。
5+
6+
## 1. 经济单位在变(增量,不是替代)
7+
8+
```text
9+
SaaS
10+
Human → Application → Seat / Month
11+
12+
API Economy
13+
Software → API → Request / Token
14+
15+
Agent Capability Economy
16+
Agent → Capability → Operation / Outcome
17+
```
18+
19+
**Alongside** seat-based SaaS and request-based APIs, callable capabilities are
20+
emerging as a new economic unit for agent-mediated software consumption. Seats will
21+
not disappear; requests will not disappear; capability/outcome units add a new layer.
22+
23+
## 2. 稀缺性下移
24+
25+
接口(Skill / MCP adapter / CLI)越来越廉价易造;稀缺层是接口背后的访问权:
26+
27+
```text
28+
Data · Compute · Action · Permission · Trust · Real-world fulfillment
29+
```
30+
31+
> **Interfaces may become cheap to create; capabilities remain scarce to deliver.**
32+
33+
## 3. 计量先于变现
34+
35+
CaaS 要能定价、计费与建立声誉,先要有共同的测量语义:
36+
37+
```text
38+
1 user task → 1 Operation → 3 retries:收 1 次还是 3 次?(Metering Policy 决定)
39+
返回成功但 Agent 忽略结果:价值交付了吗?(Consumption / Effect)
40+
预订 API 执行了但从未确认:能力履约了吗?(Effect Confirmation)
41+
任务成功了但没有这个能力也能成功:Provider 能主张价值吗?(Incrementality)
42+
```
43+
44+
**Measurement before monetization** —— 这就是 AgentMeasure 的 wedge。
45+
46+
## 4. 现实证据:商业先于计量到来
47+
48+
- Cloudflare Agents SDK:MCP Tool 按调用定价并经 x402 收费
49+
[Charge for MCP tools](https://developers.cloudflare.com/agents/agentic-payments/x402/charge-for-mcp-tools/)
50+
- Coinbase x402 Bazaar:Agent 搜索带价格/schema 的服务并经 MCP 付费调用
51+
[x402 Bazaar](https://docs.cdp.coinbase.com/x402/bazaar)
52+
- OpenAI × Stripe Agentic Commerce Protocol(ACP):真实 agentic commerce 流程
53+
[报道](https://www.digitaltransactions.net/openai-and-stripe-are-the-latest-fintechs-to-enable-agentic-commerce/)
54+
55+
> **Payment and discovery infrastructure is arriving before common capability
56+
> measurement semantics.**
57+
58+
## 5. 从测量到计费(语义链)
59+
60+
```text
61+
Capability → Discover → Choose → Use → Deliver Value → Measure → Meter → Price / Pay / Settle
62+
```
63+
64+
- AgentMeasure 渐进标准化前五步产生的数据与语义(当前已定义:Core usage semantics)
65+
- Metering / commercial attribution 是未来扩展(extensions/COMMERCIAL.md)
66+
- Payment rails 由现有支付基础设施提供
67+
68+
> **AgentMeasure standardizes economic facts, not money movement.**
69+
70+
## 6. 三条线永不耦合
71+
72+
| 线 | 负责 | 文档 |
73+
| --- | --- | --- |
74+
| Core | 测事实 | standard/(规范性) |
75+
| Commercial Extension | 事实 → 经济单位的解释 | extensions/COMMERCIAL.md(实验性) |
76+
| Product | 帮用户获得这些事实 | product/(in development) |
77+
| Payment 系统 || 外部 |
78+
79+
**CaaS 是 Vision,不是 Core Standard 成立的必要条件**:AgentMeasure Core 1.0
80+
独立毕业为 Agent 软件测量标准。
81+
82+
## 7. 叙事锁定
83+
84+
- **Today**:让开发者知道 Agent 如何真实使用自己的能力(Remote Capability Analytics)
85+
- **Direction**:让 Capability 可以跨 Agent 被统一度量、比较和计量
86+
- **Vision**:成为 CaaS 与 Agent Capability Economy 的统一计量基础

docs/adapters.md

Lines changed: 10 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -2,23 +2,25 @@
22

33
> 每个 adapter 能可靠提供什么、不能提供什么——公开声明局限是测量项目的可信度基础。
44
> ✅ = 稳定可靠 · ⚠️ = 部分/受限 · ❌ = 不提供 · ? = 待验证
5+
> 词汇(Draft 0.4.1):Selection / Operation / Attempt / Completion / Consumption;
6+
> 生命周期阶段 = selected · invoked · completed · consumed。
57
68
| Capability | Codex Hook | Claude OTel | DSH Plugin | MCP Server (wrapper) |
79
| --- | --- | --- | --- | --- |
8-
| Selected(L0|||||
9-
| Executed(L1|||||
10-
| Success(L2 outcome) | ⚠️ 不可靠(Bash 非零退出仍触发 PostToolUse) ||||
10+
| Selection(生命周期 selected|||||
11+
| Executed(invoked|||||
12+
| Completion outcome(completed| ⚠️ 不可靠(Bash 非零退出仍触发 PostToolUse) ||||
1113
| Duration | ⚠️ 无官方时间戳 ||||
1214
| Tool call ID |`tool_use_id` |`gen_ai.tool.call.id` |`callId` | depends on server |
1315
| Trace ID | ❌ 官方 schema 无 || depends ||
14-
| Result Consumed(S3||**MCP 消费信号**`mcp_server.name`/`mcp_tool.name` 出现在后续 request telemetry) | ? ||
16+
| Consumption(result consumed||**MCP 消费信号**`mcp_server.name`/`mcp_tool.name` 出现在后续 request telemetry) | ? ||
1517
| Agent-side observation |||||
1618
| Server-side observation |||||
17-
| 独立构成 E2 | ❌(单侧) | ❌(单侧) | ❌(单侧,harness 内配对仅生命周期) | ❌(单侧) |
19+
| 独立构成 cross-side corroborated | ❌(单侧) | ❌(单侧) | ❌(单侧,harness 内配对仅生命周期) | ❌(单侧) |
1820

1921
## 说明
2022

2123
- **Codex Hook 能力边界**来自官方 `PostToolUse` schema:`session_id / model / turn_id / tool_name / tool_use_id`;无 trace_id、无可靠时间戳、无成败判定。
22-
- **Claude Code 是 S3(Result Consumed)的第一个实证平台**`claude_code.tool_result` span 提供 `tool_name/tool_use_id/success/duration_ms/error_type`;v2.1.222+ 的 API request telemetry 在**实际消费了某个 MCP tool result** 时才带 `mcp_server.name/mcp_tool.name`——tool_result(tool_use_id=X)→ 下一次 request(mcp_tool.name)即消费证据链。
23-
- **DSH Plugin**`tool/call ↔ tool/result` 的 harness 内配对只证明生命周期完成(L2),不构成独立佐证;evidence 由 verifier 计算。
24-
- **E2 需要跨侧**:任何单侧 adapter 都无法独立构成 E2;E2 = 同一 invocation 的 ≥2 条独立 observer 观察。
24+
- **Claude Code 是 Consumption 的第一个实证平台**`claude_code.tool_result` span 提供 `tool_name/tool_use_id/success/duration_ms/error_type`;v2.1.222+ 的 API request telemetry 在**实际消费了某个 MCP tool result** 时才带 `mcp_server.name/mcp_tool.name`——tool_result(tool_use_id=X)→ 下一次 request(mcp_tool.name)即消费证据链。
25+
- **DSH Plugin**`tool/call ↔ tool/result` 的 harness 内配对只证明生命周期完成(completed),不构成独立佐证;evidence 由 verifier 计算。
26+
- **cross-side corroborated 需要跨侧**:任何单侧 adapter 都无法独立构成同一 attempt 的 ≥2 条独立 observer 观察。

0 commit comments

Comments
 (0)