You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
**A proposed open measurement standard for software used by AI agents.**
3
+
**Open measurement infrastructure for the Agent Capability Economy.**
4
4
5
-
> AgentMeasure is an open measurement standard that uses a unified data language to
6
-
> measure how AI agents discover, choose, and use software — and how much value that
7
-
> software ultimately creates.
8
-
>
9
-
> Traditional software metrics measure what humans download and use. AgentMeasure
10
-
> measures what agents see, what they choose, what they actually use, and whether
11
-
> those choices create value.
5
+
AgentMeasure defines a common language for measuring how AI agents discover, choose, use, and derive value from software capabilities.
6
+
7
+
**Today:** measure agent-facing capability usage.
8
+
**Next:** make capabilities comparable and meterable.
9
+
**Long term:** provide the measurement foundation for Capability as a Service (CaaS).
10
+
11
+
**Reach → Choice → Use → Utility → Value**
12
+
13
+
> AgentMeasure is **not** a payment protocol, marketplace, or universal ranking system.
14
+
> It standardizes the facts and measurement semantics those systems can build on.
15
+
16
+
---
17
+
18
+
## Why capabilities need a new measurement layer
19
+
20
+
The software consumer is changing from humans to agents, and the economic unit is shifting from software seats toward callable capabilities.
21
+
22
+
```text
23
+
Skill / MCP / CLI / SDK
24
+
↓
25
+
Describe / expose / distribute a capability
26
+
27
+
Capability
28
+
↓
29
+
Data / Compute / Action / Permission / Transaction
30
+
↓
31
+
Creates scarce economic value
32
+
```
33
+
34
+
**Interfaces may become cheap to create; capabilities remain scarce to deliver.**
35
+
36
+
The first generation of capability distribution is already here — open Skills, open MCP adapters, open CLIs. The scarce layer underneath them is what the next economy is built on: proprietary data, compute, execution, permissions, and real-world fulfillment.
If a capability is to become an economic unit that agents can discover, compare, and eventually purchase automatically, it must first be **identifiable, measurable, and comparable under a shared semantics**. That is what AgentMeasure provides.
16
53
17
-
Agents are becoming software's new consumers, but every existing signal breaks down —
18
-
download counts measure humans, not agents; self-reported installs are gameable;
19
-
registries don't expose adoption data.
54
+
Traditional usage metrics cannot support this economy — the old chain breaks at every link, and the last link is new:
20
55
21
56
```text
22
57
Install ≠ Available
@@ -25,63 +60,133 @@ Presented ≠ Selected
25
60
Selected ≠ Used
26
61
Used ≠ Useful
27
62
Useful ≠ Incremental Value
63
+
Measured Usage ≠ Billable Usage
28
64
```
29
65
30
-
AgentMeasure answers five questions: **Reach → Choice → Use → Utility → Value**
66
+
The last inequality is why the Operation/Attempt model matters commercially: 3 attempts of one operation are not 3 billable operations — unless the metering policy says so.
31
67
32
-
1.**Reach** — does my software enter an agent's selection range?
33
-
2.**Choice** — when an agent has the opportunity, does it pick me? (Observed Selection Rate / Conditional Choice Share)
34
-
3.**Use** — after being chosen, is it actually used?
35
-
4.**Utility** — does using it produce useful results? (Result Consumption)
36
-
5.**Value** — without me, would the agent's outcome be worse? (Incrementality)
68
+
## The measurement view: Reach → Choice → Use → Utility → Value
37
69
38
-
## Five-layer measurement framework
70
+
AgentMeasure defines **metric families**, not a universal KPI. A search capability, a booking API, and a compute job have different value structures.
39
71
40
72
| Layer | Question | Representative metrics |
41
73
| --- | --- | --- |
42
-
| Reach | Did it enter the agent world | Presented Opportunities、Active Clients |
43
-
| Choice | Would it be chosen when given the chance | Observed Selection Rate、Conditional Choice Share |
44
-
| Use | Is it usable once chosen | Logical Invocations、Completion Rate、Success Rate |
45
-
| Utility | Was the result used | Result Consumed Rate |
46
-
| Value | Did it create value | Incremental Task Success（Draft 0.5） |
47
-
48
-
## Core concepts
49
-
50
-
-**Decision Opportunity / Candidate Set / Presentation / Selection** — the four
51
-
objects of choice behavior; the Observed Selection Rate denominator is Presented,
52
-
not Available
53
-
-**Observed Selection Rate** = Observed Selected ÷ Presented — the probability an
54
-
agent picks you when it truly has the chance
55
-
(observed ≠ preference: a "choice" under required/forced constraint is not a preference)
56
-
-**Conditional Choice Share** — head-to-head preference when A and B actually compete
57
-
in the same candidate set
58
-
-**Software Entity / Capability / Interaction Surface** — what exists, what it can
59
-
do, and the observable interface; observation happens on surfaces, attribution
60
-
resolves to entities (see [AgentMeasure Entity](standard/ENTITY.md))
61
-
-**Operation / Attempt** — a logical use vs. a single execution; retries are multiple
62
-
attempts of one operation, not a validity class
63
-
-**Qualified Usage** — real production use after excluding benchmark / test /
64
-
synthetic / retry
65
-
-**Result Consumption** — the result was actually used by the task (≠ successful return)
66
-
-**Incrementality** — would the outcome be worse without this software?
67
-
-**Measurement Label** — a nutrition label for every public number (coverage /
68
-
sampling / policy / method)
69
-
70
-
## Who is this for?
71
-
72
-
| Audience | Entry |
74
+
|**Reach**| Did the capability enter the agent's choice range? | Eligible Opportunities · Presentations · Presentation Rate · Distribution Coverage |
75
+
|**Choice**| When the agent had the chance, did it choose it? | Observed Selection Rate · Conditional Choice Share |
76
+
|**Use**| After selection, was it actually used? | Operations · Attempts · Completion Rate · Success Rate |
77
+
|**Utility**| Did it produce a usable result or confirmed effect? | Result Consumption · Effect Confirmation |
78
+
|**Value**| Did it improve the task outcome? | Incremental Task Success（Draft 0.5） |
79
+
80
+
The five layers are the **measurement view**. The same facts map onto an economic view:
Claim discipline throughout: *observed choice is not preference*. A selection can be made by the model, a router, a workflow, the user, a policy, or the platform; **Observed Selection Rate** reports what was observed, and **Conditional Choice Share** is an *observed head-to-head choice share under comparable candidate conditions* — comparable means the same candidate set, category, choice mode, and decision axes (Decision Authority / Selection Constraint) are declared.
90
+
91
+
## How AgentMeasure works in production
92
+
93
+
```text
94
+
Agent Runtime Capability Provider
95
+
96
+
Claude / Codex
97
+
│
98
+
│ MCP / API
99
+
▼
100
+
┌──────────────────────┐
101
+
│ Customer Capability │
102
+
│ │
103
+
│ AgentMeasure SDK │
104
+
│ Business Handler │
105
+
└──────────┬───────────┘
106
+
│
107
+
observations
108
+
│
109
+
▼
110
+
Collector
111
+
│
112
+
▼
113
+
AgentMeasure Cloud
114
+
```
115
+
116
+
Four things a real developer needs to know:
117
+
118
+
> Your software does **not** need to be open source.
119
+
>
120
+
> MCP is **not** required — it is the first reference surface.
121
+
>
122
+
> Third-party agents do **not** need AgentMeasure installed for provider-side usage measurement.
123
+
>
124
+
> AgentMeasure is **not** on the critical request path.
125
+
126
+
## What AgentMeasure measures today
127
+
128
+
-**Decision Opportunity / Candidate Set / Presentation / Selection** — the four objects of choice; the Observed Selection Rate denominator is Presented, not Available
129
+
-**Software Entity → Capability → Interaction Surface** — what exists, what it can do, and the observable interface; observation happens on surfaces, attribution resolves to entities through the machine-readable registry
130
+
-**Operation / Attempt** — one logical use vs. one execution; **retries are multiple attempts of one operation**, not a validity class (they are kept as reliability signals, not counted as distinct logical uses)
131
+
-**Qualified Usage** — production usage after excluding test, benchmark, synthetic, replay, duplicate and other invalid traffic according to policy
132
+
-**Result Consumption / Effect Confirmation** — the result was used by the task, or the intended world-state change was confirmed (Interaction Classes: Information / Action / Transaction / Computation / Communication / …)
133
+
-**Measurement Label** — a nutrition label for every public number (coverage / sampling / policy / method)
134
+
135
+
The full model lives in the [Core Specification](standard/CORE.md), [Metrics](standard/METRICS.md), [Entity](standard/ENTITY.md), and [Quality](standard/QUALITY.md). The README is not a spec summary.
136
+
137
+
## From measurement to CaaS
138
+
139
+
```text
140
+
Capability
141
+
│
142
+
▼
143
+
Discover → Choose → Use → Deliver Value → Measure → Meter → Price / Pay / Settle
144
+
```
145
+
146
+
AgentMeasure standardizes the data and semantics of the **first five steps** — discovery through measurement. Metering and commercial attribution are future extensions; payment rails can be provided by existing payment infrastructure.
147
+
148
+
> **AgentMeasure standardizes economic facts, not money movement.**
python3 registry/validate_entities.py # validate the machine-readable registry
178
+
```
179
+
180
+
## Product MVP — in development
181
+
182
+
The first product path is **Remote MCP / API Capability Measurement**: an AgentMeasure Provider SDK that emits observations from the provider side (no agent-side install), feeding a collector and hosted analytics. The SDK and hosted analytics are not implemented yet — the standard, reference collector, and conformance suite are.
79
183
80
184
## Repository map
81
185
82
186
```text
83
187
AgentMeasure/
84
-
├── standard/ # the standard itself (CORE / METRICS / QUALITY / DATA / ...)
188
+
├── standard/ # the normative standard (CORE / METRICS / QUALITY / DATA / ...)
The roadmap runs on two tracks — the standard (0.4 objects & quality → 0.5 utility & economic semantics → 1.0) and the product (Remote Capability Analytics → Provider SDK + hosted analytics → metering). See [ROADMAP.md](ROADMAP.md).
0 commit comments