When several sub-agents (or several steps) contribute to one result, it is surprisingly hard to say how much each one actually mattered. The naive answers are wrong in opposite directions: splitting credit equally ignores that some steps did the real work, and crediting whoever produced the final text ignores everyone whose output it was built on. Getting this right matters for anything that routes reward, budget, or trust by contribution: which sub-agent to keep, which tool earned its cost, which step to cut.
There is a clean way to measure it borrowed from cooperative game theory. A contributor's marginal value is how much the result gets worse when you remove them, averaged over the orders in which contributors could have been added. The leave-one-out version (remove one, measure the drop) is the cheap approximation; the Shapley version (average the marginal drop across orderings) is the fairer one when contributions interact.
What the notebook would cover:
- A small multi-agent task with a measurable final score, where it is obvious by construction which sub-agent did the heavy lifting and which rode along.
- Leave-one-out credit first: re-run the task with each sub-agent removed, attribute the score drop. Show where it is fine and where it misleads (two agents that each cover for the other both look removable).
- Shapley credit on the same task: average the marginal contribution across orderings, and show it correctly separates the load-bearing agent from the passenger.
- The honest cost: exact Shapley is exponential in the number of contributors, so the notebook should show the sampling approximation and how many samples are enough for a stable ranking.
- The boundary worth stating: this measures contribution to a measured outcome. It is only as honest as the score function, so a weak or gameable score makes the attribution weak too.
It runs on the messages API with a small agent harness. I would target the agents section. This is the measurement piece under the cost and routing proposals already open — once you can attribute contribution, you can price and route by it.
When several sub-agents (or several steps) contribute to one result, it is surprisingly hard to say how much each one actually mattered. The naive answers are wrong in opposite directions: splitting credit equally ignores that some steps did the real work, and crediting whoever produced the final text ignores everyone whose output it was built on. Getting this right matters for anything that routes reward, budget, or trust by contribution: which sub-agent to keep, which tool earned its cost, which step to cut.
There is a clean way to measure it borrowed from cooperative game theory. A contributor's marginal value is how much the result gets worse when you remove them, averaged over the orders in which contributors could have been added. The leave-one-out version (remove one, measure the drop) is the cheap approximation; the Shapley version (average the marginal drop across orderings) is the fairer one when contributions interact.
What the notebook would cover:
It runs on the messages API with a small agent harness. I would target the agents section. This is the measurement piece under the cost and routing proposals already open — once you can attribute contribution, you can price and route by it.