Add gen_ai.compaction.tokens or gen_ai.compaction.enabled attributes to LLMInvocation or Inference span.

[eternalcuriouslearner]

Area(s)

No response

What's missing?

Today all modern llm apis like OpenAI's Responses api or Anthropics Messages api provide an option to compact a conversation with llm. We need to add an attribute to identify both compaction threshold and compaction identifier when llm clients are enabling compaction on their llm api call.

Reference

1. https://developers.openai.com/api/docs/guides/compaction
2. https://platform.claude.com/docs/en/build-with-claude/compaction

Describe the solution you'd like

Add two new span attributes gen_ai.compaction.tokens or gen_ai.compaction.enabled to the LLMInvocation or Inference span

Tip

_{React with 👍 to help prioritize this issue. Please use comments to provide useful context, avoiding +1 or me too, to help us triage it. Learn more here.}

[eternalcuriouslearner]

I am interested to contribute if there is interest from the community.

[Cirilla-zmh]

Is it important to record whether this identifier exists? Do we have a way to record whether a given LLM conversation was compacted? I think the latter seems more useful.

[aabmass]

This seems easy enough to just add as an attribute on inference spans. I agree

Do we have a way to record whether a given LLM conversation was compacted?

+1, is there any thing in the API response?

[eternalcuriouslearner]

On the request side, it is useful to capture whether compaction is enabled, along with the configured token threshold, so we can understand when compaction is expected to kick in over the course of a multi-turn conversation. On the response side, it is also useful to capture whether compaction actually occurred on the server for that turn. We can capture this server-side compaction signal for both the OpenAI and Anthropic APIs.

CC: @Cirilla-zmh @aabmass

[Cirilla-zmh]

On the request side, it is useful to capture whether compaction is enabled, along with the configured token threshold, so we can understand when compaction is expected to kick in over the course of a multi-turn conversation.

Great point! But in fact, I think collecting this configuration is not that meaningful. It only indicates whether the compaction behavior is enabled. In practice, different models can use very different patterns to control compaction, and some may even compact by default. In such cases, recording the compaction signal itself may be more valuable.

One concern is that, there may be many different configuration methods—such as by token count, duration, number of messages, and so on. Recording the configuration could make the attributes look bloated.

[eternalcuriouslearner]

Great point! But in fact, I think collecting this configuration is not that meaningful. It only indicates whether the compaction behavior is enabled. In practice, different models can use very different patterns to control compaction, and some may even compact by default. In such cases, recording the compaction signal itself may be more valuable.

Solid point. I did some research and found that OpenAI and Anthropic expose and enable compaction differently. Because of that, recording request-side configuration may be noisy and inconsistent across providers.

One concern is that, there may be many different configuration methods—such as by token count, duration, number of messages, and so on. Recording the configuration could make the attributes look bloated.

I agree. Capturing configuration seems likely to add attribute bloat without much practical value, especially since compaction may be controlled differently across providers and models. Recording whether compaction actually occurred feels like the more meaningful and stable runtime signal.

It's better to record server side compaction with a boolean attribute.

CC: @Cirilla-zmh