[FEATURE]: SDK support to transform and filter A2UI content for reading purposes

[jacobsimionato]

Some A2UI-based interaction with an LLM might be like:

1. Send query A
2. Generate UI response in surface A
3. Send query B
4. Generate UI response in surface B, using a template which is resolve to a complex layout that includes an embedded MCP app with a large amount of HTML content in it.
5. Send query C
6. Update surface A, to answer the question in query C
7. Send query D

At this point, when the LLM is understanding query D, it sees the history of the conversations somehow. By default, agent frameworks put the A2UI responses in a linear Turn/Part/Text|Data structure, and so the agent sees all 7 steps above, either as the LLM responded (e.g. pre template expansion) or as they were sent down the wire (post template expansion).

But, there are many ways that we could transform the message history before the LLM sees it, to conserve tokens and reduce confusion.

Possible transformations

• Convert message history to inference format DSL e.g. via decompile method on inference format
• Show the LLM the templated output and templates, rather than longer expanded output
• Strip component fields that are low value to being read, e.g. the HTML content in MCP apps. This came up in feat: align MCP app v0.9 bridge, specs, pong improvements, and data d… #1945 e.g. https://github.com/a2ui-project/a2ui/blob/main/samples/community/client/angular/projects/mcp_calculator/src/a2ui-catalog/mcp-apps-component_spec.md . Possibly, Component APIs could be annotated with some tag like @presentation_only so they can be hidden from reads. @sugoi-yuzuru who raised this issue.
• Collapsing write/update messages into a single surface snapshot, which just shows the LLM the latest state snapshot of the surface in the location in the conversation where the surface will be displayed, rather than each incremental update. This makes the LLM see the conversation in a similar way to how the user sees it. E.g. the LLM would see the latest state of surface A at turn 2, with the change from turn 6 applied. In turn 6, the LLM might see some brief generic message like "surface A updated".
• Collapsing content to a more token-efficient read-optimised format. E.g. if surfaces don't need to be updated, it may be more efficient to represent them in a hierarchical format with no component IDs shown, rather than a flat list.

Developer workflow

1. Identify catalog and inference format based on client capabilities
2. Generate prompt
3. Load message history from database
4. Convert message history for read purposes

• Apply any snapshotting, field erasure etc that can be achieved in a format-agnostic way
• Convert all A2UI content to the same DSL that we are trying to output, to avoid confusing the model.

5. Send prompt and context to LLM
6. Parse response and convert to A2UI format
7. Resolve templates
8. Store A2UI json content in message history database for future turns. A2UI SDKs may need to be involved in this step eventually, to control whether pre or post-template content is stored, and include references to templates so the data can be reinflated later possibly.
9. Send content to client

So we may need to do some work in step 4 and 8 that we haven't talked about much before.

[jacobsimionato]

@nan-yu and @gspencergoog I think we may need to support this flow in our SDKs to make Express-based inferences work well in multi-turn conversations.