Add transform function to dreamcode

[Nils-Schiwek]

What are you trying to accomplish?

I wanted to show a common use-case for the copilot-extension-sdk, as it might be used as a transformation function between LLMs. In this use-case a node service (or azure function) is serving an endpoint for copilot chat. This endpoint uses the message content from copilot to call an external streaming endpoint (for example an enterprise RAG LLM). The returned server-sent event stream is then transformed from the specific enterprise implementation to the copilot events by an intermediate function.

What approach did you choose and why?

There is currently no function in preview-sdk, which parses the sse-items. But the use of sse is a de-facto industry standard for llms by now. In my point of view it would be a good idea to streamline this into the sdk as well.

What should reviewers focus on?

Please focus on the readable-stream flow and chunk parser functions. These are the parts, that I would expect to be integrated into the sdk.

If you have any questions or further suggestions, feel free to contact me.