Open
Description
Description:
I kindly request the implementation of streaming support for model invocations in the Modus SDK. This feature is crucial for real-time applications that require incremental responses.
Expected Changes:
- Introduce a
stream: true
option in theChatModelInput
configuration. - Provide a mechanism to handle tokens as they are received.
Benefits:
- Reduces latency for real-time applications.
- Enhances user experience by providing immediate feedback.
Thank you for considering this request.
Activity