🚀 Feature

• Ability to expose assistants through an API that follows the Responses API standard and specs.

import litserve as ls

class SimpleLitAPI(ls.LitAPI):
    def setup(self, device):
        self.model = None

    def predict(self, prompt):
        # `prompt` is a list of dictionary containing role and content
        # example: [{'role': 'user', 'content': 'How can I help you today?'}]
        yield "This is a sample generated output"

if __name__ == "__main__":
    # Enable the OpenAISpec in LitServer
    api = SimpleLitAPI()
    server = ls.LitServer(api, spec=ls.ResponsesOpenAISpec())
    server.run(port=8000)

Motivation

OpenAI announced last week a new Responses API that contains the feature set of the Chat Completions API and replaces the Assistants API (the latter is now deprecated and will be sunseted mid-2026).

Azure already announced that they provide this API day one and considering OpenAI's position on the market, we can expect that it will become an industry standard as is the Chat Completions API today.

Additional context

• Official documentation for the Responses API
• Thread on OpenAI's developer forum
• Comparison between the Chat Completions API and Responses API
• A blog post from Simon Willison on the new API
• A thread on X by one of the creators of the API talking about its making