TODO.org_archive

Archived entries from file /Users/ovistoica/workspace/voice-fn/TODO.org

Add support for configuration change.

Usecase: We have an initial prompt and tools to use. We want to change it based on the custom parameters that are inputted throught the twilio websocket. Example: On the twilio websocket, we can give custom parameters like script-name, overrides like user name, etc.

We can use the config-change frame to do this. And every processor takes what it cares about from it. However, you add very specific functionality to the twilio-in transport. So, what you need to do is add a custom-params->config argument.

:transport-in {:proc transport/twilio-transport-in
               :args {:transport/in-ch in
                      :twilio/handle-event (fn [event]
                                             {:out {:llm/context ".."
                                                    :llm/registered-tools [...]}})}

add core.async.flow support

Research a way to add clj-kondo schema type hints for frames in the macro

(defframe my-cool-frame
  "This is a cool frame"
  {:type :frame.cool/hello
   :schema [:map
            [:messages LLMContextMessages]
            [:tools LLMTools]]})

Add tools calls support

Create buffered output transport that sends chunks of 20ms at a 10ms interval

Handle Start/Stop interruption frames in LLM and TTS and other assemblers

Add assembler that takes in interim transcripts based on VAD

Add VAD events from deepgram

Add schema validation with defaults

Change tool_call declaration to include the handler, to enable changing available tools on the fly

If the function :handler returns a channel, the tool-caller will block until a result is put on the channel, optionally with a timeout

:functions [{:type :function
                  :function
                  {:name "record_party_size"
                   :handler (fn [{:keys [size]}] ...)
                   :description "Record the number of people in the party"
                   :parameters
                   {:type :object
                    :properties
                    {:size {:type :integer
                            :minimum 1
                            :maximum 12}}
                    :required [:size]}
                   :transition-to :get-time}}]

After this, basically we can just emit a frame/llm-context and that will update the current context. However the scenario manager needs to

Fix end the call function