First of all thank you for your great tool! It made integrating llm into a game infinitely simpler than I anticipated before stumbling upon nobodywho
I propose this feature because of few issues what are very annoying to work around them and even then solutions leaves footguns in your project:
1) Triggering redraw from a tool call
Nobodywho allows to integrate llm model into your game and quite reasonable scenario is running npc dialog with it. And when llm controls npc you want it's tools be able to interact with the world.
Lets take "door keeper" scenario as an example:
There is a door keeper in your game and player can ask them to open the door. They can do it with a tool call:
func _door_open_tool() -> String:
if door.is_openable():
door.texture = opened_door_texture
return "door opened"
return "door can't be opened"This will throw an exception because you can't queue_redraw() from background thread. So you have to call door.texture = opened_door_texture deferred if you want to return conditional string from tool. Moreover in normal usecase tool handle is in npc script and door logic is in door script, so you just do var result: String = door.open() from tool function and then you need to design door.open() with llm tool call in mind, separating anything what would queue redraw to be called defered.
2) Awaiting signal from tool call
Sometimes you want llm powered npc to prompt player to do or "give" something to them with tool call:
signal item_given(item: ItemBase)
func _give_something_tool() -> String:
GameUi.show_item_prompt()
var item = await item_given
return "Player gave you %s" % item.item_nameIn that case your best choice is to create this monstrosity:
var tool_return_sema = Semaphore.new()
var tool_return_value: Variant
func _tool_wait() -> Variant:
tool_return_sema.wait()
return tool_return_value
func tool_return(answer: Variant):
tool_return_value = answer
tool_return_sema.post()
func _give_something_tool() -> String:
GameUi.show_item_prompt.call_deferred()
var item: ItemBase = await _tool_wait()
return "Player gave you %s" % item.item_nameBut it breaks type safety unless you're creating workaround for every answer type you can get.
So I propose moving execution of a tool call callable to main thread. Perhaps workaround from second example can become solution here:
# didn't coded enough in rust to give example in that language
var tool_return_sema = Semaphore.new()
var tool_return_string: String
func execute_tool(callable: Callable) -> String:
_tool_wrapper.call_deferred()
tool_return_sema.wait()
return tool_return_string
func _tool_wrapper(callable: Callable) -> void:
tool_return_string = callable.call()
tool_return_sema.post()Since calling tool halt's llm anyways there should be no issue with waiting for semaphore and no breaking type safety either since tool call can only return string
If backwards compatibility is concern here, calling from main thread can be added as flag on tool addition - func add_tool(callable: Callable, description: String, call_from_main_thread: bool = false) -> void
As an alternative solution there can be added a wrapper what would allow calling main thread exclusive functions and returning from them:
func _random_tool() -> String:
var result = wrap_main_exclusive(_redrawing_function)
return result
func _redrawing_function() -> String:
door.texture = opened_door_texture
return "abc"That would work too, but it is way less desirable
First of all thank you for your great tool! It made integrating llm into a game infinitely simpler than I anticipated before stumbling upon nobodywho
I propose this feature because of few issues what are very annoying to work around them and even then solutions leaves footguns in your project:
1) Triggering redraw from a tool call
Nobodywho allows to integrate llm model into your game and quite reasonable scenario is running npc dialog with it. And when llm controls npc you want it's tools be able to interact with the world.
Lets take "door keeper" scenario as an example:
There is a door keeper in your game and player can ask them to open the door. They can do it with a tool call:
This will throw an exception because you can't
queue_redraw()from background thread. So you have to calldoor.texture = opened_door_texturedeferred if you want to return conditional string from tool. Moreover in normal usecase tool handle is in npc script and door logic is in door script, so you just dovar result: String = door.open()from tool function and then you need to designdoor.open()with llm tool call in mind, separating anything what would queue redraw to be called defered.2) Awaiting signal from tool call
Sometimes you want llm powered npc to prompt player to do or "give" something to them with tool call:
In that case your best choice is to create this monstrosity:
But it breaks type safety unless you're creating workaround for every answer type you can get.
So I propose moving execution of a tool call callable to main thread. Perhaps workaround from second example can become solution here:
Since calling tool halt's llm anyways there should be no issue with waiting for semaphore and no breaking type safety either since tool call can only return string
If backwards compatibility is concern here, calling from main thread can be added as flag on tool addition -
func add_tool(callable: Callable, description: String, call_from_main_thread: bool = false) -> voidAs an alternative solution there can be added a wrapper what would allow calling main thread exclusive functions and returning from them:
That would work too, but it is way less desirable