More information and demo?

[mrtncode]

Hey @dotradepro , the project seems very nice!!
Can you provide more information about how it is working on a RPi4 4GB RAM? Does the local LLM also works good on this model? Could you provide demo figures or a video?

Thank you!

[dotradepro]

Hey! Thanks for reaching out and for the kind words about the project!

To clarify how it works on hardware like the Raspberry Pi 4:

Intent & Command Recognition: The core assistant handles intent and command recognition directly on the hardware without needing an LLM. This keeps the basic system incredibly fast, lightweight, and perfectly suited for a Raspberry Pi 4 (even the 4GB version).

The LLM Layer: For deeper, more complex conversations, the user can choose to connect an LLM themselves. It is fully flexible — you can either run a highly quantized smaller model locally via Ollama/llama.cpp, or connect to a cloud-based API.

If you run the LLM locally on an 8GB Pi 4, it handles small 3B or highly quantized models quite well. If you are on a 4GB model or want maximum speed, plugging in an external cloud LLM while letting the Pi handle the local commands is the perfect hybrid setup.

I don’t have a video demo ready at this exact moment, but I can share some resource usage screenshots or a quick clip soon!

[mrtncode]

Thank you!