ASIMOV Llamacpp Module

ASIMOV Llamacpp module.

✨ Features

• To be determined!

🛠️ Prerequisites

• Rust 1.85+ (2024 edition) if building from source code
• You must have a running llama.cpp server with a loaded GGUF model

⬇️ Installation

Installation with ASIMOV CLI

asimov module install llamacpp -v

Installation from Source Code

cargo install asimov-llamacpp-module

👉 Examples

asimov-llamacpp-prompter

⚙ Configuration

Via ASIMOV CLI

asimov module config llamacpp

Via environment variables

export ASIMOV_LLAMACPP_API_ENDPOINT="http://127.0.0.1:8080"
export ASIMOV_LLAMACPP_MODEL="TinyLlama-1.1B-Chat-v1.0"

📚 Reference

Prompt

echo "Why is the sky blue?" | asimov-llamacpp-prompter

🧪 Simple GBNF example

1. Create a tiny grammar that only allows ok:

# only_ok.gbnf
root ::= "ok" "\n"

2. Run the prompter with the grammar:

echo "Say anything." | asimov-llamacpp-prompter -g only_ok.gbnf
# => ok

👨‍💻 Development

git clone https://github.com/asimov-modules/asimov-llamacpp-module.git

---