docs: update (#56)

Author: maciejmajek

cloud_config.toml

@@ -3,8 +3,12 @@
 #
 #  Here the chat/vision endpoints are remote (backend = "openai") and therefore
 #  NOT served locally — `pixi run inference` skips backend = "openai". Only the
-#  RAG embedding/reranker run in-cluster (separate llama.cpp containers), so they
-#  carry an explicit base_url pointing at those services.
+#  RAG embedding/reranker still run locally on llama.cpp, exactly as in
+#  config.toml.
+#
+#  To use it: everything (agents, `pixi run inference`, download-models) reads
+#  config.toml, so swap this file in and provide the API key:
+#      cp cloud_config.toml config.toml && export OPENAI_API_KEY=sk-...
 #
 #  Because cloud agents use several distinct remote models, endpoints are split
 #  per model (chat_mini / vlm_mini / vlm_nano) and agents reference the right one.
@@ -30,25 +34,27 @@ backend  = "openai"
 model    = "gpt-5-nano"
 base_url = "https://api.openai.com/v1"
 
-# RAG models served by dedicated llama.cpp containers (see docker/compose.yaml).
-# Start the EMBEDDING model with:
-#   llama-server -m Qwen3-Embedding-0.6b_Q8_0.gguf --embedding --pooling last \
-#       -c 4096 -b 2048 -ub 2048 --port 8082 --host 0.0.0.0
-# Start the RERANKER model with:
-#   llama-server -m Qwen3-Reranker-0.6B_BF16.gguf --embedding --pooling rank \
-#       -c 4096 -b 2048 -ub 2048 --port 8083 --host 0.0.0.0
+# RAG models are the only endpoints still served locally (backend = "gpu"):
+# `pixi run inference` launches these two llama.cpp servers and skips the
+# remote backend = "openai" endpoints above. Same specs as config.toml.
 [endpoints.embeddings]
-type     = "embedding"
-backend  = "gpu"
-model    = "qwen3-embedding:0.6b"
-base_url = "http://safety-embeddings:8082/v1"
+type       = "embedding"
+backend    = "gpu"
+model      = "qwen3-embedding:0.6b"
+host       = "localhost"
+port       = 8082
+model_path = "${DEMO_ROOT}/models/Qwen3-Embedding-0.6B-Q8_0.gguf"
+model_url  = "https://huggingface.co/Qwen/Qwen3-Embedding-0.6B-GGUF/resolve/main/Qwen3-Embedding-0.6B-Q8_0.gguf?download=true"
 
 # NOTE: The reranker and the embedding model are two separate models!
 [endpoints.reranker]
-type     = "reranker"
-backend  = "gpu"
-model    = "qwen3-reranker:0.6b"
-base_url = "http://safety-reranker:8083/v1/reranking"
+type       = "reranker"
+backend    = "gpu"
+model      = "qwen3-reranker:0.6b"
+host       = "localhost"
+port       = 8083
+model_path = "${DEMO_ROOT}/models/qwen3-reranker-0.6b-q8_0.gguf"
+model_url  = "https://huggingface.co/ggml-org/Qwen3-Reranker-0.6B-Q8_0-GGUF/resolve/main/qwen3-reranker-0.6b-q8_0.gguf?download=true"
 
 # ─── Agents ─────────────────────────────────────────────────────────────────
 

docs/operating.md

@@ -56,6 +56,15 @@ The mission is handed to the orchestrator, which breaks it into steps and drives
 the robot. You do not need to wait for one mission to fully finish before queueing
 another; the orchestrator keeps a task queue.
 
+### Custom tasks
+
+The predefined buttons publish to the `/user_tasks` topic, which you can also use
+directly for free-form tasks:
+
+```bash
+ros2 topic pub --once /user_tasks std_msgs/msg/String "data: 'Do Housekeeping of rack J01'"
+```
+
 ## Monitoring progress
 
 While a mission runs, the **Mission** tab shows:

docs/quickstart.md

@@ -8,7 +8,7 @@ Run the full **Mobile Manipulator Demo** — simulation, ROS 2 stack, local infe
 - The host's **amdxdna** driver loaded: check with `ls /dev/accel/accel0`.
 - [Docker Engine](https://docs.docker.com/engine/install/) + [ROCm Docker prerequisites](https://rocm.docs.amd.com/projects/install-on-linux/en/latest/how-to/docker.html#prerequisites).
 - A running X server (the O3DE simulation and HMI open windows on your display).
-- ~60 GB free disk for the image, plus room under `models/` for the weights.
+- **Disk & network**: ~60 GB free for the image (the prebuilt image is a ~18 GB compressed pull), plus ~20 GB under `models/` for the weights. On a slow connection the first pull + model download takes a while.
 
 ## 1. Clone the repository
 
@@ -114,6 +114,22 @@ Before running a GPU-only image, edit `docker/compose.yaml` to:
 - remove the `/dev/accel/accel0` device and the `memlock` ulimit (NPU-only), and
 - point `config.toml`'s `vlm_safety` endpoint at a `gpu` backend — otherwise `pixi run inference` aborts on the missing NPU engine.
 
+## Cloud models
+
+To run the agents on OpenAI-hosted models instead of local inference, swap in the
+cloud config before starting (the compose bind-mounts `config.toml` and passes
+`OPENAI_API_KEY` into the container):
+
+```bash
+cp cloud_config.toml config.toml
+export OPENAI_API_KEY=sk-...
+```
+
+Only the RAG embedding + reranker are still served locally, so step 3 downloads
+just those two GGUFs. See
+[Using cloud models](setup_single_machine.md#using-cloud-models-instead-low-vram-machines)
+for details, including the required OpenAI model access.
+
 ## Troubleshooting
 
 - **O3DE doesn't start / no windows appear**: ensure you ran `xhost +local:root` and that `DISPLAY` is set (`echo $DISPLAY`). Verify your AMD GPU drivers + ROCm container prerequisites are installed.

docs/setup.md

@@ -22,7 +22,7 @@ Run HMI, agents and simulation on one computer
 
 > [!IMPORTANT]
 > This setup targets systems with approximately 48GB of available VRAM.
-> If your system does not meet this requirement, you can configure the agents to use cloud-based models instead via `cloud_config.toml`.
+> If your system does not meet this requirement, you can configure the agents to use cloud-based models instead via `cloud_config.toml` — see [Using cloud models](./setup_single_machine.md#using-cloud-models-instead-low-vram-machines).
 
 [Single-machine setup](./setup_single_machine.md)
 

docs/setup_single_machine.md

@@ -113,6 +113,28 @@ The NPU `vlm_safety` model (`gemma3:4b`) has no GGUF; `flm pull` downloads it fo
 
 ---
 
+### Using cloud models instead (low-VRAM machines)
+
+If the machine cannot serve the chat/vision models locally, switch the agents to
+OpenAI-hosted models via `cloud_config.toml`:
+
+```shell
+cp cloud_config.toml config.toml
+export OPENAI_API_KEY=sk-...
+```
+
+Your OpenAI account must have access to the models it names (`gpt-5-mini`,
+`gpt-5-nano`) — edit the `[endpoints.*]` `model` fields to use different ones.
+
+Everything reads `config.toml`, so the usual commands work unchanged:
+`download-models` now fetches only the two RAG GGUFs (embedding + reranker, the
+only endpoints still served locally) and `pixi run inference` launches just
+those two llama.cpp servers, skipping the remote `backend = "openai"` endpoints.
+The NPU is not used in this configuration, so the GPU-only `single-pc-gpu`
+environment suffices.
+
+---
+
 ## Verify your installation
 
 Once the build is done and the weights are downloaded, run one command to exercise