Skip to content

VM_SETUP.md

Latest commit

 

History

History
214 lines (146 loc) · 4.74 KB

VM_SETUP.md

File metadata and controls

214 lines (146 loc) · 4.74 KB

Orion VM Setup

This guide covers the VMware Ubuntu guest required for Orion's GUI experiments. Use it after the host Python environment is installed.

Orion runs the agent on the host, but drives a real Ubuntu desktop in VMware. The host talks to the guest through vmrun and swerex-remote.

1. Install VMware

Install the VMware product for your host:

  • macOS, Apple Silicon: VMware Fusion
  • Linux or Windows: VMware Workstation Pro

After installation, verify that vmrun is available:

which vmrun
vmrun list

On macOS, vmrun is usually located at:

/Applications/VMware Fusion.app/Contents/Public/vmrun

If which vmrun does not find it, add this to your shell profile:

export PATH="/Applications/VMware Fusion.app/Contents/Public:$PATH"

Then restart your shell and check again.

2. Get an Ubuntu VM

Use an Ubuntu VM compatible with your machine architecture.

For Apple Silicon, use the pre-installed arm64 Orion VM:

https://huggingface.co/datasets/orion-agent/orion-vm-arm

For x86 hosts, use the Orion-provided VM artifact for that architecture when it is available. The older OSWorld base images referenced in setup.md are useful for rebuilding a guest manually, but the Orion VM is the recommended reproduction target.

After downloading and extracting the VM, locate the .vmx file, for example:

/path/to/Ubuntu/Ubuntu.vmx

This path becomes HOST_VM_FILE in .env.

3. Guest Requirements

The Ubuntu guest should have:

  • Login user: user
  • Login password: password
  • Working directory: /home/user/Documents/WorkingDir
  • swerex-remote available, usually via pip install swe-rex
  • A conda env named python39
  • CellProfiler installed in python39
  • QuPath installed for pathology runs
  • napari runtime installed for image-viewer workflows
  • VMware Tools or open-vm-tools running

Inside the guest, basic checks should pass:

whoami
cd /home/user/Documents/WorkingDir
swerex-remote -h
conda activate python39
cellprofiler --version

For QuPath and napari:

./path/to/QuPath
python -c "import napari; print(napari.__version__)"

The Orion runner copies the host-side napari helper server from tools/napari_viewer_tools/napari_server/ into the guest automatically during task reset, so the guest only needs the runtime dependencies.

4. Configure Orion

In the repo root, edit .env:

HOST_VM_FILE=/absolute/path/to/Ubuntu.vmx
VM_USER=user
VM_PASSWORD=password
VM_WORKING_DIR=/home/user/Documents/WorkingDir
ENV_NAME=python39

Keep HOST_VM_FILE as an absolute path. Relative paths are easy to break because the launchers and runtime change directories.

5. Verify VMware From the Host

Start the VM from the host:

vmrun start "$HOST_VM_FILE"

Confirm VMware can see it:

vmrun list

Get the guest IP:

vmrun getGuestIPAddress "$HOST_VM_FILE" -wait

If this fails, check that VMware Tools or open-vm-tools is installed and running inside Ubuntu.

6. Run the Communication Smoke Test

From the repo root:

conda activate orion
python communication_test/test_vmware_deployment.py

When prompted, paste the absolute path to Ubuntu.vmx.

This test verifies:

  • VMware can start the VM.
  • The guest IP can be discovered.
  • swerex-remote can start inside the guest.
  • Host-to-guest command execution works.
  • Session state persists.
  • CellProfiler is available in the configured guest conda env.

Do not start full experiments until this test passes.

7. Common Failures

vmrun: command not found

Install VMware Fusion or Workstation and add vmrun to PATH.

VMware Tools are not running in the guest

Install or start open-vm-tools in Ubuntu:

sudo apt update
sudo apt install -y open-vm-tools
sudo systemctl start open-vm-tools

Waiting for swerex service... hangs

SSH or open a terminal in the guest and run:

cd /home/user/Documents/WorkingDir
swerex-remote --auth-token user:password --port 4000

Fix any import, PATH, or permission errors shown there.

CellProfiler step prints nothing

Check that the guest has the expected conda env and package:

conda activate python39
cellprofiler --version

Wrong or unreachable guest IP

Run this inside the guest:

hostname -I

The host needs a routable VMware NAT or host-only address, not 127.0.0.1.

8. Next Step After VM Setup

Once the communication smoke test passes, download the dataset for the experiment you want and run the corresponding launcher:

bash scripts/run_orion_microvqa.sh
bash scripts/run_orion_labbench.sh
bash scripts/run_orion_cellprofiler.sh
bash scripts/run_orion_qupath.sh
bash scripts/run_orion_jump_discovery.sh

Use .env to set paths, model keys, output directories, and task options.