Message-free supervisor: process split — agent ⇄ supervisor over gRPC, programs included #1849
Workflow file for this run
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| --- | |
| name: "py.test and linting" | |
| on: | |
| push: | |
| branches: [main, dev] | |
| pull_request: | |
| # The branches below must be a subset of the branches above | |
| branches: [main, dev] | |
| jobs: | |
| tests-python: | |
| name: "Test Python code" | |
| runs-on: ubuntu-24.04 | |
| services: | |
| # Run vm connector for the execution tests | |
| vm-connector: | |
| image: alephim/vm-connector:alpha | |
| ports: | |
| - 4021:4021 | |
| steps: | |
| - uses: actions/checkout@v4 | |
| - name: Workaround github issue https://github.com/actions/runner-images/issues/7192 | |
| run: sudo echo RESET grub-efi/install_devices | sudo debconf-communicate grub-pc | |
| - name: Install required system packages for installing and running tests | |
| run: | | |
| sudo apt-get update | |
| sudo apt-get install -y python3 python3-pip python3-aiohttp python3-msgpack python3-aiodns python3-alembic python3-sqlalchemy python3-setproctitle redis python3-psutil sudo acl curl systemd-container squashfs-tools debootstrap python3-packaging python3-cpuinfo python3-nftables python3-jsonschema nftables libsystemd-dev cmake libdbus-1-dev libglib2.0-dev lshw python3-jwcrypto | |
| pip install --upgrade typing-extensions types-PyYAML 'redis>=4.2.0' | |
| - name: Install required Python packages | |
| run: | | |
| # virtualenv 21 brings a breaking change that's not supported by hatch 1.16 | |
| python3 -m pip install --upgrade --ignore-installed hatch hatch-vcs coverage "virtualenv<21" | |
| # Also install for root: subsequent steps (proto check, unit tests) use | |
| # `sudo hatch ...`, and sudo's PATH doesn't see the runner's pip-installed hatch. | |
| sudo python3 -m pip install --upgrade --ignore-installed hatch hatch-vcs coverage "virtualenv<21" | |
| - name: Test style wth ruff, black and isort | |
| run: | | |
| hatch run linting:style | |
| - name: Test typing with Mypy | |
| run: | | |
| hatch run linting:typing | |
| - name: Download and build required files for running tests. Copied from packaging/Makefile. | |
| run: | | |
| sudo useradd jailman | |
| sudo mkdir --parents /opt/firecracker/ | |
| sudo curl -fsSL -o "/opt/firecracker/vmlinux.bin" "https://ipfs.aleph.cloud/ipfs/QmQvQnZg9rB2fhmuminFSYgtGGzEhfbiTMobppUXVfGBmD" | |
| rm -fr /tmp/firecracker-release | |
| mkdir --parents /tmp/firecracker-release /opt/firecracker | |
| curl -fsSL https://github.com/firecracker-microvm/firecracker/releases/download/v1.15.1/firecracker-v1.15.1-x86_64.tgz | tar -xz --no-same-owner --directory /tmp/firecracker-release | |
| # Copy binaries: | |
| cp /tmp/firecracker-release/release-v*/firecracker-v*[!.debug] /opt/firecracker/firecracker | |
| cp /tmp/firecracker-release/release-v*/jailer-v*[!.debug] /opt/firecracker/jailer | |
| chmod +x /opt/firecracker/firecracker | |
| chmod +x /opt/firecracker/jailer | |
| # this produces a 33 MB log | |
| # find /opt | |
| - name: "Build custom runtimes" | |
| run: | | |
| sudo apt update | |
| sudo apt install -y debootstrap ndppd acl cloud-image-utils qemu-utils qemu-system-x86 | |
| cd runtimes/aleph-debian-12-python && sudo ./create_disk_image.sh && cd ../.. | |
| cd runtimes/instance-rootfs && sudo ./create-ubuntu-22-04-qemu-disk.sh && cd ../.. | |
| cd runtimes/instance-rootfs && sudo ./create-debian-12-disk.sh && cd ../.. | |
| - name: "Build example volume" | |
| run: | | |
| cd examples/volumes && bash build_squashfs.sh | |
| - name: Verify proto bindings are up to date | |
| # Run with sudo so hatch creates the testing env as root — matching the | |
| # "Run unit tests" step below (which also uses sudo). Without this, the | |
| # non-sudo invocation here creates the env under /home/runner/... and the | |
| # sudo invocation later creates a *separate* root env that races to | |
| # install deps fast enough, causing import failures (e.g. redis). | |
| run: sudo hatch run testing:bash scripts/check_proto_clean.sh | |
| # Unit tests create and delete network interfaces, and therefore require to run as root | |
| - name: Run unit tests | |
| env: | |
| # Override the Aleph API endpoint without touching code. Defaults to api3.aleph.im | |
| # so CI does not depend on the flakier official.aleph.cloud. Set repo/org GH Actions | |
| # variable ALEPH_VM_API_SERVER to point at another CCN. | |
| ALEPH_VM_API_SERVER: ${{ vars.ALEPH_VM_API_SERVER || 'https://api3.aleph.im' }} | |
| run: | | |
| sudo python3 -m pip install --upgrade --ignore-installed hatch hatch-vcs coverage "virtualenv<21" | |
| sudo --preserve-env=ALEPH_VM_API_SERVER hatch run testing:cov | |
| - name: Output modules used and their version | |
| if: ${{ !cancelled() }} | |
| run: | | |
| # re-install hatch in case previous job failed and hatch didn't get installed | |
| sudo python3 -m pip install --upgrade --ignore-installed hatch hatch-vcs coverage "virtualenv<21" | |
| sudo hatch -e testing run pip freeze | |
| - name: Upload coverage reports to Codecov | |
| uses: codecov/codecov-action@v4.0.1 | |
| with: | |
| token: ${{ secrets.CODECOV_TOKEN }} | |
| slug: aleph-im/aleph-vm | |
| code-quality-shell: | |
| runs-on: ubuntu-22.04 | |
| steps: | |
| - uses: actions/checkout@v4 | |
| - name: Workaround github issue https://github.com/actions/runner-images/issues/7192 | |
| run: sudo echo RESET grub-efi/install_devices | sudo debconf-communicate grub-pc | |
| - name: Install required system packages only for Ubuntu Linux | |
| run: | | |
| sudo apt-get update | |
| sudo apt-get install -y shellcheck | |
| - name: Run Shellcheck on all shell scripts | |
| run: |- | |
| find ./ -type f -name "*.sh" -exec shellcheck {} \; |