Fix Docker deployment by HarshaSatyavardhan · Pull Request #30 · EPFLiGHT/MMIRAGE

HarshaSatyavardhan · 2026-03-25T02:58:50Z

Fixes #22

Pin base image to lmsysorg/sglang:latest
Add --platform=linux/amd64 to fix arm64 CI build
Add NVIDIA_VISIBLE_DEVICES and NVIDIA_DRIVER_CAPABILITIES env vars for GPU visibility
Add ENTRYPOINT to make the container executable
Add docker-compose.yml with NVIDIA GPU support
Add Docker section to README

Copilot

Pull request overview

Updates the project’s Docker packaging to make deployments reproducible and runnable via docker compose, with explicit GPU-related defaults.

Changes:

Pin the Docker base image and force amd64 builds in the Dockerfile; add GPU env vars and an executable ENTRYPOINT.
Add a docker-compose.yml service definition requesting NVIDIA GPUs.
Add .dockerignore and document Docker build/run steps in the README.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

File	Description
docker/Dockerfile	Pins base image, sets NVIDIA env vars, installs package, and defines ENTRYPOINT.
docker-compose.yml	Adds a compose service intended to request NVIDIA GPU access.
README.md	Documents `docker compose build` and `docker compose run` usage.
.dockerignore	Reduces Docker build context size by ignoring caches/artifacts.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-03-27T18:15:35Z

+
+RUN pip install --no-cache-dir .
+
+ENTRYPOINT ["python3", "-m", "mmirage.shard_process"]


The container ENTRYPOINT runs mmirage.shard_process, but the current config loading relies on ProcessorRegistry being populated via importing processor modules (e.g., the llm processor registers itself at import time). shard_process.py/load_mmirage_config() don’t import mmirage.core.process (or the processor modules), so --config runs are likely to fail with Processor llm not registered. Consider ensuring processor registration happens before calling load_mmirage_config (e.g., import mmirage.core.process in the CLI/module or add explicit registration in config loading).

Suggested change

ENTRYPOINT ["python3", "-m", "mmirage.shard_process"]

ENTRYPOINT ["python3", "-c", "import mmirage.core.process, runpy; runpy.run_module('mmirage.shard_process', run_name='__main__')"]

Copilot · 2026-03-27T18:15:36Z

+The container requires an NVIDIA GPU. The `docker-compose.yml` handles GPU access automatically.
+


The README implies GPU access will work automatically via docker-compose.yml, but users still need host-side prerequisites (NVIDIA drivers + NVIDIA Container Toolkit / nvidia-container-runtime) for the container to see GPUs. Consider adding a short prerequisite note (and any minimum Docker/Compose versions) so docker compose run failures are easier to diagnose.

Suggested change

The container requires an NVIDIA GPU. The `docker-compose.yml` handles GPU access automatically.

The container requires an NVIDIA GPU. The `docker-compose.yml` is configured to request GPU access, but the host must have:

- NVIDIA GPU drivers installed

- NVIDIA Container Toolkit / `nvidia-container-runtime` configured for Docker

- A recent Docker Engine and Docker Compose version with GPU support enabled

Without these host-side prerequisites, `docker compose run` may fail to detect or use the GPU.

HarshaSatyavardhan · 2026-03-28T11:11:42Z

@fabnemEPFL The repetitive Error: Username and password required crashes in the CI were happening because GitHub Actions intentionally hides repository secrets when a Pull Request is submitted from a fork. Because the original workflow attempted to authenticate with Docker Hub on every single PR, it was crashing when those hidden secrets evaluated to empty strings.

To fix this, I updated the workflow to safely skip the login-action and push steps when github.event_name == 'pull_request'. This allows the CI to verify that the Docker image builds successfully without crashing on authentication.

I also applied the Copilot suggestions for the ENTRYPOINT in the Dockerfile and updated the GPU prerequisites in the README.md. I tested this workflow on my own fork and successfully built both the amd64 and arm64 images. I've resolved the merge conflicts and restored the original IMAGE_NAME, so this should be ready to review!"

fabnemEPFL · 2026-03-30T13:11:05Z

@@ -1,5 +1,11 @@
-FROM docker.io/lmsysorg/sglang:latest
+FROM --platform=linux/amd64 lmsysorg/sglang:v0.5.8.post1-runtime


why only AMD?

and why not keep the latest image?

Tested on a local AMD system and forgot to remove the platform flag before pushing, my
bad 😅 removed it now.

Pinned the version for reproducibility since latest keeps changing. Switched back to
latest now.

fabnemEPFL · 2026-03-30T13:13:30Z

+docker compose run --rm mmirage --config configs/your_config.yaml
+```
+
+The container requires an NVIDIA GPU. The `docker-compose.yml` is configured to request GPU access, but the host must have:


there should also be a CPU-only image (there will soon be support for API-based LLMs that do not require GPUs)

i have created the Dockerfile.cpu using python:3.11-slim but sglang is a hard dependency in the pyproject.toml. so its fails because it needs CUDA headers doesnt exist in CPU image.

one thing we can do is move sglang and realted dependencies to project.optional-dependencies in project.toml
or we can wrap the SGlang import in the llm_processor.py with try/except
how would you like me to proceed?

you can take inspiration from mmore's dependencies, with rules separated depending on CPU/GPU

fix/docker-deployment branch

restore image name

HarshaSatyavardhan · 2026-04-02T21:08:44Z

moved sglang and related packages to [project.optional-dependencies.gpu] in pyproject.toml
wrapped the import in llm_processor.py with try/except
updated both Dockerfiles (pip install .[gpu] for GPU, pip install . for CPU)

Copilot

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated 7 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

        """
        super().__init__(engine_args, **kwargs)
+        if not SGLANG_AVAILABLE:
+            raise RuntimeError("SGLang is not installed. Install with: pip install '.[gpu]'")


@@ -14,14 +14,43 @@ To install the library, you can clone it from GitHub and then use pip to install

 ```bash
 git clone git@github.com:EPFLiGHT/MMIRAGE.git
-pip install -e ./MMIRAGE
+pip install -e './MMIRAGE[gpu]'
 ```


+
+RUN pip install --no-cache-dir .[gpu]
+
+ENTRYPOINT ["python3", "-c", "import mmirage.core.process, runpy; runpy.run_module('mmirage.shard_process', run_name='__main__')"]


+
+RUN pip install --no-cache-dir .
+
+ENTRYPOINT ["python3", "-c", "import mmirage.core.process, runpy; runpy.run_module('mmirage.shard_process', run_name='__main__')"]


HarshaSatyavardhan · 2026-05-10T23:57:48Z

+FROM lmsysorg/sglang:latest
+
+ENV NVIDIA_VISIBLE_DEVICES=all
+ENV NVIDIA_DRIVER_CAPABILITIES=compute,utility


updated the description

            ${{ secrets.DOCKERHUB_USERNAME }}/${{ matrix.name }}:latest-${{ matrix.tag_base }}
            ${{ secrets.DOCKERHUB_USERNAME }}/${{ matrix.name }}:${{ github.sha }}-${{ matrix.tag_base }}


@@ -26,6 +29,14 @@ jobs:
            path: docker/Dockerfile
            tag_base: arm64
            name: mmirage-git


JCHAVEROT

Thanks @HarshaSatyavardhan for your contribution!

I will most likely address the comments I made in a subsequent PR, so no need to make any changes

JCHAVEROT · 2026-05-12T08:49:01Z

+          - platform: ubuntu-latest
+            path: docker/Dockerfile.cpu
+            tag_base: amd64
+            name: mmirage-git-cpu
          - platform: ubuntu-24.04-arm
-            path: docker/Dockerfile
+            path: docker/Dockerfile.cpu
            tag_base: arm64
-            name: mmirage-git
+            name: mmirage-git-cpu


It's a bit weird to not use the same base image depending on the target platform, aditionnallyubuntu-latest does support linux/arm64 as per the documentation

As a future improvement we could have a workflow that reuse ubuntu-latest with several target platforms

JCHAVEROT · 2026-05-12T09:00:25Z

+
+RUN pip install --no-cache-dir .[gpu]
+
+ENTRYPOINT ["python3", "-m", "mmirage.shard_process"]


it's wrong to use this entrypoint, this means you can only run the mmirage.shard_process and not MMIRAGE's CLI

Suggested change

ENTRYPOINT ["python3", "-m", "mmirage.shard_process"]

ENTRYPOINT ["mmirage"]

This way if you can directly call the CLI command run specifically designed to call mmirage.shard_process, and you can easily call the other CLI commands without workaround

After discussing with Fabrice, to be coherent with the documentation we will drop the entrypoint so that we just prefix each command by mmirage as said everywhere

JCHAVEROT · 2026-05-12T09:02:39Z

+```bash
+docker compose build
+```


This means you build both the GPU and CPU images while only actually using one of the two in the end

On my computer the build takes >10min, we could make it more optimized

JCHAVEROT · 2026-05-12T09:26:01Z

@@ -1,5 +1,11 @@
-FROM docker.io/lmsysorg/sglang:latest
+FROM lmsysorg/sglang:latest


After testing on RCP, lmsysorg/sglang:latest is based over CUDA 13 but dependency sgl_kernel requires a CUDA 12 lib, so we'll have to fix the base image to an older version under CUDA 12

JCHAVEROT · 2026-05-12T09:34:29Z


 [project.optional-dependencies]
+gpu = [
+  "sglang>=0.5.2",


Suggested change

"sglang>=0.5.2",

"sglang==0.5.10",

HarshaSatyavardhan added 2 commits March 24, 2026 23:35

Fix Docker deployment: pin SGLang, add .dockerignore, add entrypoint

5ab4b24

Add --platform=linux/amd64, docker-compose, and README Docker section

c4d5d9d

HarshaSatyavardhan had a problem deploying to docker March 27, 2026 18:11 — with GitHub Actions Error

HarshaSatyavardhan had a problem deploying to docker March 27, 2026 18:11 — with GitHub Actions Failure

fabnemEPFL requested a review from Copilot March 27, 2026 18:11

Copilot started reviewing on behalf of fabnemEPFL March 27, 2026 18:11 View session

Copilot AI reviewed Mar 27, 2026

View reviewed changes

HarshaSatyavardhan added 3 commits March 28, 2026 14:48

Fix CI secrets leak for PRs, fix Docker entrypoint, and update README

5b50722

Trigger GitHub Actions

af09dcf

Resolve CI conflicts, skip PR logins, and restore original image name

c673dc9

fabnemEPFL requested changes Mar 30, 2026

View reviewed changes

HarshaSatyavardhan added 11 commits April 1, 2026 21:32

using the latest and remove the amd platform tag

b9e3665

trigger CI on

ef8ff05

fix/docker-deployment branch

comment out branch trigger,

c3bf9cf

restore image name

add CPU-only image and service for API-based LLMs

3e685ed

add CPU-only image

ec43f29

revert CPU image changes pending upstream guidance on optional deps

45cd130

clean up CPU artifacts, restore original workflow

3b249d1

make sglang optional dep, add CPU image support

a7f404c

trigger CI on branch

50a401a

add CPU image matrix entries to CI

6023cb2

fix docker cpu

7484eec

fabnemEPFL self-requested a review April 14, 2026 11:40

HarshaSatyavardhan added 3 commits April 15, 2026 04:00

resolve conflict with upstream docker workflow changes

3957f8e

fix CPU image and add mmirage-cpu compose service

864a42a

remove inline comments in typing.override shim

8e5ded7

HarshaSatyavardhan mentioned this pull request Apr 26, 2026

feat: add fuzzy deduplication post-processing #37

Open

fabnemEPFL requested a review from Copilot April 29, 2026 16:30

Copilot started reviewing on behalf of fabnemEPFL April 29, 2026 16:31 View session

Copilot AI reviewed Apr 29, 2026

View reviewed changes

Address Docker deployment review feedback

560d384

HarshaSatyavardhan temporarily deployed to docker-ci May 11, 2026 08:57 — with GitHub Actions Inactive

Merge branch 'main' into fix/docker-deployment

b18d843

JCHAVEROT self-requested a review May 12, 2026 08:15

fabnemEPFL temporarily deployed to docker-ci May 12, 2026 08:45 — with GitHub Actions Inactive

JCHAVEROT approved these changes May 12, 2026

View reviewed changes

JCHAVEROT reviewed May 12, 2026

View reviewed changes

JCHAVEROT temporarily deployed to docker-ci May 12, 2026 12:26 — with GitHub Actions Inactive

JCHAVEROT mentioned this pull request May 12, 2026

Fix docker configurations #44

Merged

fabnemEPFL approved these changes May 12, 2026

View reviewed changes

fabnemEPFL merged commit 4668761 into EPFLiGHT:main May 12, 2026
6 checks passed


		RUN pip install --no-cache-dir .

		ENTRYPOINT ["python3", "-m", "mmirage.shard_process"]

	ENTRYPOINT ["python3", "-m", "mmirage.shard_process"]
	ENTRYPOINT ["python3", "-c", "import mmirage.core.process, runpy; runpy.run_module('mmirage.shard_process', run_name='__main__')"]

		The container requires an NVIDIA GPU. The `docker-compose.yml` handles GPU access automatically.

-The container requires an NVIDIA GPU. The `docker-compose.yml` handles GPU access automatically.
+The container requires an NVIDIA GPU. The `docker-compose.yml` is configured to request GPU access, but the host must have:
+- NVIDIA GPU drivers installed
+- NVIDIA Container Toolkit / `nvidia-container-runtime` configured for Docker
+- A recent Docker Engine and Docker Compose version with GPU support enabled
+Without these host-side prerequisites, `docker compose run` may fail to detect or use the GPU.

		@@ -1,5 +1,11 @@
		FROM docker.io/lmsysorg/sglang:latest
		FROM --platform=linux/amd64 lmsysorg/sglang:v0.5.8.post1-runtime


		RUN pip install --no-cache-dir .[gpu]

		ENTRYPOINT ["python3", "-c", "import mmirage.core.process, runpy; runpy.run_module('mmirage.shard_process', run_name='__main__')"]

		${{ secrets.DOCKERHUB_USERNAME }}/${{ matrix.name }}:latest-${{ matrix.tag_base }}
		${{ secrets.DOCKERHUB_USERNAME }}/${{ matrix.name }}:${{ github.sha }}-${{ matrix.tag_base }}

	ENTRYPOINT ["python3", "-m", "mmirage.shard_process"]
	ENTRYPOINT ["mmirage"]

		@@ -1,5 +1,11 @@
		FROM docker.io/lmsysorg/sglang:latest
		FROM lmsysorg/sglang:latest

Conversation

HarshaSatyavardhan commented Mar 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Mar 27, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 27, 2026

Choose a reason for hiding this comment

Uh oh!

HarshaSatyavardhan commented Mar 28, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

HarshaSatyavardhan commented Apr 2, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Choose a reason for hiding this comment

Uh oh!

JCHAVEROT left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

JCHAVEROT May 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

HarshaSatyavardhan commented Mar 25, 2026 •

edited

Loading

JCHAVEROT left a comment •

edited

Loading

JCHAVEROT May 12, 2026 •

edited

Loading