Skip to content

Fix Docker deployment #30

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
fabnemEPFL merged 21 commits into from
May 12, 2026
Merged

Fix Docker deployment #30

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
5ab4b24
Fix Docker deployment: pin SGLang, add .dockerignore, add entrypoint
HarshaSatyavardhan Mar 24, 2026
c4d5d9d
Add --platform=linux/amd64, docker-compose, and README Docker section
HarshaSatyavardhan Mar 25, 2026
5b50722
Fix CI secrets leak for PRs, fix Docker entrypoint, and update README
HarshaSatyavardhan Mar 28, 2026
af09dcf
Trigger GitHub Actions
HarshaSatyavardhan Mar 28, 2026
c673dc9
Resolve CI conflicts, skip PR logins, and restore original image name
HarshaSatyavardhan Mar 28, 2026
b9e3665
using the latest and remove the amd platform tag
HarshaSatyavardhan Apr 1, 2026
ef8ff05
trigger CI on
HarshaSatyavardhan Apr 1, 2026
c3bf9cf
comment out branch trigger,
HarshaSatyavardhan Apr 1, 2026
3e685ed
add CPU-only image and service for API-based LLMs
HarshaSatyavardhan Apr 1, 2026
ec43f29
add CPU-only image
HarshaSatyavardhan Apr 1, 2026
45cd130
revert CPU image changes pending upstream guidance on optional deps
HarshaSatyavardhan Apr 1, 2026
3b249d1
clean up CPU artifacts, restore original workflow
HarshaSatyavardhan Apr 1, 2026
a7f404c
make sglang optional dep, add CPU image support
HarshaSatyavardhan Apr 2, 2026
50a401a
trigger CI on branch
HarshaSatyavardhan Apr 2, 2026
6023cb2
add CPU image matrix entries to CI
HarshaSatyavardhan Apr 2, 2026
7484eec
fix docker cpu
HarshaSatyavardhan Apr 2, 2026
3957f8e
resolve conflict with upstream docker workflow changes
HarshaSatyavardhan Apr 14, 2026
864a42a
fix CPU image and add mmirage-cpu compose service
HarshaSatyavardhan Apr 25, 2026
8e5ded7
remove inline comments in typing.override shim
HarshaSatyavardhan Apr 25, 2026
560d384
Address Docker deployment review feedback
HarshaSatyavardhan May 10, 2026
b18d843
Merge branch 'main' into fix/docker-deployment
fabnemEPFL May 12, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions .dockerignore
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
.git
__pycache__
*.pyc
*.pyo
*.egg-info
build/
dist/
.env
.venv
env/
venv/
tests/output/
tests/merged/
logs/
.mypy_cache/
.pytest_cache/
.ruff_cache/
16 changes: 16 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,22 @@ For testing and scripts that make use of the library, it is advised to create a
./scripts/generate_env.sh
```

## Docker

### Build

```bash
docker compose build
```
Comment on lines +37 to +39
Copy link
Copy Markdown

@JCHAVEROT JCHAVEROT May 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This means you build both the GPU and CPU images while only actually using one of the two in the end

On my computer the build takes >10min, we could make it more optimized


### Run

```bash
docker compose run --rm mmirage --config configs/your_config.yaml
```

The container requires an NVIDIA GPU. The `docker-compose.yml` handles GPU access automatically.

Copy link

Copilot AI Mar 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The README implies GPU access will work automatically via docker-compose.yml, but users still need host-side prerequisites (NVIDIA drivers + NVIDIA Container Toolkit / nvidia-container-runtime) for the container to see GPUs. Consider adding a short prerequisite note (and any minimum Docker/Compose versions) so docker compose run failures are easier to diagnose.

Suggested change
The container requires an NVIDIA GPU. The `docker-compose.yml` handles GPU access automatically.
The container requires an NVIDIA GPU. The `docker-compose.yml` is configured to request GPU access, but the host must have:
- NVIDIA GPU drivers installed
- NVIDIA Container Toolkit / `nvidia-container-runtime` configured for Docker
- A recent Docker Engine and Docker Compose version with GPU support enabled
Without these host-side prerequisites, `docker compose run` may fail to detect or use the GPU.

Copilot uses AI. Check for mistakes.
## Key features

- **Multimodal Support**: Process both text and images with vision-language models
Expand Down
12 changes: 12 additions & 0 deletions docker-compose.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
services:
mmirage:
build:
context: .
dockerfile: docker/Dockerfile
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: all
capabilities: [gpu]
10 changes: 8 additions & 2 deletions docker/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,11 @@
FROM docker.io/lmsysorg/sglang:latest
FROM --platform=linux/amd64 lmsysorg/sglang:v0.5.8.post1-runtime
Copy link
Copy Markdown
Collaborator

@fabnemEPFL fabnemEPFL Mar 30, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why only AMD?

Copy link
Copy Markdown
Collaborator

@fabnemEPFL fabnemEPFL Mar 30, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and why not keep the latest image?

Copy link
Copy Markdown
Contributor Author

@HarshaSatyavardhan HarshaSatyavardhan Apr 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Tested on a local AMD system and forgot to remove the platform flag before pushing, my
    bad 😅 removed it now.
  • Pinned the version for reproducibility since latest keeps changing. Switched back to
    latest now.


ENV NVIDIA_VISIBLE_DEVICES=all
ENV NVIDIA_DRIVER_CAPABILITIES=compute,utility
Comment on lines +1 to +4
Copy link
Copy Markdown
Contributor Author

@HarshaSatyavardhan HarshaSatyavardhan May 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated the description


COPY . /workspace/MMIRAGE
WORKDIR /workspace/MMIRAGE
RUN pip install --no-cache-dir -e .

RUN pip install --no-cache-dir .

ENTRYPOINT ["python3", "-m", "mmirage.shard_process"]
Copy link

Copilot AI Mar 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The container ENTRYPOINT runs mmirage.shard_process, but the current config loading relies on ProcessorRegistry being populated via importing processor modules (e.g., the llm processor registers itself at import time). shard_process.py/load_mmirage_config() don’t import mmirage.core.process (or the processor modules), so --config runs are likely to fail with Processor llm not registered. Consider ensuring processor registration happens before calling load_mmirage_config (e.g., import mmirage.core.process in the CLI/module or add explicit registration in config loading).

Suggested change
ENTRYPOINT ["python3", "-m", "mmirage.shard_process"]
ENTRYPOINT ["python3", "-c", "import mmirage.core.process, runpy; runpy.run_module('mmirage.shard_process', run_name='__main__')"]

Copilot uses AI. Check for mistakes.
Copy link
Copy Markdown

@JCHAVEROT JCHAVEROT May 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's wrong to use this entrypoint, this means you can only run the mmirage.shard_process and not MMIRAGE's CLI

Suggested change
ENTRYPOINT ["python3", "-m", "mmirage.shard_process"]
ENTRYPOINT ["mmirage"]

This way if you can directly call the CLI command run specifically designed to call mmirage.shard_process, and you can easily call the other CLI commands without workaround

Copy link
Copy Markdown

@JCHAVEROT JCHAVEROT May 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After discussing with Fabrice, to be coherent with the documentation we will drop the entrypoint so that we just prefix each command by mmirage as said everywhere

Loading