Skip to content
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
5ab4b24
Fix Docker deployment: pin SGLang, add .dockerignore, add entrypoint
HarshaSatyavardhan Mar 24, 2026
c4d5d9d
Add --platform=linux/amd64, docker-compose, and README Docker section
HarshaSatyavardhan Mar 25, 2026
5b50722
Fix CI secrets leak for PRs, fix Docker entrypoint, and update README
HarshaSatyavardhan Mar 28, 2026
af09dcf
Trigger GitHub Actions
HarshaSatyavardhan Mar 28, 2026
c673dc9
Resolve CI conflicts, skip PR logins, and restore original image name
HarshaSatyavardhan Mar 28, 2026
b9e3665
using the latest and remove the amd platform tag
HarshaSatyavardhan Apr 1, 2026
ef8ff05
trigger CI on
HarshaSatyavardhan Apr 1, 2026
c3bf9cf
comment out branch trigger,
HarshaSatyavardhan Apr 1, 2026
3e685ed
add CPU-only image and service for API-based LLMs
HarshaSatyavardhan Apr 1, 2026
ec43f29
add CPU-only image
HarshaSatyavardhan Apr 1, 2026
45cd130
revert CPU image changes pending upstream guidance on optional deps
HarshaSatyavardhan Apr 1, 2026
3b249d1
clean up CPU artifacts, restore original workflow
HarshaSatyavardhan Apr 1, 2026
a7f404c
make sglang optional dep, add CPU image support
HarshaSatyavardhan Apr 2, 2026
50a401a
trigger CI on branch
HarshaSatyavardhan Apr 2, 2026
6023cb2
add CPU image matrix entries to CI
HarshaSatyavardhan Apr 2, 2026
7484eec
fix docker cpu
HarshaSatyavardhan Apr 2, 2026
3957f8e
resolve conflict with upstream docker workflow changes
HarshaSatyavardhan Apr 14, 2026
864a42a
fix CPU image and add mmirage-cpu compose service
HarshaSatyavardhan Apr 25, 2026
8e5ded7
remove inline comments in typing.override shim
HarshaSatyavardhan Apr 25, 2026
560d384
Address Docker deployment review feedback
HarshaSatyavardhan May 10, 2026
b18d843
Merge branch 'main' into fix/docker-deployment
fabnemEPFL May 12, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
.git
__pycache__
*.pyc
*.pyo
*.egg-info
build/
dist/
.env
.venv
env/
venv/
tests/output/
tests/merged/
logs/
.mypy_cache/
.pytest_cache/
.ruff_cache/
9 changes: 6 additions & 3 deletions .github/workflows/docker.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,14 @@ on:
push:
branches:
- main
# - fix/docker-deployment
pull_request:
branches:
- main
workflow_dispatch:

env:
IMAGE_NAME: michelducartier24/mirage
IMAGE_NAME: fabnemepfl/mirage
REGISTRY: docker.io

jobs:
Expand Down Expand Up @@ -62,6 +64,7 @@ jobs:
uses: actions/checkout@v4

- name: Log in to DockerHub
if: github.event_name != 'pull_request'
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKER_USERNAME }}
Expand All @@ -75,7 +78,7 @@ jobs:
with:
context: .
file: ${{ matrix.path }}
push: true
push: ${{ github.event_name != 'pull_request' }}
tags: |
${{ secrets.DOCKER_USERNAME }}/${{ matrix.name }}:latest-${{ matrix.tag_base }}
${{ secrets.DOCKER_USERNAME }}/${{ matrix.name }}:${{ github.sha }}-${{ matrix.tag_base }}
${{ secrets.DOCKER_USERNAME }}/${{ matrix.name }}:${{ github.sha }}-${{ matrix.tag_base }}
21 changes: 21 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,27 @@ For testing and scripts that make use of the library, it is advised to create a
./scripts/generate_env.sh
```

## Docker

### Build

```bash
docker compose build
```
Comment on lines +37 to +39
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This means you build both the GPU and CPU images while only actually using one of the two in the end

On my computer the build takes >10min, we could make it more optimized


### Run

```bash
docker compose run --rm mmirage --config configs/your_config.yaml
```

The container requires an NVIDIA GPU. The `docker-compose.yml` is configured to request GPU access, but the host must have:
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there should also be a CPU-only image (there will soon be support for API-based LLMs that do not require GPUs)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i have created the Dockerfile.cpu using python:3.11-slim but sglang is a hard dependency in the pyproject.toml. so its fails because it needs CUDA headers doesnt exist in CPU image.

one thing we can do is move sglang and realted dependencies to project.optional-dependencies in project.toml
or we can wrap the SGlang import in the llm_processor.py with try/except
how would you like me to proceed?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you can take inspiration from mmore's dependencies, with rules separated depending on CPU/GPU

- NVIDIA GPU drivers installed
- NVIDIA Container Toolkit / `nvidia-container-runtime` configured for Docker
- A recent Docker Engine and Docker Compose version with GPU support enabled

Without these host-side prerequisites, `docker compose run` may fail to detect or use the GPU.

## Key features

- **Multimodal Support**: Process both text and images with vision-language models
Expand Down
12 changes: 12 additions & 0 deletions docker-compose.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
services:
mmirage:
build:
context: .
dockerfile: docker/Dockerfile
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: all
capabilities: [gpu]
10 changes: 8 additions & 2 deletions docker/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,5 +1,11 @@
FROM docker.io/lmsysorg/sglang:latest
FROM --platform=linux/amd64 lmsysorg/sglang:v0.5.8.post1-runtime
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why only AMD?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and why not keep the latest image?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Tested on a local AMD system and forgot to remove the platform flag before pushing, my
    bad 😅 removed it now.
  • Pinned the version for reproducibility since latest keeps changing. Switched back to
    latest now.


ENV NVIDIA_VISIBLE_DEVICES=all
ENV NVIDIA_DRIVER_CAPABILITIES=compute,utility
Comment on lines +1 to +4
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated the description


COPY . /workspace/MMIRAGE
WORKDIR /workspace/MMIRAGE
RUN pip install --no-cache-dir -e .

RUN pip install --no-cache-dir .

ENTRYPOINT ["python3", "-c", "import mmirage.core.process, runpy; runpy.run_module('mmirage.shard_process', run_name='__main__')"]