[Question/feature request] Update LMI image to use latest vllm v0.7.2

[n0thing233]

Hi,

We need an upgrade to the LMI image to incorporate the latest vLLM v0.7.2 so we can deploy a Qwen2.5-VL model. Is there any ongoing effort to make this happen?

I also have a general question: Are the build process and Dockerfiles for these images open-sourced? If so, where can I find them, and how can I contribute?

If they are not open-sourced, what is the recommended way to build on top of the existing LMI image to use the latest vLLM on my end?

I’ve tried upgrading vLLM and the Transformers library using the following:

FROM 763104351884.dkr.ecr.us-west-2.amazonaws.com/djl-inference:0.31.0-lmi13.0.0-cu124
RUN pip install --no-cache-dir --upgrade pip \
    && pip uninstall -y vllm \
    && pip install --no-cache-dir vllm==0.7.2 \
    && pip uninstall -y transformers \
    && pip install --no-cache-dir git+https://github.com/huggingface/transformers accelerate \
    && pip uninstall -y peft \
    && pip install --no-cache-dir peft==0.14.0

Inference runs successfully, but the Qwen2.5-VL model only recognizes text capabilities and does not process or understand image inputs. Something still seems off.

Any guidance would be appreciated.
Thanks!