Install Cohere Melody vLLM

[tannerdwood]

Purpose

Install Cohere Melody to add support for Cohere AI Models

Test Plan

Manually verified on EC2 p4d.24xlarge (8x A100) using the vLLM DLC with
cohere_melody installed at runtime
Served CohereLabs/command-a-plus-05-2026-w4a4 with --tool-call-parser cohere_command4
--reasoning-parser cohere_command4 --enable-auto-tool-choice
Confirmed successful response from /v1/chat/completions with reasoning field populated
Ran vllm bench serve benchmark: 1611 output tok/s, 12.59 req/s at batch size 64

Test Result

• Model served successfully on EC2
• Reasoning parser working (reasoning field present in response)
• CI pending

---

Toggle if you are merging into master Branch

By default, docker image builds and tests are disabled. Two ways to run builds and tests:

1. Using dlc_developer_config.toml
2. Using this PR description (currently only supported for PyTorch, TensorFlow, vllm, and base images)

How to use the helper utility for updating dlc_developer_config.toml

Assuming your remote is called origin (you can find out more with git remote -v)...

• Run default builds and tests for a particular buildspec - also commits and pushes changes to remote; Example:

python src/prepare_dlc_dev_environment.py -b </path/to/buildspec.yml> -cp origin

• Enable specific tests for a buildspec or set of buildspecs - also commits and pushes changes to remote; Example:

python src/prepare_dlc_dev_environment.py -b </path/to/buildspec.yml> -t sanity_tests -cp origin

• Restore TOML file when ready to merge

python src/prepare_dlc_dev_environment.py -rcp origin

NOTE: If you are creating a PR for a new framework version, please ensure success of the local, standard, rc, and efa sagemaker tests by updating the dlc_developer_config.toml file:

• sagemaker_remote_tests = true
• sagemaker_efa_tests = true
• sagemaker_rc_tests = true
• sagemaker_local_tests = true

How to use PR description

Use the code block below to uncomment commands and run the PR CodeBuild jobs. There are two commands available:

• # /buildspec <buildspec_path>
  • e.g.: # /buildspec pytorch/training/buildspec.yml
  • If this line is commented out, dlc_developer_config.toml will be used.
• # /tests <test_list>
  • e.g.: # /tests sanity security ec2
  • If this line is commented out, it will run the default set of tests (same as the defaults in dlc_developer_config.toml): sanity, security, ec2, ecs, eks, sagemaker, sagemaker-local.

# /buildspec <buildspec_path>
# /tests <test_list>

Toggle if you are merging into main Branch

PR Checklist

• I ran pre-commit run --all-files locally before creating this PR. (Read DEVELOPMENT.md for details).