Skip to content

Support Qwen3-VL-30B-A3B-Instruct#124

Merged
dongwang218 merged 3 commits intomainfrom
add_ray_job_http_port
Dec 30, 2025
Merged

Support Qwen3-VL-30B-A3B-Instruct#124
dongwang218 merged 3 commits intomainfrom
add_ray_job_http_port

Conversation

@dongwang218
Copy link
Contributor

Why ?

support Qwen3-VL-30B-A3B-Instruct

How ?

  • add Qwen3-VL-30B-A3B-Instruct to the llm_config
  • another fix is to create random job submit http port to avoid ray conflict
  • bump version

Test plan

matrix deploy_applications --action add --applications "[{'model_name': '/checkpoint/data/shared/Qwen3-VL-30B-A3B-Instruct', 'use_grpc': 'false', 'min_replica': 8, 'name': 'qwen3vl', 'model_size': 'Qwen3-VL-30B-A3B-Instruct', 'enable_tools': 'true'}]"

matrix check_health --app_name qwen3vl --use_image True --use_curl False
{
  "request": {
    "model": "/checkpoint/data/shared/Qwen3-VL-30B-A3B-Instruct",
    "messages": [
      {
        "role": "system",
        "content": "You are a helpful assistant."
      },
      {
        "role": "user",
        "content": [
          {
            "type": "image_url",
            "image_url": {
              "url": "https://ofasys-multimodal-wlcb-3-toshanghai.oss-accelerate.aliyuncs.com/wpf272043/keepme/image/receipt.png"
            }
          },
          {
            "type": "text",
            "text": "Read all the text in the image."
          }
        ]
      }
    ],
    "temperature": 0.7,
    "metadata": {
      "request_timestamp": 1767114966.6636734
    }
  },
  "response": {
    "finish_reason": ["stop"],
    "response_timestamp": 1767114970.4188695,
    "text": [
      "Auntie Anne's\nCINNAMON SUGAR\n1 x 17,000 17,000\nSUB TOTAL 17,000\nGRAND TOTAL 17,000\nCASH IDR 20,000\nCHANGE DUE 3,000"
    ],
    "usage": {
      "prompt_tokens": 257,
      "completion_tokens": 74,
      "total_tokens": 331
    }
  }
}

@meta-cla meta-cla bot added the CLA Signed This label is managed by the Meta Open Source bot. label Dec 30, 2025
Copy link
Contributor

@swdanielli swdanielli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dongwang218 dongwang218 merged commit a84f379 into main Dec 30, 2025
8 checks passed
@dongwang218 dongwang218 deleted the add_ray_job_http_port branch December 30, 2025 19:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Meta Open Source bot.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants