Skip to content

Conversation

@mivehk
Copy link
Contributor

@mivehk mivehk commented Dec 14, 2025

Purpose

The ‘parse_raw_prompts’ can benefit from the stricter input validation in the array-of-arrays input path. This is for when input is a list of lists; however, we verify the nested inner list is a non-null list of integers beyond the first inner list verification.

Test Plan

Added a pytest coverage for invalid and edge-case inputs of mixed-type nested lists for parse_raw_prompts. Updated the array-of-arrays input condition of parse_raw_prompts to strictly check empty prompt lists within list and mixed-type nested lists. Validated these invalid inputs by running “pytest tests/test_inputs.py” to confirm correct exception types are raised for malformed inputs.

Test Result

Before this change, mixed-type nested prompt inputs were not checked for rejection and could lead to ambiguous behavior. After applying strict validation, the new and existing tests pass, and malformed inputs deterministically raise TypeError or ValueError as expected.


Essential Elements of an Effective PR Description Checklist
  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan, such as providing test command.
  • The test results, such as pasting the results comparison before and after, or e2e results
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
  • (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

@chatgpt-codex-connector
Copy link

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.

Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request aims to strengthen input validation for parse_raw_prompts, particularly for list-of-lists inputs. The changes introduce stricter checks and add a new test case. While the intent is good, the new validation logic in vllm/inputs/parse.py is not fully robust. It fails to correctly validate all elements in nested lists and doesn't consistently check for empty sub-lists. I've provided a critical review comment with a suggested code change to fix these validation flaws, ensuring all sub-prompts are correctly and thoroughly validated.

@mergify
Copy link

mergify bot commented Dec 14, 2025

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @mivehk.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

@mergify mergify bot added the needs-rebase label Dec 14, 2025
@mivehk mivehk force-pushed the improve-parse-tests-v2 branch from 4c270c1 to 28d588a Compare December 14, 2025 19:41
@mergify mergify bot removed the needs-rebase label Dec 14, 2025
Copy link
Member

@DarkLight1337 DarkLight1337 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks

@DarkLight1337 DarkLight1337 enabled auto-merge (squash) December 15, 2025 00:47
@github-actions github-actions bot added the ready ONLY add when PR is ready to merge/full CI is needed label Dec 15, 2025
@DarkLight1337
Copy link
Member

PTAL at the failing test

auto-merge was automatically disabled December 15, 2025 14:31

Head branch was pushed to by a user without write access

@mivehk mivehk force-pushed the improve-parse-tests-v2 branch from 66685ab to 598e47d Compare December 16, 2025 02:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready ONLY add when PR is ready to merge/full CI is needed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants