Fix smoothquant DDP test and refactor the test#2557
Conversation
|
👋 Hi! Thank you for contributing to llm-compressor. Please add the ready label when the PR is ready for review. Note: This is required to complete the testing suite, please only add the label once the PR is code complete and local testing has been performed. |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces a distributed runner for SmoothQuant testing and updates the corresponding integration tests to use this runner. The review identified a potential issue where the number of calibration samples passed to the oneshot function in the distributed runner should be based on the local dataset shard size rather than the total sample count. Additionally, improvements were suggested for the command-line argument validation to use standard error output and cleaner string formatting.
kylesayrs
left a comment
kylesayrs
left a comment
There was a problem hiding this comment.
This will run as part of the per-commit tests. Typically we want the oneshot tests to run as part of the transformers tests
original test had an issue where it would call tok.apply_chat_template without having a chat template being defined. This was hard to catch because the whole script was in a string. I turned everything into functions and pulled the DDP part into a separate file to make it all more readable. Signed-off-by: Author Name <authoremail@example.com>
Summary Signed-off-by: HDCharles <charlesdavidhernandez@gmail.com>
HDCharles
force-pushed
the
91_fix_ddp_smooth
branch
from
93b3219 to
fb91c72
Compare
original test had an issue where it would call tok.apply_chat_template without having a chat template being defined. This was hard to catch because the whole script was in a string. I turned everything into functions and pulled the DDP part into a separate file to make it all more readable. Test Plan: pytest /home/HDCharles/repos/llm-compressor/tests/llmcompressor/modifiers/transform/smoothquant/test_smoothquant_distributed.py --------- Signed-off-by: Author Name <authoremail@example.com> Signed-off-by: HDCharles <charlesdavidhernandez@gmail.com> Signed-off-by: Ziming <frankziming26@outlook.com>
3 participants
original test had an issue where it would call tok.apply_chat_template without having a chat template being defined.
This was hard to catch because the whole script was in a string.
I turned everything into functions and pulled the DDP part into a separate file to make it all more readable.
Test Plan:
pytest /home/HDCharles/repos/llm-compressor/tests/llmcompressor/modifiers/transform/smoothquant/test_smoothquant_distributed.py