Minor refactor for LLMC #993
Draft
+107
−64
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: yiliu30 <[email protected]>
Signed-off-by: yiliu30 <[email protected]>
Signed-off-by: yiliu30 <[email protected]>
Signed-off-by: yiliu30 <[email protected]>
Signed-off-by: yiliu30 <[email protected]>
Signed-off-by: yiliu30 <[email protected]>
Signed-off-by: yiliu30 <[email protected]>
Signed-off-by: yiliu30 <[email protected]>
Signed-off-by: yiliu30 <[email protected]>
Signed-off-by: yiliu30 <[email protected]>
Signed-off-by: yiliu30 <[email protected]>
Signed-off-by: yiliu30 <[email protected]>
Signed-off-by: yiliu30 <[email protected]>
Signed-off-by: yiliu30 <[email protected]>
Signed-off-by: yiliu30 <[email protected]>
Signed-off-by: yiliu30 <[email protected]>
Signed-off-by: yiliu30 <[email protected]>
Signed-off-by: yiliu30 <[email protected]>
Signed-off-by: yiliu30 <[email protected]>
Signed-off-by: yiliu30 <[email protected]>
Signed-off-by: yiliu30 <[email protected]>
Signed-off-by: yiliu30 <[email protected]>
Signed-off-by: yiliu30 <[email protected]>
Signed-off-by: yi <[email protected]>
Signed-off-by: root <[email protected]>
Signed-off-by: root <[email protected]>
Signed-off-by: root <[email protected]>
Signed-off-by: yiliu30 <[email protected]>
Signed-off-by: yiliu30 <[email protected]>
wenhuach21
reviewed
Nov 5, 2025
| block, self.device_map, input_ids, self.low_gpu_mem_usage, self.batch_size, device | ||
| ) | ||
| if normalize_inputs: | ||
| input_ids, input_others = normalize_input(inputs) |
There was a problem hiding this comment.
why not moving these change to llmc side
wenhuach21
reviewed
Nov 5, 2025
auto_round/compressors/base.py
Outdated
|
|
||
| # function decorator to dump the func time | ||
| def time_logger(func: Callable) -> Callable: | ||
| """Decorator to log the execution time of a function. |
wenhuach21
reviewed
Nov 5, 2025
| self.inner_supported_types = tuple(x for x in INNER_SUPPORTED_LAYER_TYPES if x != "FP8Linear") | ||
| self.batch_dim = None | ||
| # TODO: check with heng/weiwei | ||
| self.batch_dim = 0 |
There was a problem hiding this comment.
if this is required, hidden in kwargs, and add comments
wenhuach21
reviewed
Nov 5, 2025
| q_inputs = q_inputs.pop(input_id_str[0], None) | ||
| return inputs, q_inputs | ||
|
|
||
| def configure_layer_config(self, enable_gguf_official_mixed: None | bool = False): |
There was a problem hiding this comment.
better set the enable_gguf_official_mixed to True by default
wenhuach21
reviewed
Nov 5, 2025
auto_round/compressors/base.py
Outdated
| q_input: Union[torch.Tensor, dict, None] = None, | ||
| normalize_inputs: bool = False, | ||
| device: Union[str, torch.device] = "cpu", | ||
| auto_offload=True |
There was a problem hiding this comment.
@n1ck-guo is working on splitting this function. We should avoid fusing too many operations that are not actually used by AutoRound itself in this function.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.