Skip to content

Commit 37dfb23

Browse files
danielhanchenErland366bradhiltonSpaceHunterInfXiaochen Zhu
authored
Patch (#138)
* Update compiler.py * Update patching_utils.py * Update temporary_patches.py * Update compiler.py * Update compiler.py * Update temporary_patches.py * Update temporary_patches.py * Update temporary_patches.py * Update temporary_patches.py * Update temporary_patches.py * Update temporary_patches.py * Update temporary_patches.py * Update temporary_patches.py * Update compiler.py * Update compiler.py * Update compiler.py * Update compiler.py * Update compiler.py * Update compiler.py * Update compiler.py * Update compiler.py * Update compiler.py * Update compiler.py * Update compiler.py * Update compiler.py * Update compiler.py * Update compiler.py * Update compiler.py * Update compiler.py * Update compiler.py * Update compiler.py * Update compiler.py * compiler * Update gradient_checkpointing.py * Update temporary_patches.py * Update temporary_patches.py * Update temporary_patches.py * Update temporary_patches.py * Update temporary_patches.py * Update temporary_patches.py * Update temporary_patches.py * Update temporary_patches.py * Update temporary_patches.py * Update temporary_patches.py * Update temporary_patches.py * Update temporary_patches.py * Update temporary_patches.py * Update temporary_patches.py * Update temporary_patches.py * Update temporary_patches.py * Update temporary_patches.py * Update temporary_patches.py * Update temporary_patches.py * causal mask dtype * Fix checkpoint and save from local file (#74) * Enhance gradient checkpointing and add original model ID retrieval in saving utilities * In case adapter_config.json as well * Update patching_utils.py * Update patching_utils.py * Update temporary_patches.py * Update temporary_patches.py * Update compiler.py * Update loss_utils.py * Update compiler.py * Update vllm_utils.py * Update compiler.py * Update peft_utils.py * Update rl_replacements.py * Update vllm_utils.py * Update temporary_patches.py * Update temporary_patches.py * Update temporary_patches.py * Update temporary_patches.py * Update temporary_patches.py * Update temporary_patches.py * Update temporary_patches.py * Update temporary_patches.py * Update temporary_patches.py * Update temporary_patches.py * Update temporary_patches.py * Update temporary_patches.py * Update temporary_patches.py * Update temporary_patches.py * Update compiler.py * Update vllm_lora_worker_manager.py * Update utils.py * Update temporary_patches.py * Update temporary_patches.py * Update temporary_patches.py * Update temporary_patches.py * Update temporary_patches.py * Update temporary_patches.py * Update temporary_patches.py * Update temporary_patches.py * Update temporary_patches.py * Update temporary_patches.py * Update vllm_utils.py * Update vllm_utils.py * Update vllm_utils.py * Update vllm_utils.py * Update dataset_utils.py * bidirectional attention * Update vllm_utils.py * Update __init__.py * Update temporary_patches.py * Update temporary_patches.py * Update temporary_patches.py * Update vllm_utils.py * Update vllm_utils.py * Update vllm_utils.py * Update vllm_utils.py * Update vllm_utils.py * Update vllm_utils.py * Update vllm_lora_worker_manager.py * Update vllm_lora_worker_manager.py * Update vllm_lora_worker_manager.py * Update temporary_patches.py * Update temporary_patches.py * Update temporary_patches.py * Update temporary_patches.py * Update loss_utils.py * Update loss_utils.py * Update loss_utils.py * Update loss_utils.py * Update loss_utils.py * Update __init__.py * fix: AsyncLLMEngine bugs (#82) * fixed a typo in L119, removing unnecessary len() (#84) Co-authored-by: Xiaochen Zhu <[email protected]> * Fix gradient checkpointing warning filter implementation * Input grads fix for gemma3 (#96) * gemma require gradients fix * Update peft_utils.py --------- Co-authored-by: Daniel Han <[email protected]> * Update vision_utils.py * Vision requires grad * Check SDPA for Mistral / Pixtral * Update compiler.py * Update vision_utils.py * Update vision_utils.py * Update vision_utils.py * Update __init__.py * Update vision_utils.py * Update vision_utils.py * Update vision_utils.py * Update vision_utils.py * Update vision_utils.py * Update vision_utils.py * Update vision_utils.py * Update vision_utils.py * Update vision_utils.py * Update vision_utils.py * Update vision_utils.py * Update vision_utils.py * Update vision_utils.py * Update vision_utils.py * Update vllm_utils.py (#99) Fix bugs in generate_batches.py.Original output = [] will result in duplication of results. * Update vision_utils.py * Fixes to support IterableDataset (#98) * Support Iterable Datasets * Update dataset_utils.py * Update dataset_utils.py * Update dataset_utils.py * Update dataset_utils.py * Preserve batch size from iterable dataset * Preserve batch size from iterable dataset * Support train_on_response_only with IterableDataset * Support train_on_response_only with IterableDataset * Support train_on_response_only with IterableDataset * Support train_on_response_only with IterableDataset * Update vllm_utils.py * Create vllm_rlhf_utils.py * Update vllm_rlhf_utils.py * Update vllm_rlhf_utils.py * Update vllm_rlhf_utils.py * Update vllm_rlhf_utils.py * Update vllm_rlhf_utils.py * Update vllm_rlhf_utils.py * Update vllm_rlhf_utils.py * Update vllm_rlhf_utils.py * Update vllm_rlhf_utils.py * vLLM for Qwen 3 * Update vllm_utils.py * Update vllm_utils.py * Update vllm_utils.py * Update vllm_utils.py * Update vllm_utils.py * Update vllm_utils.py * Update vllm_utils.py * Update vllm_utils.py * Update vllm_utils.py * Update vllm_utils.py * Update vllm_utils.py * Update vllm_utils.py * Update vllm_utils.py * Update vllm_utils.py * Update vllm_utils.py * Update vllm_utils.py * Update vllm_utils.py * Update vllm_utils.py * Update vllm_utils.py * Update vllm_utils.py * Update vllm_utils.py * Update vllm_utils.py * Update vllm_utils.py * Update vllm_utils.py * Update vllm_utils.py * Update vllm_utils.py * Update vllm_utils.py * Update vllm_utils.py * Update vllm_utils.py * Update vllm_utils.py * Update vllm_utils.py * Update vllm_utils.py * Update compiler.py * Update vllm_utils.py * Update vllm_utils.py * Update vllm_utils.py * Update vllm_utils.py * Update vllm_utils.py * Update vllm_utils.py * Update vllm_utils.py * Update vllm_utils.py * Update vllm_utils.py * Update vllm_utils.py * Update vllm_utils.py * Update vllm_utils.py * Update vllm_utils.py * Update vllm_utils.py * Update rl_replacements.py * Update rl_replacements.py * Update rl_replacements.py * Update rl_replacements.py * Swap space reduce * Update vllm_utils.py * Update vllm_utils.py * Update rl_replacements.py * Update vllm_utils.py * Update vllm_utils.py * Update vllm_utils.py * Update vllm_utils.py * Update __init__.py * Update rl_replacements.py * Update vllm_utils.py * Update vllm_utils.py * Update vllm_utils.py * Update vllm_utils.py * Update rl_replacements.py * Update vllm_utils.py * Update rl_replacements.py * Revert "Update rl_replacements.py" This reverts commit c0a4022. * Update __init__.py --------- Co-authored-by: Edd <[email protected]> Co-authored-by: Brad Hilton <[email protected]> Co-authored-by: SpaceHunter <[email protected]> Co-authored-by: Xiaochen Zhu <[email protected]> Co-authored-by: Roland Tannous <[email protected]> Co-authored-by: DoubleMathew <[email protected]> Co-authored-by: Michael Han <[email protected]> Co-authored-by: Qian Wu <[email protected]> Co-authored-by: marcandrelarochelle <[email protected]>
1 parent a1cc72e commit 37dfb23

File tree

2 files changed

+9
-5
lines changed

2 files changed

+9
-5
lines changed

‎unsloth_zoo/__init__.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@
1414
# You should have received a copy of the GNU Lesser General Public License
1515
# along with this program. If not, see <https://www.gnu.org/licenses/>.
1616

17-
__version__ = "2025.5.4"
17+
__version__ = "2025.5.5"
1818

1919
from importlib.util import find_spec
2020
if find_spec("unsloth") is None:

‎unsloth_zoo/vllm_utils.py

Lines changed: 8 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -414,11 +414,12 @@ def unpatch_bitsandbytes_compute_dtype():
414414
pass
415415

416416

417-
def patch_vllm():
417+
def patch_vllm(debug = True):
418418
# Temporary patch to disable multiprocessing for vLLM
419419
# Allows accessing model_executor
420420
os.environ["VLLM_ENABLE_V1_MULTIPROCESSING"] = "0"
421-
os.environ["VLLM_LOGGING_LEVEL"] = "DEBUG"
421+
if debug:
422+
os.environ["VLLM_LOGGING_LEVEL"] = "DEBUG"
422423
# os.environ["VLLM_TRACE_FUNCTION"] = "1"
423424
patch_vllm_set_inductor_config()
424425
patch_bitsandbytes_quant_state()
@@ -1524,7 +1525,6 @@ def generate_batches(llm, inputs, n_batches = None, lora_request = None, *args,
15241525

15251526
def delete_vllm(llm = None):
15261527
# From https://github.com/vllm-project/vllm/issues/1908
1527-
import ray
15281528
from vllm.distributed.parallel_state import (
15291529
destroy_model_parallel,
15301530
destroy_distributed_environment,
@@ -1540,7 +1540,11 @@ def delete_vllm(llm = None):
15401540
torch.distributed.destroy_process_group()
15411541
gc.collect()
15421542
torch.cuda.empty_cache()
1543-
ray.shutdown()
1543+
try:
1544+
import ray
1545+
ray.shutdown()
1546+
except:
1547+
pass
15441548
return llm
15451549
pass
15461550

0 commit comments

Comments
 (0)