Skip to content

fix(deps): update dependency accelerate to v1.14.0#321

Open
konflux-internal-p02[bot] wants to merge 1 commit into
mainfrom
konflux/mintmaker/main/accelerate-1.x
Open

fix(deps): update dependency accelerate to v1.14.0#321
konflux-internal-p02[bot] wants to merge 1 commit into
mainfrom
konflux/mintmaker/main/accelerate-1.x

Conversation

@konflux-internal-p02

@konflux-internal-p02 konflux-internal-p02 Bot commented Nov 7, 2025 •
edited
Loading

Copy link
Copy Markdown

ℹ️ Note

This PR body was truncated due to platform limits.

This PR contains the following updates:

Package Change Age Confidence
accelerate ==1.0.1 → ==1.14.0 age confidence

Release Notes

huggingface/accelerate (accelerate)

v1.14.0: : AMD ROCm support, FSDP2 hardening

Compare Source

FSDP2 Improvements

This release brings a large batch of FSDP2 fixes and quality-of-life improvements: correct dtype handling on load, sharding of embeddings/norms, QLoRA crash prevention, and a more robust auto-wrap policy.

AMD ROCm support

Accelerate now works end-to-end on AMD ROCm devices. Thanks @​Abdennacer-Badaoui!

Neuron

Further Neuron improvements to reduce recompilation and cover missing device cases.

Quantization & Offloading

We improved offloading support for quantized models, including Torchao, int8, and tied-weight handling.

Data Loading
  • Feat: Support dynamic batch size in BatchSamplerShard with even_batches by @​yuxinyuan in #​3969
  • Fix iterable dataset sharding condition when n_shards == num_processes by @​SunMarc in #​3958
  • Fix implicit padding in split_between_processes when apply_padding=False and num_samples < num_processes by @​3manifold in #​4052
Minor fixes

Full Changelog: huggingface/accelerate@v1.13.0...v1.14.0

v1.13.0: : Neuron support, IPEX removal, and distributed training fixes

Compare Source

AWS Neuron support

We now have support for AWS Neuron (Trainium/Inferentia) devices. Thanks @​michaelbenayoun for adding this.

XPU Improvements

We've removed IPEX dependency and improved device-agnostic code for XPU.

FSDP2 Improvements

We've added a bunch of important fixes for FSDP2 users: upcasting only grad-requiring params, better tied embedding errors, DCP optimizer loading, bf16 optimizer step crash fix, and torch < 2.7.0 compatibility.

DeepSpeed Sequence Parallelism

We've added several fixes to the DeepSpeed + Sequence Parallelism integration introduced in v1.12.0, including evaluation support during SP training and proper process group handling.

FP8

We've enhanced FP8 training. Thanks @​shimizust for fixing torchao support.

Performance

Accelerate now imports faster by deferring heavy dependencies, and torch.compile hooks are disabled lazily.

Minor fixes

v1.12.0: : Deepspeed Ulysses/ALST

Compare Source

Deepspeed Ulysses/ALST integration

Deepspeed Ulysses/ALST is an efficient way of training on long sequences by employing sequence parallelism and attention head parallelism. You can learn more about this technology in this paper https://arxiv.org/abs/2506.13996 or this deepspeed tutorial https://www.deepspeed.ai/tutorials/ulysses-alst-sequence-parallelism/.

0d8bd9e0

To enable Deepspeed Ulysses, you first need to create ParallelismConfig and setting sp related args:

parallelism_config = ParallelismConfig(
    sp_backend="deepspeed",
    sp_size=2,
    sp_handler=DeepSpeedSequenceParallelConfig(...),
)

Then, you need to make sure to compute the correct loss as described on our docs

        ...
        losses_per_rank = torch.distributed.nn.functional.all_gather(loss, group=sp_group)
        good_tokens = (shift_labels != -100).view(-1).sum()
        good_tokens_per_rank = torch.distributed.nn.functional.all_gather(good_tokens, group=sp_group)
        total_loss = sum(
            losses_per_rank[rank] * good_tokens_per_rank[rank]
            for rank in range(sp_world_size)
            if good_tokens_per_rank[rank] > 0
        )
        total_good_tokens = sum(good_tokens_per_rank)
        loss = total_loss / max(total_good_tokens, 1)

Thanks @​S1ro1 for starting this work and for @​stas00 for finishing this work. Also thanks @​kashif for adding docs and reviewing/testing this PR !

This feature will also be available in HF Trainer thanks for this PR from @​stas00: huggingface/transformers#41832

Minor changes

New Contributors

Full Changelog: huggingface/accelerate@v1.11.0...v1.12.0

v1.11.0: : TE MXFP8, FP16/BF16 with MPS, Python 3.10

Compare Source

TE MXFP8 support

We've added support for MXFP8 in our TransformerEngine integration. To use that, you need to set use_mxfp8_block_scaling in fp8_config. See nvidia docs [here]. (https://docs.nvidia.com/deeplearning/transformer-engine/user-guide/examples/fp8_primer.html#MXFP8-and-block-scaling)

FP16/BF16 Training for MPS devices

BF16 and FP16 support for MPS devices is finally here. You can now pass mixed_precision = "fp16" or "bf16" when training on a mac (fp16 requires torch 2.8 and bf16 requires torch 2.6)

FSDP updates

The following PRs add respectively support to ignored_params and no_sync() for FSDPv2:

Mixed precision can now be passed as a dtype string from accelerate cli flag or fsdp_config in accelerate config file:

Nd-parallel updates

Some minor updates concerning nd-parallelism.

Bump to Python 3.10

We've dropped support for python 3.9 as it reached EOL in October.

Lots of minor fixes:

New Contributors

Full Changelog: huggingface/accelerate@v1.10.1...v1.11.0

v1.10.1: : Patchfix

Compare Source

Full Changelog: huggingface/accelerate@v1.10.0...v1.10.1

v1.10.0: : N-D Parallelism

Compare Source

N-D Parallelism

Training large models across multiple GPUs can be complex, especially when combining different parallelism strategies (e.g TP, CP, DP). To simplify this process, we've collaborated with Axolotl to introduce an easy-to-use integration that allows you to apply any combination of parallelism strategies directly in your training script. Just pass a ParallelismConfig specifying the size of each parallelism type—it's that simple.
Learn more about how it works in our latest blogpost.

parallelism_config = ParallelismConfig(
    dp_shard_size=2,
    dp_replicate_size=2,
    cp_size=2,
    tp_size=2,
)
accelerator = Accelerator(
    parallelism_config=parallelism_config,
   ...
)
model = AutoModelForCausalLM.from_pretrained("your-model-name", device_mesh=accelerator.torch_device_mesh)
model = accelerator.prepare(model)
FSDP improvements

We've fixed ignored modules attribute. With this, it is now possible to train PEFT model that moe layers that contrains q_proj and v_proj parameters. This is especially important for fine-tuning gpt-oss model.

Minor improvements
New Contributors

Full Changelog: huggingface/accelerate@v1.9.0...v1.10.0

v1.9.0: : Trackio support, Model loading speedup, Minor distributed improvements

Compare Source

Trackio tracker support

We've added support for a trackio, lightweight, 💯 free experiment tracking Python library built on top of 🤗 Datasets and Spaces.

Screen Recording 2025-06-11 at 5 39 32 PM

Main features are:

  • Local-first design: dashboard runs locally by default. You can also host it on Spaces by specifying a space_id.
  • Persists logs locally (or in a private Hugging Face Dataset)
  • Visualize experiments with a Gradio dashboard locally (or on Hugging Face Spaces)
  • Everything here, including hosting on Hugging Faces, is free!

To use it with accelerate, you need to set log_with and initialize the trackers

accelerator = Accelerator(log_with="trackio")
config={"learning_rate": 0.001, "batch_size": 32}

# init_kwargs in order to host the dashboard on spaces
init_kwargs = {"trackio": {"space_id": "hf_username/space_name"}
accelerator.init_trackers("example_project", config=config, init_kwargs=init_kwargs})

Thanks @​pcuenca for the integration !

Model loading speedup when relying set_module_tensor_to_device

Setting tensor while clearing cache is very slow, so we added clear_device option to disable it.
Another small optimization is using non_blocking everywhere and syncing just before returning control to the user. This makes the loading slightly faster.

FDSP, Deepspeed, FP8 minor improvements
🚨🚨🚨 Breaking changes 🚨🚨🚨

find_executable_batch_size() will no longer halves the batch after every OOM. Instead, we will multiply the batch size by 0.9. This should help user not waste gpu capacity.

What's Changed
New Contributors

Full Changelog: huggingface/accelerate@v1.8.1...v1.9.0

v1.8.1: : Patchfix

Compare Source

Full Changelog: huggingface/accelerate@v1.8.0...v1.8.1

v1.8.0: : FSDPv2 + FP8, Regional Compilation for DeepSpeed, Faster Distributed Training on Intel CPUs, ipex.optimize deprecation

Compare Source

FSDPv2 refactor + FP8 support

We've simplified how to prepare FSDPv2 models, as there were too many ways to compose FSDP2 with other features (e.g., FP8, torch.compile, activation checkpointing, etc.). Although the setup is now more restrictive, it leads to fewer errors and a more performant user experience. We’ve also added support for FP8. You can read about the results here. Thanks to @​S1ro1 for this contribution!

Faster Distributed Training on Intel CPUs

We updated the CCL_WORKER_COUNT variable and added KMP parameters for Intel CPU users. This significantly improves distributed training performance (e.g., Tensor Parallelism), with up to a 40% speed-up on Intel 4th Gen Xeon when training transformer TP models.

Regional Compilation for DeepSpeed

We added support for regional compilation with the DeepSpeed engine. DeepSpeed’s .compile() modifies models in-place using torch.nn.Module.compile(...), rather than the out-of-place torch.compile(...), so we had to account for that. Thanks @​IlyasMoutawwakil for this feature!

ipex.optimize deprecation

ipex.optimize is being deprecated. Most optimizations have been upstreamed to PyTorch, and future improvements will land there directly. For users without PyTorch 2.8, we’ll continue to rely on IPEX for now.

Better XPU Support

We've greatly expanded and stabilized support for Intel XPUs:

Trackers

We've added support for SwanLab as an experiment tracking backend. Huge thanks to @​ShaohonChen for this contribution ! We also deferred all tracker initializations to prevent premature setup of distributed environments.

What's Changed

✂ Note

PR body was truncated to here.

Configuration

📅 Schedule: (UTC)

  • Branch creation
    • At any time (no schedule defined)
  • Automerge
    • At any time (no schedule defined)

🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.

♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 Ignore: Close this PR and you won't be reminded about these updates again.

  • If you want to rebase/retry this PR, check this box

To execute skipped test pipelines write comment /ok-to-test.

Documentation

Find out how to configure dependency updates in MintMaker documentation or see all available configuration options in Renovate documentation.

@konflux-internal-p02 konflux-internal-p02 Bot force-pushed the konflux/mintmaker/main/accelerate-1.x branch from 6411323 to 6dda555 Compare November 21, 2025 17:12
@konflux-internal-p02 konflux-internal-p02 Bot changed the title chore(deps): update dependency accelerate to v1.11.0 chore(deps): update dependency accelerate to v1.12.0 Nov 21, 2025
@konflux-internal-p02 konflux-internal-p02 Bot force-pushed the konflux/mintmaker/main/accelerate-1.x branch 2 times, most recently from c8504f6 to 2467f98 Compare December 18, 2025 18:59
@konflux-internal-p02 konflux-internal-p02 Bot force-pushed the konflux/mintmaker/main/accelerate-1.x branch from 2467f98 to 48e29cf Compare January 8, 2026 05:13
@konflux-internal-p02 konflux-internal-p02 Bot force-pushed the konflux/mintmaker/main/accelerate-1.x branch from 48e29cf to 97f4b91 Compare January 16, 2026 21:13
@konflux-internal-p02 konflux-internal-p02 Bot force-pushed the konflux/mintmaker/main/accelerate-1.x branch from 97f4b91 to b6bd8a2 Compare February 10, 2026 17:35
@konflux-internal-p02 konflux-internal-p02 Bot changed the title chore(deps): update dependency accelerate to v1.12.0 chore(deps): update dependency accelerate to v1.12.0 - autoclosed Feb 18, 2026
@konflux-internal-p02 konflux-internal-p02 Bot deleted the konflux/mintmaker/main/accelerate-1.x branch February 18, 2026 16:59
@konflux-internal-p02 konflux-internal-p02 Bot changed the title chore(deps): update dependency accelerate to v1.12.0 - autoclosed chore(deps): update dependency accelerate to v1.12.0 Feb 21, 2026
@konflux-internal-p02 konflux-internal-p02 Bot force-pushed the konflux/mintmaker/main/accelerate-1.x branch 2 times, most recently from b6bd8a2 to 5c4e700 Compare February 21, 2026 01:36
@konflux-internal-p02 konflux-internal-p02 Bot force-pushed the konflux/mintmaker/main/accelerate-1.x branch from 5c4e700 to 74510d8 Compare March 4, 2026 21:39
@konflux-internal-p02 konflux-internal-p02 Bot changed the title chore(deps): update dependency accelerate to v1.12.0 chore(deps): update dependency accelerate to v1.13.0 Mar 4, 2026
@konflux-internal-p02 konflux-internal-p02 Bot force-pushed the konflux/mintmaker/main/accelerate-1.x branch from 74510d8 to 4423db8 Compare March 11, 2026 17:47
@konflux-internal-p02 konflux-internal-p02 Bot force-pushed the konflux/mintmaker/main/accelerate-1.x branch from 4423db8 to 84a870a Compare March 26, 2026 17:49
@konflux-internal-p02 konflux-internal-p02 Bot force-pushed the konflux/mintmaker/main/accelerate-1.x branch from 84a870a to c9e9643 Compare April 3, 2026 01:46
@konflux-internal-p02 konflux-internal-p02 Bot force-pushed the konflux/mintmaker/main/accelerate-1.x branch from c9e9643 to 0b3cf29 Compare April 13, 2026 23:07
@konflux-internal-p02 konflux-internal-p02 Bot force-pushed the konflux/mintmaker/main/accelerate-1.x branch from 0b3cf29 to d232ac4 Compare May 8, 2026 16:14
@konflux-internal-p02 konflux-internal-p02 Bot force-pushed the konflux/mintmaker/main/accelerate-1.x branch from d232ac4 to 6869687 Compare June 16, 2026 01:00
@konflux-internal-p02 konflux-internal-p02 Bot changed the title chore(deps): update dependency accelerate to v1.13.0 chore(deps): update dependency accelerate to v1.14.0 Jun 16, 2026
@konflux-internal-p02 konflux-internal-p02 Bot changed the title chore(deps): update dependency accelerate to v1.14.0 fix(deps): update dependency accelerate to v1.14.0 Jun 25, 2026
@konflux-internal-p02
fix(deps): update dependency accelerate to v1.14.0
c73814a 
Signed-off-by: konflux-internal-p02 <170854209+konflux-internal-p02[bot]@users.noreply.github.com>
@konflux-internal-p02 konflux-internal-p02 Bot force-pushed the konflux/mintmaker/main/accelerate-1.x branch from 6869687 to c73814a Compare June 29, 2026 09:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

0 participants