Search results

HSDP causes loss instability

Status: Open.

#813 In pytorch/torchtitan;

· apkumar opened on Jan 31, 2025

FSDP checkpoints don't load when run is restarted with greater world size

bugSomething isn't workingSomething isn't working

documentationImprovements or additions to documentationImprovements or additions to documentation

enhancementNew feature or requestNew feature or request

Status: Open.

#811 In pytorch/torchtitan;

· darkmirage opened on Jan 28, 2025

 ·  

torchtitan v1.0.0 release

debug model training hangs on NVIDIA B200 with >1 GPU

bugSomething isn't workingSomething isn't working

Status: Open.

#810 In pytorch/torchtitan;

· vkuzo opened on Jan 28, 2025

Loss metrics dramatically change after resuming from checkpoint

bugSomething isn't workingSomething isn't working

Status: Open.

#809 In pytorch/torchtitan;

· darkmirage opened on Jan 28, 2025

unable to run 8b llama

bugSomething isn't workingSomething isn't working

Status: Open.

#807 In pytorch/torchtitan;

· asahni04 opened on Jan 27, 2025

Gradient Scaling With Pipeline Parallelism

questionFurther information is requestedFurther information is requested

Status: Open.

#803 In pytorch/torchtitan;

· windsornguyen opened on Jan 24, 2025

should we have an extension point for model transforms out of tree?

enhancementNew feature or requestNew feature or request

Status: Open.

#790 In pytorch/torchtitan;

· vkuzo opened on Jan 15, 2025

[Bug] Unexpected performance drop with float8 training + compiling only nn.Linear layers + using selective per op AC

bugSomething isn't workingSomething isn't working

Status: Open.

#786 In pytorch/torchtitan;

· danielvegamyhre opened on Jan 10, 2025

Why use RowwiseParallel for nn.Embedding instead of ColwiseParallel?

questionFurther information is requestedFurther information is requested

Status: Open.

#785 In pytorch/torchtitan;

· corey-lambda opened on Jan 10, 2025

BUG: early_step_in_backward with pipeline parallelism and len(model_parts) > 1

bugSomething isn't workingSomething isn't working

Status: Open.

#777 In pytorch/torchtitan;

· cassanof opened on Jan 7, 2025

PP hangs when pipeline_parallel_microbatches < pipeline_parallel_degree

bugSomething isn't workingSomething isn't working

Status: Open.

#775 In pytorch/torchtitan;

· cassanof opened on Jan 6, 2025

ZBVZeroBubble error

bugSomething isn't workingSomething isn't working

Status: Open.

#774 In pytorch/torchtitan;

· hhaAndroid opened on Jan 3, 2025