Skip to content

PP hangs when pipeline_parallel_microbatches < pipeline_parallel_degree #775

Open
@cassanof

Description

@cassanof

Pipeline parallelism seem to hang when the number of microbatches is less than the degree.
This issue occurs for both the standard and interleaved 1F1B schedules. Have not tested other schedules.

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions