Skip to content

Distributed fine tuning of LLMs  #49

Open
@Shreyanand

Description

@Shreyanand

The finetuning notebook uses 1 GPU and LoRA technique to fine tune a T5 model with 3B parameters. The task to be completed in this issue is to fine tune the same model (or 7B version of the model) on multiple GPU nodes. Use Instascale and Codeflare to schedule the training job and retrieve the finetuned model. Create a notebook that demos this.

Metadata

Metadata

Assignees

Labels

help wantedExtra attention is needed

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions