Skip to content

Fix formatting function to ensure correct fine-tuning in gemma-peft.md #3260

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
dboyker wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Fix formatting function to ensure correct fine-tuning in gemma-peft.md #3260

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 2 additions & 3 deletions gemma-peft.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -112,7 +112,6 @@ To begin with, let's select an English quotes dataset [Abirate/english_quotes](h
from datasets import load_dataset

data = load_dataset("Abirate/english_quotes")
data = data.map(lambda samples: tokenizer(samples["quote"]), batched=True)
```

Now let’s finetune this model using the LoRA config stated above:
Expand All @@ -122,8 +121,8 @@ import transformers
from trl import SFTTrainer

def formatting_func(example):
text = f"Quote: {example['quote'][0]}\nAuthor: {example['author'][0]}<eos>"
return [text]
text = f"Quote: {example['quote']}\nAuthor: {example['author']}<eos>"
return text

trainer = SFTTrainer(
model=model,
Expand Down