Skip to content

replace 'evaluation_strategy' with 'eval_strategy' #914

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

replace 'evaluation_strategy' with 'eval_strategy' #914

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions chapters/de/chapter3/3.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,7 @@ trainer.train()

Dadurch wird das Fein-tunen gestartet (was auf einer GPU ein paar Minuten dauern sollte) und der Trainingsverlust wird alle 500 Schritte gemeldet. Es wird jedoch nicht zurückgegeben, wie gut (oder schlecht) das Modell funktioniert. Dies liegt an folgenden Punkten:

1. Wir haben dem `Trainer` nicht mitgeteilt die Performance in der Trainingsschleife auszuwerten, indem wir `evaluation_strategy` entweder auf `"steps"` (alle `eval_steps` auswerten) oder `"epoch"` (am Ende jeder Epoche evaluieren) gesetzt haben.
1. Wir haben dem `Trainer` nicht mitgeteilt die Performance in der Trainingsschleife auszuwerten, indem wir `eval_strategy` entweder auf `"steps"` (alle `eval_steps` auswerten) oder `"epoch"` (am Ende jeder Epoche evaluieren) gesetzt haben.
2. Wir haben dem `Trainer` keine Funktion `compute_metrics()` zur Verfügung gestellt, um während der Evaluation eine Metrik zu berechnen (sonst hätte die Evaluation nur den Verlust ausgegeben, was keine sehr intuitive Zahl ist).


Expand Down Expand Up @@ -138,7 +138,7 @@ def compute_metrics(eval_preds):
Um diese Funktion in Aktion zu sehen, definieren wir einen neuen `Trainer` mit der Funktion "compute_metrics()", um am Ende jeder Epoche Metriken zu melden:

```py
training_args = TrainingArguments("test-trainer", evaluation_strategy="epoch")
training_args = TrainingArguments("test-trainer", eval_strategy="epoch")
model = AutoModelForSequenceClassification.from_pretrained(checkpoint, num_labels=2)

trainer = Trainer(
Expand All @@ -152,7 +152,7 @@ trainer = Trainer(
)
```

Hier ein Hinweis, dass wir ein neues `TrainingArguments` errstellen, dessen `evaluation_strategy` auf `"epoch"` gesetzt ist, und ein neues Modell definieren - andernfalls würden wir nur das Training des momentanen Modells fortführen, das wir bereits trainiert haben. Um einen neuen Trainingslauf zu starten, führen wir folgendes aus:
Hier ein Hinweis, dass wir ein neues `TrainingArguments` errstellen, dessen `eval_strategy` auf `"epoch"` gesetzt ist, und ein neues Modell definieren - andernfalls würden wir nur das Training des momentanen Modells fortführen, das wir bereits trainiert haben. Um einen neuen Trainingslauf zu starten, führen wir folgendes aus:

```
trainer.train()
Expand Down
2 changes: 1 addition & 1 deletion chapters/en/chapter3/6.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -193,7 +193,7 @@ Test what you learned in this chapter!
},
{
text: "It just contains the hyperparameters used for training.",
explain: "In the example, we used an <code>evaluation_strategy</code> as well, so this impacts evaluation. Try again!"
explain: "In the example, we used an <code>eval_strategy</code> as well, so this impacts evaluation. Try again!"
}
]}
/>
Expand Down
2 changes: 1 addition & 1 deletion chapters/en/chapter7/2.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -710,7 +710,7 @@ from transformers import TrainingArguments

args = TrainingArguments(
"bert-finetuned-ner",
evaluation_strategy="epoch",
eval_strategy="epoch",
save_strategy="epoch",
learning_rate=2e-5,
num_train_epochs=3,
Expand Down
2 changes: 1 addition & 1 deletion chapters/en/chapter7/3.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -697,7 +697,7 @@ model_name = model_checkpoint.split("/")[-1]
training_args = TrainingArguments(
output_dir=f"{model_name}-finetuned-imdb",
overwrite_output_dir=True,
evaluation_strategy="epoch",
eval_strategy="epoch",
learning_rate=2e-5,
weight_decay=0.01,
per_device_train_batch_size=batch_size,
Expand Down
2 changes: 1 addition & 1 deletion chapters/en/chapter7/4.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -676,7 +676,7 @@ from transformers import Seq2SeqTrainingArguments

args = Seq2SeqTrainingArguments(
f"marian-finetuned-kde4-en-to-fr",
evaluation_strategy="no",
eval_strategy="no",
save_strategy="epoch",
learning_rate=2e-5,
per_device_train_batch_size=32,
Expand Down
2 changes: 1 addition & 1 deletion chapters/en/chapter7/5.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -513,7 +513,7 @@ model_name = model_checkpoint.split("/")[-1]

args = Seq2SeqTrainingArguments(
output_dir=f"{model_name}-finetuned-amazon-en-es",
evaluation_strategy="epoch",
eval_strategy="epoch",
learning_rate=5.6e-5,
per_device_train_batch_size=batch_size,
per_device_eval_batch_size=batch_size,
Expand Down
2 changes: 1 addition & 1 deletion chapters/en/chapter7/6.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -432,7 +432,7 @@ args = TrainingArguments(
output_dir="codeparrot-ds",
per_device_train_batch_size=32,
per_device_eval_batch_size=32,
evaluation_strategy="steps",
eval_strategy="steps",
eval_steps=5_000,
logging_steps=5_000,
gradient_accumulation_steps=8,
Expand Down
2 changes: 1 addition & 1 deletion chapters/en/chapter7/7.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -837,7 +837,7 @@ from transformers import TrainingArguments

args = TrainingArguments(
"bert-finetuned-squad",
evaluation_strategy="no",
eval_strategy="no",
save_strategy="epoch",
learning_rate=2e-5,
num_train_epochs=3,
Expand Down
10 changes: 5 additions & 5 deletions chapters/en/chapter8/4.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ model = AutoModelForSequenceClassification.from_pretrained(model_checkpoint)

args = TrainingArguments(
f"distilbert-finetuned-mnli",
evaluation_strategy="epoch",
eval_strategy="epoch",
save_strategy="epoch",
learning_rate=2e-5,
num_train_epochs=3,
Expand Down Expand Up @@ -123,7 +123,7 @@ model = AutoModelForSequenceClassification.from_pretrained(model_checkpoint)

args = TrainingArguments(
f"distilbert-finetuned-mnli",
evaluation_strategy="epoch",
eval_strategy="epoch",
save_strategy="epoch",
learning_rate=2e-5,
num_train_epochs=3,
Expand Down Expand Up @@ -318,7 +318,7 @@ model = AutoModelForSequenceClassification.from_pretrained(model_checkpoint)

args = TrainingArguments(
f"distilbert-finetuned-mnli",
evaluation_strategy="epoch",
eval_strategy="epoch",
save_strategy="epoch",
learning_rate=2e-5,
num_train_epochs=3,
Expand Down Expand Up @@ -444,7 +444,7 @@ model = AutoModelForSequenceClassification.from_pretrained(model_checkpoint, num

args = TrainingArguments(
f"distilbert-finetuned-mnli",
evaluation_strategy="epoch",
eval_strategy="epoch",
save_strategy="epoch",
learning_rate=2e-5,
num_train_epochs=3,
Expand Down Expand Up @@ -655,7 +655,7 @@ model = AutoModelForSequenceClassification.from_pretrained(model_checkpoint, num

args = TrainingArguments(
f"distilbert-finetuned-mnli",
evaluation_strategy="epoch",
eval_strategy="epoch",
save_strategy="epoch",
learning_rate=2e-5,
num_train_epochs=3,
Expand Down
4 changes: 2 additions & 2 deletions chapters/es/chapter3/3.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,7 @@ trainer.train()

Esto iniciará el ajuste (que debería tardar un par de minutos en una GPU) e informará de la training loss cada 500 pasos. Sin embargo, no te dirá lo bien (o mal) que está rindiendo tu modelo. Esto se debe a que:

1. No le hemos dicho al `Trainer` que evalúe el modelo durante el entrenamiento especificando un valor para `evaluation_strategy`: `steps` (evaluar cada `eval_steps`) o `epoch` (evaluar al final de cada época).
1. No le hemos dicho al `Trainer` que evalúe el modelo durante el entrenamiento especificando un valor para `eval_strategy`: `steps` (evaluar cada `eval_steps`) o `epoch` (evaluar al final de cada época).
2. No hemos proporcionado al `Trainer` una función `compute_metrics()` para calcular una métrica durante dicha evaluación (de lo contrario, la evaluación sólo habría impreso la pérdida, que no es un número muy intuitivo).

### Evaluación
Expand Down Expand Up @@ -161,7 +161,7 @@ trainer = Trainer(
)
```

Ten en cuenta que hemos creado un nuevo `TrainingArguments` con su `evaluation_strategy` configurado como `"epoch"` y un nuevo modelo. De lo contrario sólo estaríamos continuando el entrenamiento del modelo que ya habíamos entrenado. Para lanzar una nueva ejecución de entrenamiento, ejecutamos:
Ten en cuenta que hemos creado un nuevo `TrainingArguments` con su `eval_strategy` configurado como `"epoch"` y un nuevo modelo. De lo contrario sólo estaríamos continuando el entrenamiento del modelo que ya habíamos entrenado. Para lanzar una nueva ejecución de entrenamiento, ejecutamos:

```py
trainer.train()
Expand Down
2 changes: 1 addition & 1 deletion chapters/es/chapter3/6.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -214,7 +214,7 @@ A ver qué has aprendido en este capítulo:
{
text: "Solo contiene los hiperparámetros utilizados para el entrenamiento.",
explain:
"En el ejemplo también utilizamos <code>evaluation_strategy</code>, que afecta a la evaluación. ¡Inténtalo de nuevo!",
"En el ejemplo también utilizamos <code>eval_strategy</code>, que afecta a la evaluación. ¡Inténtalo de nuevo!",
},
]}
/>
Expand Down
6 changes: 3 additions & 3 deletions chapters/fa/chapter3/3.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -107,7 +107,7 @@ trainer.train()

این کار، کوک کردن را شروع می‌کند (که باید چند دقیقه روی GPU طول بکشد) و هزینه تعلیم را هر ۵۰۰ مرحله یک‌بار گزارش می‌کند. با این حال به شما نمی‌گوید که مدل‌تان چقدر خوب (یا بد) عمل می‌کند. این به این خاطر است که:

۱. ما به `Trainer` نگفتیم که در حین تعلیم کیفیت مدل را اندازه‌گیری کند. کاری که می‌توانستیم با مقداردهی پارامتر `evaluation_strategy` به `"steps"` (برای ارزیابی در هر `eval_steps`) یا به `"epoch"` (برای ارزیابی در انتهای هر epoch) انجام دهیم.
۱. ما به `Trainer` نگفتیم که در حین تعلیم کیفیت مدل را اندازه‌گیری کند. کاری که می‌توانستیم با مقداردهی پارامتر `eval_strategy` به `"steps"` (برای ارزیابی در هر `eval_steps`) یا به `"epoch"` (برای ارزیابی در انتهای هر epoch) انجام دهیم.

۲. ما تابع <span dir="ltr">`compute_metrics()`</span> را برای `Trainer` فراهم نکردیم تا بتواند معیارها را در حین اصطلاحا ارزیابی محاسبه کند (که در غیر این صورت، ارزیابی فقط هزینه را چاپ می‌کند که عدد چندان گویایی هم نیست) .

Expand Down Expand Up @@ -180,7 +180,7 @@ def compute_metrics(eval_preds):
<div dir="ltr">

```py
training_args = TrainingArguments("test-trainer", evaluation_strategy="epoch")
training_args = TrainingArguments("test-trainer", eval_strategy="epoch")
model = AutoModelForSequenceClassification.from_pretrained(checkpoint, num_labels=2)

trainer = Trainer(
Expand All @@ -196,7 +196,7 @@ trainer = Trainer(

</div>

توجه داشته باشید که ما مدلی جدید و `TrainingArguments` جدیدی که `evaluation_strategy` آن `"epoch"` است می‌سازیم - در غیر این صورت فقط تعلیم مدلی که از پیش تعلیم دیده بود را ادامه می‌دادیم. برای راه‌اندازی دور جدید تعلیم، دستور زیر را اجرا می‌کنیم:
توجه داشته باشید که ما مدلی جدید و `TrainingArguments` جدیدی که `eval_strategy` آن `"epoch"` است می‌سازیم - در غیر این صورت فقط تعلیم مدلی که از پیش تعلیم دیده بود را ادامه می‌دادیم. برای راه‌اندازی دور جدید تعلیم، دستور زیر را اجرا می‌کنیم:

<div dir="ltr">

Expand Down
6 changes: 3 additions & 3 deletions chapters/fr/chapter3/3.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -85,7 +85,7 @@ trainer.train()

Cela lancera le *finetuning* (qui devrait prendre quelques minutes sur un GPU) et indiquera la perte d'entraînement tous les 500 pas. Cependant, elle ne vous dira pas si votre modèle fonctionne bien (ou mal). Ceci est dû au fait que :

1. nous n'avons pas dit au `Trainer` d'évaluer pendant l'entraînement en réglant `evaluation_strategy` à soit `"steps"` (évaluer chaque `eval_steps`) ou `"epoch"` (évaluer à la fin de chaque *epoch*).
1. nous n'avons pas dit au `Trainer` d'évaluer pendant l'entraînement en réglant `eval_strategy` à soit `"steps"` (évaluer chaque `eval_steps`) ou `"epoch"` (évaluer à la fin de chaque *epoch*).
2. nous n'avons pas fourni au `Trainer` une fonction `compute_metrics()` pour calculer une métrique pendant ladite évaluation (sinon l'évaluation aurait juste affiché la perte, qui n'est pas un nombre très intuitif).


Expand Down Expand Up @@ -140,7 +140,7 @@ def compute_metrics(eval_preds):
Et pour le voir utilisé en action pour rapporter les métriques à la fin de chaque époque, voici comment nous définissons un nouveau `Trainer` avec cette fonction `compute_metrics()` :

```py
training_args = TrainingArguments("test-trainer", evaluation_strategy="epoch")
training_args = TrainingArguments("test-trainer", eval_strategy="epoch")
model = AutoModelForSequenceClassification.from_pretrained(checkpoint, num_labels=2)

trainer = Trainer(
Expand All @@ -154,7 +154,7 @@ trainer = Trainer(
)
```

Notez que nous créons un nouveau `TrainingArguments` avec sa `evaluation_strategy` définie sur `"epoch"` et un nouveau modèle. Sinon, nous ne ferions que continuer l'entraînement du modèle que nous avons déjà entraîné. Pour lancer un nouveau cycle d'entraînement, nous exécutons :
Notez que nous créons un nouveau `TrainingArguments` avec sa `eval_strategy` définie sur `"epoch"` et un nouveau modèle. Sinon, nous ne ferions que continuer l'entraînement du modèle que nous avons déjà entraîné. Pour lancer un nouveau cycle d'entraînement, nous exécutons :

```
trainer.train()
Expand Down
2 changes: 1 addition & 1 deletion chapters/fr/chapter3/6.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -193,7 +193,7 @@ Testez ce que vous avez appris dans ce chapitre !
},
{
text: "Contenir seulement les hyperparamètres utilisés pour l'entraînement.",
explain: "Dans l'exemple, nous avons utilisé une <code>evaluation_strategy</code> également, ce qui a un impact sur l'évaluation. Essayez à nouveau !"
explain: "Dans l'exemple, nous avons utilisé une <code>eval_strategy</code> également, ce qui a un impact sur l'évaluation. Essayez à nouveau !"
}
]}
/>
Expand Down
2 changes: 1 addition & 1 deletion chapters/fr/chapter7/2.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -713,7 +713,7 @@ from transformers import TrainingArguments

args = TrainingArguments(
"bert-finetuned-ner",
evaluation_strategy="epoch",
eval_strategy="epoch",
save_strategy="epoch",
learning_rate=2e-5,
num_train_epochs=3,
Expand Down
2 changes: 1 addition & 1 deletion chapters/fr/chapter7/3.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -702,7 +702,7 @@ model_name = model_checkpoint.split("/")[-1]
training_args = TrainingArguments(
output_dir=f"{model_name}-finetuned-imdb",
overwrite_output_dir=True,
evaluation_strategy="epoch",
eval_strategy="epoch",
learning_rate=2e-5,
weight_decay=0.01,
per_device_train_batch_size=batch_size,
Expand Down
2 changes: 1 addition & 1 deletion chapters/fr/chapter7/4.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -688,7 +688,7 @@ from transformers import Seq2SeqTrainingArguments

args = Seq2SeqTrainingArguments(
f"marian-finetuned-kde4-en-to-fr",
evaluation_strategy="no",
eval_strategy="no",
save_strategy="epoch",
learning_rate=2e-5,
per_device_train_batch_size=32,
Expand Down
2 changes: 1 addition & 1 deletion chapters/fr/chapter7/5.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -541,7 +541,7 @@ model_name = model_checkpoint.split("/")[-1]

args = Seq2SeqTrainingArguments(
output_dir=f"{model_name}-finetuned-amazon-en-es",
evaluation_strategy="epoch",
eval_strategy="epoch",
learning_rate=5.6e-5,
per_device_train_batch_size=batch_size,
per_device_eval_batch_size=batch_size,
Expand Down
2 changes: 1 addition & 1 deletion chapters/fr/chapter7/6.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -431,7 +431,7 @@ args = TrainingArguments(
output_dir="codeparrot-ds",
per_device_train_batch_size=32,
per_device_eval_batch_size=32,
evaluation_strategy="steps",
eval_strategy="steps",
eval_steps=5_000,
logging_steps=5_000,
gradient_accumulation_steps=8,
Expand Down
2 changes: 1 addition & 1 deletion chapters/fr/chapter7/7.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -858,7 +858,7 @@ from transformers import TrainingArguments

args = TrainingArguments(
"bert-finetuned-squad",
evaluation_strategy="no",
eval_strategy="no",
save_strategy="epoch",
learning_rate=2e-5,
num_train_epochs=3,
Expand Down
10 changes: 5 additions & 5 deletions chapters/fr/chapter8/4.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ model = AutoModelForSequenceClassification.from_pretrained(model_checkpoint)

args = TrainingArguments(
f"distilbert-finetuned-mnli",
evaluation_strategy="epoch",
eval_strategy="epoch",
save_strategy="epoch",
learning_rate=2e-5,
num_train_epochs=3,
Expand Down Expand Up @@ -125,7 +125,7 @@ model = AutoModelForSequenceClassification.from_pretrained(model_checkpoint)

args = TrainingArguments(
f"distilbert-finetuned-mnli",
evaluation_strategy="epoch",
eval_strategy="epoch",
save_strategy="epoch",
learning_rate=2e-5,
num_train_epochs=3,
Expand Down Expand Up @@ -320,7 +320,7 @@ model = AutoModelForSequenceClassification.from_pretrained(model_checkpoint)

args = TrainingArguments(
f"distilbert-finetuned-mnli",
evaluation_strategy="epoch",
eval_strategy="epoch",
save_strategy="epoch",
learning_rate=2e-5,
num_train_epochs=3,
Expand Down Expand Up @@ -447,7 +447,7 @@ model = AutoModelForSequenceClassification.from_pretrained(model_checkpoint, num

args = TrainingArguments(
f"distilbert-finetuned-mnli",
evaluation_strategy="epoch",
eval_strategy="epoch",
save_strategy="epoch",
learning_rate=2e-5,
num_train_epochs=3,
Expand Down Expand Up @@ -658,7 +658,7 @@ model = AutoModelForSequenceClassification.from_pretrained(model_checkpoint, num

args = TrainingArguments(
f"distilbert-finetuned-mnli",
evaluation_strategy="epoch",
eval_strategy="epoch",
save_strategy="epoch",
learning_rate=2e-5,
num_train_epochs=3,
Expand Down
Loading