@@ -275,18 +275,20 @@ Let's first copy over the config to our local working directory so we can make c
275
275
276
276
$ tune cp generation ./custom_generation_config.yaml
277
277
Copied file to custom_generation_config.yaml
278
+ $ mkdir /tmp/torchtune/llama3_2_3B/lora_single_device/out
278
279
279
280
Let's modify ``custom_generation_config.yaml `` to include the following changes. Again, you only need
280
281
to replace two fields: ``output_dir `` and ``checkpoint_files ``
281
282
282
283
.. code-block :: yaml
283
284
284
- output_dir : /tmp/torchtune/llama3_2_3B/lora_single_device/epoch_0
285
+ checkpoint_dir : /tmp/torchtune/llama3_2_3B/lora_single_device/epoch_0
286
+ output_dir : /tmp/torchtune/llama3_2_3B/lora_single_device/out
285
287
286
288
# Tokenizer
287
289
tokenizer :
288
290
_component_ : torchtune.models.llama3.llama3_tokenizer
289
- path : ${output_dir }/original/tokenizer.model
291
+ path : ${checkpoint_dir }/original/tokenizer.model
290
292
prompt_template : null
291
293
292
294
model :
@@ -295,7 +297,7 @@ Let's modify ``custom_generation_config.yaml`` to include the following changes.
295
297
296
298
checkpointer :
297
299
_component_ : torchtune.training.FullModelHFCheckpointer
298
- checkpoint_dir : ${output_dir }
300
+ checkpoint_dir : ${checkpoint_dir }
299
301
checkpoint_files : [
300
302
ft-model-00001-of-00002.safetensors,
301
303
ft-model-00002-of-00002.safetensors,
@@ -312,8 +314,8 @@ Let's modify ``custom_generation_config.yaml`` to include the following changes.
312
314
313
315
# Generation arguments; defaults taken from gpt-fast
314
316
prompt :
315
- system : null
316
- user : " Tell me a joke. "
317
+ system : null
318
+ user : " Tell me a joke. "
317
319
max_new_tokens : 300
318
320
temperature : 0.6 # 0.8 and 0.6 are popular values to try
319
321
top_k : 300
@@ -330,7 +332,7 @@ these parameters.
330
332
331
333
.. code-block :: text
332
334
333
- $ tune run generate --config ./custom_generation_config.yaml prompt="tell me a joke. "
335
+ $ tune run generate --config ./custom_generation_config.yaml prompt.user="Tell me a joke. "
334
336
Tell me a joke. Here's a joke for you:
335
337
336
338
What do you call a fake noodle?
0 commit comments