Add a playbook of llama factory fine-tuning

[zhangnju]

This playbook is based on llama factory, and listed the below info:1) playbook duration and risk 2) detailed fine-tuning instructions 3) the introduction to important components of llama factory

[adamlam2-amd]

Thanks for the PR - overall nice job adding the functionalities and nuances of llama-factory.

Couple higher level comments:

• we will use Pytorch+ROCm that's already installed on the Box; thus, we don't need any docker commands
• I think in general we might be able to shorten the playbook. We can focus on one main outcome in the actual playbook - lora/qlora/etc - and reference the rest of them in the next steps section. We don't want to overload the user with too much information during the actual steps.
• Would recommend using the llama-factory UI as it seems more intuitive(?)
• General comments regarding wording, conciseness, and consistency of words/headings.

[danielholanda]

@adamlam2-amd any other comments here?
@iswaryaalex please also make sure to review at some point this week.

[adamlam2-amd]

This playbook is still problematic. There are 4 major questions that need to be resolved, among other more minor things.

1. Are we recommending to use Docker? @danielholanda need your guidance, as the HaloBox comes pre-installed with Pytorch already. But if we open it up to other AMD hardware, what is the strategy here? I recommend not using Docker at all and installing from source. This includes removing the 'Optional' bits as they can confuse users.

2. Do we need to recommend bits and bytes? I feel this will complicate the overall process. We can mention it in the next steps instead.

3. Must we use the CLI tool? LlaMA Factory has a great UI that I think will be a much better user experience. We can add pictures as well. I recommend not using the CLI in favour of the GUI.

4. What model and fine-tuning method is recommended? Let's have 1 for each so the user can understand, and then we can add more in the 'next steps' section.

As a reference, this LlaMA Factory guide is more helpful: https://www.datacamp.com/tutorial/llama-factory-web-ui-guide-fine-tuning-llms. In general, we want to make a new-developer have a positive and seamless experience with this tutorial.

@zhangnju please spend some time to make the required fixes. DM me for more info. Thank you!

[adamlam2-amd]

We will also need Windows specific content, if any. I suspect the git commands will differ slightly, as well as the pictures may differ as well.

[zhangnju]

HI @adamlam2-amd Thanks for your review.

1. for docker, I have

This playbook is still problematic. There are 4 major questions that need to be resolved, among other more minor things.

1. Are we recommending to use Docker? @danielholanda need your guidance, as the HaloBox comes pre-installed with Pytorch already. But if we open it up to other AMD hardware, what is the strategy here? I recommend not using Docker at all and installing from source. This includes removing the 'Optional' bits as they can confuse users.
2. Do we need to recommend bits and bytes? I feel this will complicate the overall process. We can mention it in the next steps instead.
3. Must we use the CLI tool? LlaMA Factory has a great UI that I think will be a much better user experience. We can add pictures as well. I recommend not using the CLI in favour of the GUI.
4. What model and fine-tuning method is recommended? Let's have 1 for each so the user can understand, and then we can add more in the 'next steps' section.

As a reference, this LlaMA Factory guide is more helpful: https://www.datacamp.com/tutorial/llama-factory-web-ui-guide-fine-tuning-llms. In general, we want to make a new-developer have a positive and seamless experience with this tutorial.

@zhangnju please spend some time to make the required fixes. DM me for more info. Thank you!

for item 1, if Halo linux also has pytorch installed , I can remove the optional pytorch docker sections @danielholanda could you help confirm it?
for item 2, bisandbytes is an important library for developer, and many developers are interested in it. that is why I introduced how to install it from source codes, otherwise developer may meet some issues on Radeon
for item 3, I have talked with some developers, and they still prefer cli tools instead of UI tool. NV DGX spark playbook also introduced cli tool for llama factory, not UI tool. I have added UI into next steps sections
for item 4, I used qwen3 model, and mainly introduced lora finetuning. since some developers are interested in qlora, I also give a sample command for qlora for their reference. if you also think qlora is not needed, I can remove it @danielholanda

[adamlam2-amd]

Hey,

regarding 2), bits and bytes doesnt work on Windows (I think?) Please confirm if it does.

regarding 3), we have 2 other playbooks that teach users how to do LLM Fine-tuning - Unsloth and Pytorch. Let's use the GUI for LlamaFactory to appeal to the no-code quick-start developer community.

regarding 4), we also have qlora and lora explained in another playbook, which is why I mentioned.

[zhangnju]

Hey,

regarding 2), bits and bytes doesnt work on Windows (I think?) Please confirm if it does.

regarding 3), we have 2 other playbooks that teach users how to do LLM Fine-tuning - Unsloth and Pytorch. Let's use the GUI for LlamaFactory to appeal to the no-code quick-start developer community.

regarding 4), we also have qlora and lora explained in another playbook, which is why I mentioned.

HI @adamlam2-amd

I have removed the bitsandbytes and docker setup sections in the playbook.

some developers prefer to use unsloth as finetuning tool, and some developers prefer to use llama factory. so, even if we have unlosth and pytorch playbook, we still need to have a llama factory playbook, which can tell them AMD device can also support llama factory and you can have a try.

[danielholanda]

@adamlam2-amd Please let us know if there is any additional requirements here before we merge this

[danielholanda]

@iswaryaalex Please remember to take a look and review this.

[iswaryaalex]

Overall clean playbooks, I like logical flow of install → finetune → test/export

Key changes to consider

• Dependencies: This is critical for playbook. As we are not using docker, I highly encourage to add references to prerequisites that are already pre-installed in the section Dependencies
• For additioanl Dependencies introduced in llama-factory finetuning, clarify it in Additional Dependencies
• Typos in README, need to get fixed

[danielholanda]

@zhangnju Please ping the reviewers here once this is ready for another round of reviews

[zhangnju]

@zhangnju Please ping the reviewers here once this is ready for another round of reviews

sure. I have updated the playbook according to the latest feedback. @danielholanda @iswaryaalex @adamlam2-amd

[danielholanda]

@iswaryaalex @adamlam2-amd Can you please take another look?

[iswaryaalex]

Looks good to me! Changes in playbook.json needed to pass the checks

[iswaryaalex]

LGTM!

[zhangnju]

Looks good to me! Changes in playbook.json needed to pass the checks

I have updated playbook.json

[adamlam2-amd]

I made some changes myself to improve general UI and text formatting/grammar. Please review if you wish. Otherwise, looks good.

Approving so it can pass into QA.