Skip to content

[LLM Tutorial] Restructure codes #3850

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 9 commits into
base: main
Choose a base branch
from
Draft

[LLM Tutorial] Restructure codes #3850

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
ad3aa6c
restructure custom code
holgerroth Nov 19, 2025
15b19da
use job.py
holgerroth Nov 19, 2025
2dd770f
Apply suggestion from @greptile-apps[bot]
holgerroth Nov 19, 2025
4773852
Apply suggestion from @greptile-apps[bot]
holgerroth Nov 19, 2025
5cf73c2
update notebooks
holgerroth Nov 19, 2025
f5b4afd
fix script path
holgerroth Nov 19, 2025
600a775
Merge branch 'llm_tutorial_update' of github.com:holgerroth/NVFlare i…
holgerroth Nov 19, 2025
6ec7706
fix wrong files
holgerroth Nov 19, 2025
989f7a2
update bert example
holgerroth Nov 19, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
0 ...training/08.1_fed_bert/code/src/nlp_fl.py → ...ated_LLM_training/08.1_fed_bert/client.py
View file
Open in desktop
File renamed without changes.
4 changes: 0 additions & 4 deletions ...ed_federated_learning/chapter-8_federated_LLM_training/08.1_fed_bert/code/prepare_data.sh
View file
Open in desktop

This file was deleted.

0 ...g/08.1_fed_bert/code/src/data_sequence.py → ...M_training/08.1_fed_bert/data_sequence.py
View file
Open in desktop
File renamed without changes.
0 ...ng/08.1_fed_bert/code/utils/data_split.py → ..._LLM_training/08.1_fed_bert/data_split.py
View file
Open in desktop
File renamed without changes.
26 changes: 14 additions & 12 deletions ...ted_learning/chapter-8_federated_LLM_training/08.1_fed_bert/federated_nlp_with_bert.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,17 +23,21 @@
"metadata": {},
"source": [
"## Setup\n",
"Install required packages for training"
"Install required packages for training\n",
"\n",
"> **Note**: This example requires PyTorch 2.2.0 or later for compatibility with the latest Transformers library."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "ee1d1f87-502a-4c30-aa40-f55ae65a1da7",
"metadata": {},
"metadata": {
"scrolled": true
},
"outputs": [],
"source": [
"%pip install -r code/requirements.txt"
"%pip install -r requirements.txt"
]
},
{
Expand Down Expand Up @@ -79,7 +83,7 @@
"metadata": {},
"outputs": [],
"source": [
"! code/prepare_data.sh /tmp/nvflare/dataset/nlp_ner"
"! prepare_data.sh /tmp/nvflare/dataset/nlp_ner"
]
},
{
Expand Down Expand Up @@ -119,12 +123,12 @@
"cell_type": "code",
"execution_count": null,
"id": "f48f1d5f-e656-4f71-b925-94035c60ace0",
"metadata": {},
"metadata": {
"scrolled": true
},
"outputs": [],
"source": [
"%cd code\n",
"! python nlp_fl_job.py --model_name Bert\n",
"%cd .."
"! python job.py --model_name Bert"
]
},
{
Expand Down Expand Up @@ -169,9 +173,7 @@
"metadata": {},
"outputs": [],
"source": [
"%cd code\n",
"! sh test_global_model.sh /tmp/nvflare/dataset/nlp_ner\n",
"%cd .."
"! sh test_global_model.sh /tmp/nvflare/dataset/nlp_ner"
]
},
{
Expand Down Expand Up @@ -226,7 +228,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.0"
"version": "3.10.12"
}
},
"nbformat": 4,
Expand Down
4 changes: 2 additions & 2 deletions ...training/08.1_fed_bert/code/nlp_fl_job.py → ...derated_LLM_training/08.1_fed_bert/job.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@

import argparse

from src.nlp_models import BertModel, GPTModel
from nlp_models import BertModel, GPTModel

from nvflare.app_common.widgets.intime_model_selector import IntimeModelSelector
from nvflare.app_common.workflows.fedavg import FedAvg
Expand Down Expand Up @@ -57,7 +57,7 @@ def main():
# Local training parameters
num_rounds = 5
dataset_path = f"/tmp/nvflare/dataset/nlp_ner/{num_clients}_split"
train_script = "src/nlp_fl.py"
train_script = "client.py"
train_args = f"--dataset_path {dataset_path} --model_name {train_model_name}"

# Define the controller workflow and send to server
Expand Down
4 changes: 2 additions & 2 deletions ...ning/08.1_fed_bert/code/ner_model_test.py → ..._training/08.1_fed_bert/ner_model_test.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,8 @@
import pandas as pd
import torch
from seqeval.metrics import classification_report
from src.data_sequence import DataSequence
from src.nlp_models import BertModel, GPTModel
from data_sequence import DataSequence
from nlp_models import BertModel, GPTModel
from torch.utils.data import DataLoader

os.environ["TOKENIZERS_PARALLELISM"] = "False"
Expand Down
0 ...ning/08.1_fed_bert/code/src/nlp_models.py → ..._LLM_training/08.1_fed_bert/nlp_models.py
View file
Open in desktop
File renamed without changes.
4 changes: 4 additions & 0 deletions ...dvanced_federated_learning/chapter-8_federated_LLM_training/08.1_fed_bert/prepare_data.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
#!/usr/bin/env bash
DATASET_ROOT=${1}
echo "4-client"
python3 data_split.py --data_path ${DATASET_ROOT} --num_clients 4 --random_seed 0 --site_name_prefix 'site-'
4 changes: 2 additions & 2 deletions ...ining/08.1_fed_bert/code/requirements.txt → ...M_training/08.1_fed_bert/requirements.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
torch
torch>=2.2.0
torchvision
tensorboard
transformers
transformers>=4.40.0
pandas
seqeval
0 ...g/08.1_fed_bert/code/test_global_model.sh → ...aining/08.1_fed_bert/test_global_model.sh
View file
Open in desktop
File renamed without changes.
2 changes: 1 addition & 1 deletion ...4_advanced_federated_learning/chapter-8_federated_LLM_training/08.2_llm_sft/LLM_SFT.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -168,7 +168,7 @@
"metadata": {},
"outputs": [],
"source": [
"! python sft_job.py --client_ids dolly --data_path /tmp/nvflare/dataset/llm/ --workspace_dir /tmp/nvflare/workspace/llm/dolly_fl_sft --job_dir /tmp/nvflare/workspace/jobs/llm_hf_sft --train_mode SFT "
"! python job.py --client_ids dolly --data_path /tmp/nvflare/dataset/llm/ --workspace_dir /tmp/nvflare/workspace/llm/dolly_fl_sft --job_dir /tmp/nvflare/workspace/jobs/llm_hf_sft --train_mode SFT "
]
},
{
Expand Down
0 ...aining/08.2_llm_sft/src/hf_sft_peft_fl.py → ...rated_LLM_training/08.2_llm_sft/client.py
View file
Open in desktop
File renamed without changes.
0 ...training/08.2_llm_sft/src/hf_sft_model.py → ...LLM_training/08.2_llm_sft/hf_sft_model.py
View file
Open in desktop
File renamed without changes.
8 changes: 4 additions & 4 deletions ...ated_LLM_training/08.2_llm_sft/sft_job.py → ...ederated_LLM_training/08.2_llm_sft/job.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@

def main():
args = define_parser()
train_script = "src/hf_sft_peft_fl.py"
train_script = "client.py"
client_ids = args.client_ids
num_clients = len(client_ids)

Expand Down Expand Up @@ -74,9 +74,9 @@ def main():

# Define the model persistor and send to server
# First send the model to the server
job.to("src/hf_sft_model.py", "server")
job.to("hf_sft_model.py", "server")
# Then send the model persistor to the server
model_args = {"path": "src.hf_sft_model.CausalLMModel", "args": {"model_name_or_path": model_name_or_path}}
model_args = {"path": "hf_sft_model.CausalLMModel", "args": {"model_name_or_path": model_name_or_path}}
job.to(PTFileModelPersistor(model=model_args), "server", id="persistor")

# Add model selection widget and send to server
Expand Down Expand Up @@ -126,7 +126,7 @@ def define_parser():
nargs="+",
type=str,
default="",
help="Clinet IDs, used to get the data path for each client",
help="Client IDs, used to get the data path for each client",
)
parser.add_argument(
"--num_rounds",
Expand Down
2 changes: 1 addition & 1 deletion ...advanced_federated_learning/chapter-8_federated_LLM_training/08.3_llm_peft/LLM_PEFT.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,7 @@
"metadata": {},
"outputs": [],
"source": [
"! python peft_job.py --client_ids dolly oasst1 --data_path /tmp/nvflare/dataset/llm/ --workspace_dir /tmp/nvflare/workspace/llm/all_fl_peft --job_dir /tmp/nvflare/workspace/jobs/llm_fl_peft --train_mode PEFT --threads 2 "
"! python job.py --client_ids dolly oasst1 --data_path /tmp/nvflare/dataset/llm/ --workspace_dir /tmp/nvflare/workspace/llm/all_fl_peft --job_dir /tmp/nvflare/workspace/jobs/llm_fl_peft --train_mode PEFT --threads 2 "
]
},
{
Expand Down
0 ...ining/08.3_llm_peft/src/hf_sft_peft_fl.py → ...ated_LLM_training/08.3_llm_peft/client.py
View file
Open in desktop
File renamed without changes.
0 ...aining/08.3_llm_peft/src/hf_peft_model.py → ...M_training/08.3_llm_peft/hf_peft_model.py
View file
Open in desktop
File renamed without changes.
8 changes: 4 additions & 4 deletions ...ed_LLM_training/08.3_llm_peft/peft_job.py → ...derated_LLM_training/08.3_llm_peft/job.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@

def main():
args = define_parser()
train_script = "src/hf_sft_peft_fl.py"
train_script = "client.py"
client_ids = args.client_ids
num_clients = len(client_ids)

Expand Down Expand Up @@ -74,9 +74,9 @@ def main():

# Define the model persistor and send to server
# First send the model to the server
job.to("src/hf_peft_model.py", "server")
job.to("hf_peft_model.py", "server")
# Then send the model persistor to the server
model_args = {"path": "src.hf_peft_model.CausalLMPEFTModel", "args": {"model_name_or_path": model_name_or_path}}
model_args = {"path": "hf_peft_model.CausalLMPEFTModel", "args": {"model_name_or_path": model_name_or_path}}
job.to(PTFileModelPersistor(model=model_args), "server", id="persistor")

# Add model selection widget and send to server
Expand Down Expand Up @@ -126,7 +126,7 @@ def define_parser():
nargs="+",
type=str,
default="",
help="Clinet IDs, used to get the data path for each client",
help="Client IDs, used to get the data path for each client",
)
parser.add_argument(
"--num_rounds",
Expand Down
12 changes: 6 additions & 6 deletions ...ed_learning/chapter-8_federated_LLM_training/08.4_llm_quantization/LLM_quantization.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,10 +47,10 @@
"metadata": {},
"outputs": [],
"source": [
"! python sft_job.py --client_ids dolly --data_path /tmp/nvflare/dataset/llm/ --workspace_dir /tmp/nvflare/workspace/llm/dolly_fl_sft_16 --job_dir /tmp/nvflare/workspace/jobs/llm_hf_sft_16 --train_mode SFT --quantize_mode float16\n",
"! python sft_job.py --client_ids dolly --data_path /tmp/nvflare/dataset/llm/ --workspace_dir /tmp/nvflare/workspace/llm/dolly_fl_sft_8 --job_dir /tmp/nvflare/workspace/jobs/llm_hf_sft_8 --train_mode SFT --quantize_mode blockwise8\n",
"! python sft_job.py --client_ids dolly --data_path /tmp/nvflare/dataset/llm/ --workspace_dir /tmp/nvflare/workspace/llm/dolly_fl_sft_fp4 --job_dir /tmp/nvflare/workspace/jobs/llm_hf_sft_fp4 --train_mode SFT --quantize_mode float4\n",
"! python sft_job.py --client_ids dolly --data_path /tmp/nvflare/dataset/llm/ --workspace_dir /tmp/nvflare/workspace/llm/dolly_fl_sft_nf4 --job_dir /tmp/nvflare/workspace/jobs/llm_hf_sft_nf4 --train_mode SFT --quantize_mode normfloat4"
"! python job.py --client_ids dolly --data_path /tmp/nvflare/dataset/llm/ --workspace_dir /tmp/nvflare/workspace/llm/dolly_fl_sft_16 --job_dir /tmp/nvflare/workspace/jobs/llm_hf_sft_16 --train_mode SFT --quantize_mode float16\n",
"! python job.py --client_ids dolly --data_path /tmp/nvflare/dataset/llm/ --workspace_dir /tmp/nvflare/workspace/llm/dolly_fl_sft_8 --job_dir /tmp/nvflare/workspace/jobs/llm_hf_sft_8 --train_mode SFT --quantize_mode blockwise8\n",
"! python job.py --client_ids dolly --data_path /tmp/nvflare/dataset/llm/ --workspace_dir /tmp/nvflare/workspace/llm/dolly_fl_sft_fp4 --job_dir /tmp/nvflare/workspace/jobs/llm_hf_sft_fp4 --train_mode SFT --quantize_mode float4\n",
"! python job.py --client_ids dolly --data_path /tmp/nvflare/dataset/llm/ --workspace_dir /tmp/nvflare/workspace/llm/dolly_fl_sft_nf4 --job_dir /tmp/nvflare/workspace/jobs/llm_hf_sft_nf4 --train_mode SFT --quantize_mode normfloat4"
]
},
{
Expand Down Expand Up @@ -81,8 +81,8 @@
"metadata": {},
"outputs": [],
"source": [
"! python sft_job.py --client_ids dolly --data_path /tmp/nvflare/dataset/llm/ --workspace_dir /tmp/nvflare/workspace/llm/dolly_fl_sft_tensor --job_dir /tmp/nvflare/workspace/jobs/llm_hf_sft_tensor --train_mode SFT --message_mode tensor\n",
"! python sft_job.py --client_ids dolly --data_path /tmp/nvflare/dataset/llm/ --workspace_dir /tmp/nvflare/workspace/llm/dolly_fl_sft_tensor_fp4 --job_dir /tmp/nvflare/workspace/jobs/llm_hf_sft_tensor_fp4 --train_mode SFT --message_mode tensor --quantize_mode float4"
"! python job.py --client_ids dolly --data_path /tmp/nvflare/dataset/llm/ --workspace_dir /tmp/nvflare/workspace/llm/dolly_fl_sft_tensor --job_dir /tmp/nvflare/workspace/jobs/llm_hf_sft_tensor --train_mode SFT --message_mode tensor\n",
"! python job.py --client_ids dolly --data_path /tmp/nvflare/dataset/llm/ --workspace_dir /tmp/nvflare/workspace/llm/dolly_fl_sft_tensor_fp4 --job_dir /tmp/nvflare/workspace/jobs/llm_hf_sft_tensor_fp4 --train_mode SFT --message_mode tensor --quantize_mode float4"
]
},
{
Expand Down
0 ....4_llm_quantization/src/hf_sft_peft_fl.py → ..._training/08.4_llm_quantization/client.py
View file
Open in desktop
File renamed without changes.
0 ...08.4_llm_quantization/src/hf_sft_model.py → ...ing/08.4_llm_quantization/hf_sft_model.py
View file
Open in desktop
File renamed without changes.
6 changes: 3 additions & 3 deletions ...training/08.4_llm_quantization/sft_job.py → ...LLM_training/08.4_llm_quantization/job.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@

def main():
args = define_parser()
train_script = "src/hf_sft_peft_fl.py"
train_script = "client.py"
client_ids = args.client_ids
num_clients = len(client_ids)

Expand Down Expand Up @@ -74,9 +74,9 @@ def main():

# Define the model persistor and send to server
# First send the model to the server
job.to("src/hf_sft_model.py", "server")
job.to("hf_sft_model.py", "server")
# Then send the model persistor to the server
model_args = {"path": "src.hf_sft_model.CausalLMModel", "args": {"model_name_or_path": model_name_or_path}}
model_args = {"path": "hf_sft_model.CausalLMModel", "args": {"model_name_or_path": model_name_or_path}}
job.to(PTFileModelPersistor(model=model_args), "server", id="persistor")

# Add model selection widget and send to server
Expand Down
2 changes: 1 addition & 1 deletion ...ederated_learning/chapter-8_federated_LLM_training/08.5_llm_streaming/container_stream.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
bash utils/log_memory.sh >>/tmp/nvflare/logs/container.txt &
python streaming_job.py --retriever_mode container
python job.py --retriever_mode container
2 changes: 1 addition & 1 deletion ...ced_federated_learning/chapter-8_federated_LLM_training/08.5_llm_streaming/file_stream.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
bash utils/log_memory.sh >>/tmp/nvflare/logs/file.txt &
python streaming_job.py --retriever_mode file
python job.py --retriever_mode file
0 ...ining/08.5_llm_streaming/streaming_job.py → ...ed_LLM_training/08.5_llm_streaming/job.py
View file
Open in desktop
File renamed without changes.
2 changes: 1 addition & 1 deletion ...ated_learning/chapter-8_federated_LLM_training/08.5_llm_streaming/regular_transmission.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
mkdir /tmp/nvflare/logs/
bash utils/log_memory.sh >>/tmp/nvflare/logs/regular.txt &
python streaming_job.py
python job.py
0 ...llm_streaming/src/streaming_controller.py → ...8.5_llm_streaming/streaming_controller.py
View file
Open in desktop
File renamed without changes.
0 ...5_llm_streaming/src/streaming_executor.py → .../08.5_llm_streaming/streaming_executor.py
View file
Open in desktop
File renamed without changes.
Loading