Skip to content

Commit 6269c15

Browse files
committed
fine-tuning README update with latest result
1 parent cc93b73 commit 6269c15

1 file changed

Lines changed: 9 additions & 3 deletions

File tree

  • end-to-end-use-cases/coding/text2sql/fine-tuning

end-to-end-use-cases/coding/text2sql/fine-tuning/README.md

Lines changed: 9 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -10,19 +10,25 @@ This folder contains scripts to:
1010

1111
## Eval Results of the Fine-tuned Models
1212

13-
The eval results of SFT Llama 3.1 8B with different options (epochs is 3) are summarized in the table below:
13+
The eval results of SFT Llama 3.1 8B with different options (epochs is 3) are summarized below:
1414

1515
| Fine-tuning Combination | Accuracy |
1616
|-----------------------------|----------|
1717
| Non-Quantized, CoT, PEFT | 43.35% |
1818
| Quantized, CoT, PEFT | 42.89% |
19+
| Non-Quantized, CoT, FFT | 42.44% |
1920
| Non-Quantized, No CoT, PEFT | 39.31% |
2021
| Quantized, No CoT, PEFT | 39.31% |
21-
| Non-Quantized, CoT, FFT | 38.46% |
22-
| Non-Quantized, No CoT, FFT | 33.70% |
22+
| Non-Quantized, No CoT, FFT | 36.31% |
2323
| Quantized, CoT, FFT | N/A |
2424
| Quantized, No CoT, FFT | N/A |
2525

26+
The table above shows that:
27+
28+
1. The CoT FFT/PEFT model (with or without quantization) outperforms the no CoT FFT/PEFT model (with or without quantization) by 3.5% to 6.1%.
29+
30+
2. The non-quantized PEFT model (CoT or not) is slightly better than the non-quantized FFT model.
31+
2632
## SFT with the BIRD TRAIN dataset (No Reasoning)
2733

2834
We'll first use the BIRD TRAIN dataset to prepare for supervised fine-tuning with no reasoning info in the dataset.

0 commit comments

Comments
 (0)