@@ -10,19 +10,25 @@ This folder contains scripts to:
1010
1111## Eval Results of the Fine-tuned Models
1212
13- The eval results of SFT Llama 3.1 8B with different options (epochs is 3) are summarized in the table below:
13+ The eval results of SFT Llama 3.1 8B with different options (epochs is 3) are summarized below:
1414
1515| Fine-tuning Combination | Accuracy |
1616| -----------------------------| ----------|
1717| Non-Quantized, CoT, PEFT | 43.35% |
1818| Quantized, CoT, PEFT | 42.89% |
19+ | Non-Quantized, CoT, FFT | 42.44% |
1920| Non-Quantized, No CoT, PEFT | 39.31% |
2021| Quantized, No CoT, PEFT | 39.31% |
21- | Non-Quantized, CoT, FFT | 38.46% |
22- | Non-Quantized, No CoT, FFT | 33.70% |
22+ | Non-Quantized, No CoT, FFT | 36.31% |
2323| Quantized, CoT, FFT | N/A |
2424| Quantized, No CoT, FFT | N/A |
2525
26+ The table above shows that:
27+
28+ 1 . The CoT FFT/PEFT model (with or without quantization) outperforms the no CoT FFT/PEFT model (with or without quantization) by 3.5% to 6.1%.
29+
30+ 2 . The non-quantized PEFT model (CoT or not) is slightly better than the non-quantized FFT model.
31+
2632## SFT with the BIRD TRAIN dataset (No Reasoning)
2733
2834We'll first use the BIRD TRAIN dataset to prepare for supervised fine-tuning with no reasoning info in the dataset.
0 commit comments