Skip to content

Commit 27a23af

Browse files
committed
main README
1 parent 82bb008 commit 27a23af

1 file changed

Lines changed: 8 additions & 6 deletions

File tree

  • end-to-end-use-cases/coding/text2sql

end-to-end-use-cases/coding/text2sql/README.md

Lines changed: 8 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,8 @@
33
This recipe is step by step guide to improve Llama performance on Text2SQL measured with the popular [BIRD](https://bird-bench.github.io) benchmark. We generate a synthetic Chain of Thought(CoT) dataset and fine-tune Llama models on it.
44

55
Results:
6+
7+
| Fine-tuning Combination | Accuracy |
68
|-----------------------------|-------------------------------|
79
| baseline | 39.47% |
810
| CoT, PEFT | 43.35% |
@@ -11,20 +13,20 @@ Results:
1113

1214
The complete steps are:
1315

14-
1. Pre-processing the BIRD TRAIN datset by converting SQL statements into the conversation format.
16+
1. Pre-processing the [BIRD](https://bird-bench.github.io) TRAIN datset by converting text, schema, external knowledge, and SQL statements into the conversation format.
1517

16-
2. We use the conversations from step 1, add CoT to these existing conversations using Llama-3.3-70B.
18+
2. Using Llama-3.3-70B to add CoT to the conversation format dataset.
1719

18-
3. Fine-tuning Llama-3.1-8B on the dataset from step 2.
20+
3. Fine-tuning Llama-3.1-8B on the CoT dataset from step 2.
1921

20-
4. We provide scripts to simplify running the [BIRD](https://bird-bench.github.io) eval benchmark on the fine-tuned models and compare it with out of the model.
22+
4. Running the BIRD DEV eval benchmark on the fine-tuned models and compare it with out of the model.
2123

2224
## Folder Structure
2325

2426
- quickstart folder: contains a notebook to ask Llama 3.3 to convert natural language queries into SQL queries.
2527
- data folder: contains scripts to download the BIRD TRAIN and DEV datasets;
26-
- fine-tune folder: contains scripts to generate non-CoT and CoT datasets based on the BIRD TRAIN set and to supervised fine-tune Llama models using the datasets, with different SFT options (quantization or not, full fine-tuning or parameter-efficient fine-tuning);
27-
- eval folder: contains scripts to evaluate Llama models (original and fine-tuned) on the BIRD dataset;
28+
- fine-tune folder: contains scripts to generate CoT dataset based on the BIRD TRAIN set and to supervised fine-tune Llama models using the dataset, with different SFT options (quantization or not, full fine-tuning or parameter-efficient fine-tuning);
29+
- eval folder: contains scripts to evaluate Llama models (original and fine-tuned) on the BIRD dataset.
2830

2931
We also experimented with supervised fine-tuning (SFT) without CoT which resulted in slightly lower accuracy.
3032

0 commit comments

Comments
 (0)