You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: end-to-end-use-cases/coding/text2sql/README.md
+8-6Lines changed: 8 additions & 6 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -3,6 +3,8 @@
3
3
This recipe is step by step guide to improve Llama performance on Text2SQL measured with the popular [BIRD](https://bird-bench.github.io) benchmark. We generate a synthetic Chain of Thought(CoT) dataset and fine-tune Llama models on it.
1. Pre-processing the BIRD TRAIN datset by converting SQL statements into the conversation format.
16
+
1. Pre-processing the [BIRD](https://bird-bench.github.io) TRAIN datset by converting text, schema, external knowledge, and SQL statements into the conversation format.
15
17
16
-
2.We use the conversations from step 1, add CoT to these existing conversations using Llama-3.3-70B.
18
+
2.Using Llama-3.3-70B to add CoT to the conversation format dataset.
17
19
18
-
3. Fine-tuning Llama-3.1-8B on the dataset from step 2.
20
+
3. Fine-tuning Llama-3.1-8B on the CoT dataset from step 2.
19
21
20
-
4.We provide scripts to simplify running the [BIRD](https://bird-bench.github.io) eval benchmark on the fine-tuned models and compare it with out of the model.
22
+
4.Running the BIRD DEV eval benchmark on the fine-tuned models and compare it with out of the model.
21
23
22
24
## Folder Structure
23
25
24
26
- quickstart folder: contains a notebook to ask Llama 3.3 to convert natural language queries into SQL queries.
25
27
- data folder: contains scripts to download the BIRD TRAIN and DEV datasets;
26
-
- fine-tune folder: contains scripts to generate non-CoT and CoT datasets based on the BIRD TRAIN set and to supervised fine-tune Llama models using the datasets, with different SFT options (quantization or not, full fine-tuning or parameter-efficient fine-tuning);
27
-
- eval folder: contains scripts to evaluate Llama models (original and fine-tuned) on the BIRD dataset;
28
+
- fine-tune folder: contains scripts to generate CoT dataset based on the BIRD TRAIN set and to supervised fine-tune Llama models using the dataset, with different SFT options (quantization or not, full fine-tuning or parameter-efficient fine-tuning);
29
+
- eval folder: contains scripts to evaluate Llama models (original and fine-tuned) on the BIRD dataset.
28
30
29
31
We also experimented with supervised fine-tuning (SFT) without CoT which resulted in slightly lower accuracy.
0 commit comments