Update README.md
Browse files
README.md
CHANGED
|
@@ -32,7 +32,7 @@ We evaluate our models on three text-to-SQL benchmarks: Spider, Bird, and text2s
|
|
| 32 |
|
| 33 |
## Training Procedure
|
| 34 |
|
| 35 |
-
|
| 36 |
|
| 37 |
### Hyperparameters
|
| 38 |
|
|
|
|
| 32 |
|
| 33 |
## Training Procedure
|
| 34 |
|
| 35 |
+
SambaCoder-nsql-llama-2-70b was trained using cross-entropy loss to maximize the likelihood of sequential inputs. For finetuning on text-to-SQL pairs, we only compute the loss over the SQL portion of the pair. The model is trained using SambaNova's in-house Reconfigurable Dataflow Unit (RDU), leveraging data and model parallelism. We pre-trained for 2 epochs and fine-tuned for 10 epochs.
|
| 36 |
|
| 37 |
### Hyperparameters
|
| 38 |
|