Update README.md
Browse files
README.md
CHANGED
@@ -21,7 +21,7 @@ We explore supervised multitask pre-training by proposing ***Instruction Pre-Tra
|
|
21 |
**************************** **Updates** ****************************
|
22 |
* 2024/8/29: Updated [guidelines](https://huggingface.co/instruction-pretrain/finance-Llama3-8B) on evaluating any 🤗Huggingface models on the domain-specific tasks
|
23 |
* 2024/7/31: Updated pre-training suggestions in the `Advanced Usage` section of [instruction-synthesizer](https://huggingface.co/instruction-pretrain/instruction-synthesizer)
|
24 |
-
* 2024/7/15: We scaled up the pre-trained tokens from 100B to 250B, with the number of synthesized instruction-response pairs reaching 500M.
|
25 |
<p align='left'>
|
26 |
<img src="https://cdn-uploads.huggingface.co/production/uploads/66711d2ee12fa6cc5f5dfc89/0okCfRkC6uALTfuNxt0Fa.png" width="500">
|
27 |
</p>
|
@@ -83,7 +83,7 @@ You can use the following script to reproduce our results and evaluate any other
|
|
83 |
|
84 |
2). Evaluate the Model
|
85 |
```bash
|
86 |
-
# Select the domain from ['biomedicine', 'finance'
|
87 |
DOMAIN='finance'
|
88 |
|
89 |
# Specify any Huggingface LM name (Not applicable to models requiring specific prompt templates)
|
|
|
21 |
**************************** **Updates** ****************************
|
22 |
* 2024/8/29: Updated [guidelines](https://huggingface.co/instruction-pretrain/finance-Llama3-8B) on evaluating any 🤗Huggingface models on the domain-specific tasks
|
23 |
* 2024/7/31: Updated pre-training suggestions in the `Advanced Usage` section of [instruction-synthesizer](https://huggingface.co/instruction-pretrain/instruction-synthesizer)
|
24 |
+
* 2024/7/15: We scaled up the pre-trained tokens from 100B to 250B, with the number of synthesized instruction-response pairs reaching 500M. The performance trend on downstream tasks throughout the pre-training process:
|
25 |
<p align='left'>
|
26 |
<img src="https://cdn-uploads.huggingface.co/production/uploads/66711d2ee12fa6cc5f5dfc89/0okCfRkC6uALTfuNxt0Fa.png" width="500">
|
27 |
</p>
|
|
|
83 |
|
84 |
2). Evaluate the Model
|
85 |
```bash
|
86 |
+
# Select the domain from ['biomedicine', 'finance']
|
87 |
DOMAIN='finance'
|
88 |
|
89 |
# Specify any Huggingface LM name (Not applicable to models requiring specific prompt templates)
|