Update README.md
Browse files
README.md
CHANGED
@@ -19,8 +19,9 @@ We explore supervised multitask pre-training by proposing ***Instruction Pre-Tra
|
|
19 |
</p>
|
20 |
|
21 |
**************************** **Updates** ****************************
|
|
|
22 |
* 2024/7/31: Updated pre-training suggestions in the `Advanced Usage` section of [instruction-synthesizer](https://huggingface.co/instruction-pretrain/instruction-synthesizer)
|
23 |
-
* 2024/7/15: We scaled up the pre-trained tokens from 100B to 250B, with the number of synthesized instruction-response pairs reaching 500M
|
24 |
<p align='left'>
|
25 |
<img src="https://cdn-uploads.huggingface.co/production/uploads/66711d2ee12fa6cc5f5dfc89/0okCfRkC6uALTfuNxt0Fa.png" width="500">
|
26 |
</p>
|
@@ -71,7 +72,7 @@ print(pred)
|
|
71 |
```
|
72 |
|
73 |
### 2. evaluate any Huggingface LMs on domain-dpecific tasks (💡New!)
|
74 |
-
You can use the following
|
75 |
|
76 |
1). Set Up Dependencies
|
77 |
```bash
|
|
|
19 |
</p>
|
20 |
|
21 |
**************************** **Updates** ****************************
|
22 |
+
* 2024/8/29: Updated [guidelines](https://huggingface.co/instruction-pretrain/finance-Llama3-8B) on evaluating any 🤗Huggingface models on the domain-specific tasks
|
23 |
* 2024/7/31: Updated pre-training suggestions in the `Advanced Usage` section of [instruction-synthesizer](https://huggingface.co/instruction-pretrain/instruction-synthesizer)
|
24 |
+
* 2024/7/15: We scaled up the pre-trained tokens from 100B to 250B, with the number of synthesized instruction-response pairs reaching 500M. Below, we show the performance trend on downstream tasks throughout the pre-training process:
|
25 |
<p align='left'>
|
26 |
<img src="https://cdn-uploads.huggingface.co/production/uploads/66711d2ee12fa6cc5f5dfc89/0okCfRkC6uALTfuNxt0Fa.png" width="500">
|
27 |
</p>
|
|
|
72 |
```
|
73 |
|
74 |
### 2. evaluate any Huggingface LMs on domain-dpecific tasks (💡New!)
|
75 |
+
You can use the following script to reproduce our results and evaluate any other Huggingface models on domain-specific tasks. Note that the script is NOT applicable to models that require specific prompt templates (e.g., Llama2-chat, Llama3-Instruct).
|
76 |
|
77 |
1). Set Up Dependencies
|
78 |
```bash
|