Text Generation
Transformers
Safetensors
English
llama
finance
text-generation-inference
instruction-pretrain commited on
Commit
d4f8703
·
verified ·
1 Parent(s): 7639c51

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +3 -2
README.md CHANGED
@@ -19,8 +19,9 @@ We explore supervised multitask pre-training by proposing ***Instruction Pre-Tra
19
  </p>
20
 
21
  **************************** **Updates** ****************************
 
22
  * 2024/7/31: Updated pre-training suggestions in the `Advanced Usage` section of [instruction-synthesizer](https://huggingface.co/instruction-pretrain/instruction-synthesizer)
23
- * 2024/7/15: We scaled up the pre-trained tokens from 100B to 250B, with the number of synthesized instruction-response pairs reaching 500M! Below, we show the performance trend on downstream tasks throughout the pre-training process:
24
  <p align='left'>
25
  <img src="https://cdn-uploads.huggingface.co/production/uploads/66711d2ee12fa6cc5f5dfc89/0okCfRkC6uALTfuNxt0Fa.png" width="500">
26
  </p>
@@ -71,7 +72,7 @@ print(pred)
71
  ```
72
 
73
  ### 2. evaluate any Huggingface LMs on domain-dpecific tasks (💡New!)
74
- You can use the following scripts to reproduce our results and evaluate any other Huggingface models on domain-specific tasks. Note that these scripts are not applicable to models that require specific prompt templates (e.g., Llama2-chat, Llama3-Instruct).
75
 
76
  1). Set Up Dependencies
77
  ```bash
 
19
  </p>
20
 
21
  **************************** **Updates** ****************************
22
+ * 2024/8/29: Updated [guidelines](https://huggingface.co/instruction-pretrain/finance-Llama3-8B) on evaluating any 🤗Huggingface models on the domain-specific tasks
23
  * 2024/7/31: Updated pre-training suggestions in the `Advanced Usage` section of [instruction-synthesizer](https://huggingface.co/instruction-pretrain/instruction-synthesizer)
24
+ * 2024/7/15: We scaled up the pre-trained tokens from 100B to 250B, with the number of synthesized instruction-response pairs reaching 500M. Below, we show the performance trend on downstream tasks throughout the pre-training process:
25
  <p align='left'>
26
  <img src="https://cdn-uploads.huggingface.co/production/uploads/66711d2ee12fa6cc5f5dfc89/0okCfRkC6uALTfuNxt0Fa.png" width="500">
27
  </p>
 
72
  ```
73
 
74
  ### 2. evaluate any Huggingface LMs on domain-dpecific tasks (💡New!)
75
+ You can use the following script to reproduce our results and evaluate any other Huggingface models on domain-specific tasks. Note that the script is NOT applicable to models that require specific prompt templates (e.g., Llama2-chat, Llama3-Instruct).
76
 
77
  1). Set Up Dependencies
78
  ```bash