Text Generation
Transformers
Safetensors
English
llama
finance
text-generation-inference
instruction-pretrain commited on
Commit
fd7cbde
·
verified ·
1 Parent(s): d4f8703

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -2
README.md CHANGED
@@ -21,7 +21,7 @@ We explore supervised multitask pre-training by proposing ***Instruction Pre-Tra
21
  **************************** **Updates** ****************************
22
  * 2024/8/29: Updated [guidelines](https://huggingface.co/instruction-pretrain/finance-Llama3-8B) on evaluating any 🤗Huggingface models on the domain-specific tasks
23
  * 2024/7/31: Updated pre-training suggestions in the `Advanced Usage` section of [instruction-synthesizer](https://huggingface.co/instruction-pretrain/instruction-synthesizer)
24
- * 2024/7/15: We scaled up the pre-trained tokens from 100B to 250B, with the number of synthesized instruction-response pairs reaching 500M. Below, we show the performance trend on downstream tasks throughout the pre-training process:
25
  <p align='left'>
26
  <img src="https://cdn-uploads.huggingface.co/production/uploads/66711d2ee12fa6cc5f5dfc89/0okCfRkC6uALTfuNxt0Fa.png" width="500">
27
  </p>
@@ -83,7 +83,7 @@ You can use the following script to reproduce our results and evaluate any other
83
 
84
  2). Evaluate the Model
85
  ```bash
86
- # Select the domain from ['biomedicine', 'finance', 'law']
87
  DOMAIN='finance'
88
 
89
  # Specify any Huggingface LM name (Not applicable to models requiring specific prompt templates)
 
21
  **************************** **Updates** ****************************
22
  * 2024/8/29: Updated [guidelines](https://huggingface.co/instruction-pretrain/finance-Llama3-8B) on evaluating any 🤗Huggingface models on the domain-specific tasks
23
  * 2024/7/31: Updated pre-training suggestions in the `Advanced Usage` section of [instruction-synthesizer](https://huggingface.co/instruction-pretrain/instruction-synthesizer)
24
+ * 2024/7/15: We scaled up the pre-trained tokens from 100B to 250B, with the number of synthesized instruction-response pairs reaching 500M. The performance trend on downstream tasks throughout the pre-training process:
25
  <p align='left'>
26
  <img src="https://cdn-uploads.huggingface.co/production/uploads/66711d2ee12fa6cc5f5dfc89/0okCfRkC6uALTfuNxt0Fa.png" width="500">
27
  </p>
 
83
 
84
  2). Evaluate the Model
85
  ```bash
86
+ # Select the domain from ['biomedicine', 'finance']
87
  DOMAIN='finance'
88
 
89
  # Specify any Huggingface LM name (Not applicable to models requiring specific prompt templates)