Upload 9 files

README.md

@@ -36,7 +36,7 @@ inference:
 
 `Aira-Instruct-PT-560M` is a instruction-tuned GPT-style model based on [BLOOM](https://huggingface.co/bigscience/bloom-560m). The model was trained with a dataset composed of `prompt`, `completions`, generated via the [Self-Instruct](https://github.com/yizhongw/self-instruct) framework. `Aira-Instruct-PT-560M` instruction-tuning was achieved via conditional text generation.
 
-The dataset used to train this model combines two main sources of data: the [`synthetic-instruct-gptj-pairwise`](https://huggingface.co/datasets/Dahoas/synthetic-instruct-gptj-pairwise) dataset and a subset of [Aira's](https://github.com/Nkluge-correa/Aira-EXPERT) fine-tuning dataset focused on Ethics, AI, AI safety, and related topics. The dataset is available in both Portuguese and English.
+The dataset used to train this model combines the following sources of data: the [`synthetic-instruct-gptj-pairwise`](https://huggingface.co/datasets/Dahoas/synthetic-instruct-gptj-pairwise) dataset, the [`databricks_dolly_15k`](https://huggingface.co/datasets/HuggingFaceH4/databricks_dolly_15k) dataset, the [`instruction-dataset`](https://huggingface.co/datasets/HuggingFaceH4/instruction-dataset) dataset, and a subset of [Aira's](https://github.com/Nkluge-correa/Aira-EXPERT) fine-tuning dataset, focused on Q&A related to Ethics, AI, AI safety, and other related topics. The dataset is available in both Portuguese and English.
 
 Check our gradio-demo in [Spaces](https://huggingface.co/spaces/nicholasKluge/Aira-Demo-Portuguese).
 
@@ -50,16 +50,16 @@ Check our gradio-demo in [Spaces](https://huggingface.co/spaces/nicholasKluge/Ai
 - **Optimizer:** `torch.optim.AdamW` (warmup_steps = 1e2, learning_rate = 5e-4, epsilon = 1e-8)
 - **GPU:** 1 NVIDIA A100-SXM4-40GB
 
-| Epoch/Loss|Training|Validation|
+| Epoch|Training Loss|Validation Loss|
 |---|---|---|
-| 1 |0.924344|0.694394|
-| 2 |0.539829|0.62107|
+| 1 |0.932357|0.740638|
+| 2 |0.578139|0.668778|
 
 This repository has the notebook used to train this model.
 
 ## Usage
 
-Two special tokens are used to mark the user side of the interaction and the model's response: 
+Two special tokens are used to mark the user side of the interaction and the model's response:
 
 `<|startoftext|>`What is a language model?`<|endoftext|>`A language model is a probability distribution over a vocabulary.`<|endoftext|>`
 
@@ -93,7 +93,6 @@ responses = aira.generate(**inputs,
 print(f"Question: 👤 {question}\n")
 
 for i, response in  enumerate(responses):
-	# print only the response and remove the question
 	print(f'Response {i+1}: 🤖 {tokenizer.decode(response, skip_special_tokens=True).replace(question, "")}')
 ```
 
@@ -130,4 +129,4 @@ The model will output something like:
 
 ## License
 
-The `Aira-Instruct-PT-560M` is licensed under the Apache License, Version 2.0. See the [LICENSE](LICENSE) file for more details.
+The `Aira-Instruct-PT-560M` is licensed under the Apache License, Version 2.0. See the [LICENSE](LICENSE) file for more details.

config.json

@@ -25,7 +25,7 @@
   "skip_bias_add_qkv": false,
   "slow_but_exact": false,
   "torch_dtype": "float32",
-  "transformers_version": "4.29.2",
+  "transformers_version": "4.30.2",
   "unk_token_id": 0,
   "use_cache": true,
   "vocab_size": 250683

generation_config.json

@@ -3,5 +3,5 @@
   "bos_token_id": 1,
   "eos_token_id": 2,
   "pad_token_id": 3,
-  "transformers_version": "4.29.2"
+  "transformers_version": "4.30.2"
 }

pytorch_model.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:9c0f3b1d27e0dd60ba4e7d8e145eb319cffd8b6038b6955a4b46b3eea42ae055
+oid sha256:fe38e953bc76866a6d9b0eaf870311db8fca4768c6d0fd5ff66c67b3d8f08ac7
 size 2236150625

training_stats.parquet

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:7164db923be9d5c096ba5f7be8fd41e29070a349aac9497ff7273def2a6b14fd
-size 3041
+oid sha256:3be55941ee6155d9fe50d80a97931091829f3222dd41b3370a8a45dfcfd9cfa5
+size 3042