Update README.md
Browse files
README.md
CHANGED
@@ -5,4 +5,4 @@ language:
|
|
5 |
---
|
6 |
## Very early test model (untested as of the moment.)
|
7 |
# Base model: https://huggingface.co/arcee-ai/AFM-4.5B-Preview
|
8 |
-
### 5k entries of mixed sft data at lr 2e-6 in 4bit qlora with ebs 32 (bs 8 grad_accum 4) for a total of 2 epochs using cosine.
|
|
|
5 |
---
|
6 |
## Very early test model (untested as of the moment.)
|
7 |
# Base model: https://huggingface.co/arcee-ai/AFM-4.5B-Preview
|
8 |
+
### 5k entries of mixed sft data at lr 2e-6 in rank/alpha 32 4bit qlora with ebs 32 (bs 8 grad_accum 4) for a total of 2 epochs using cosine.
|