Nitral-AI commited on
Commit
70df6ef
·
verified ·
1 Parent(s): 51233af

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -5,4 +5,4 @@ language:
5
  ---
6
  ## Very early test model (untested as of the moment.)
7
  # Base model: https://huggingface.co/arcee-ai/AFM-4.5B-Preview
8
- ### 5k entries of mixed sft data at lr 2e-6 in 4bit qlora with ebs 32 (bs 8 grad_accum 4) for a total of 2 epochs using cosine.
 
5
  ---
6
  ## Very early test model (untested as of the moment.)
7
  # Base model: https://huggingface.co/arcee-ai/AFM-4.5B-Preview
8
+ ### 5k entries of mixed sft data at lr 2e-6 in rank/alpha 32 4bit qlora with ebs 32 (bs 8 grad_accum 4) for a total of 2 epochs using cosine.