Update README.md

README.md

@@ -8,8 +8,6 @@ model-index:
   results: []
 ---
 
-<!-- This model card has been generated automatically according to the information the Trainer had access to. You
-should probably proofread and complete it, then remove this comment. -->
 
 [<img src="https://raw.githubusercontent.com/OpenAccess-AI-Collective/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/OpenAccess-AI-Collective/axolotl)
 <details><summary>See axolotl config</summary>
@@ -107,19 +105,23 @@ special_tokens:
 
 # qlora-yi-6b-200k-rawrr-run2
 
-This model was trained from scratch on the None dataset.
+This model was trained using DPO from base Yi-6B-200K
 
 ## Model description
 
-More information needed
+I figured I will be sharing my attempts as I go. \
+It's my third DPO attempt and the first one where outputs aren't completely garbled up.
+Learnings: 
+- DPO needs about 10-100 lower learning rate than SFT fine-tuning.
+- You probably don't want to add special tokens in your DPO, who could have guessed...
 
 ## Intended uses & limitations
 
-More information needed
+I intend to run aezakmi SFT finetune on this base, let's see whether this is any better than normal AEZAKMI.
 
 ## Training and evaluation data
 
-More information needed
+DPO on synthetic dataset rawrr_v1.
 
 ## Training procedure
 
@@ -139,7 +141,7 @@ The following hyperparameters were used during training:
 
 ### Training results
 
-
+Outputs are actually somewhat coherent!
 
 ### Framework versions