Update README.md
Browse files
README.md
CHANGED
|
@@ -1,3 +1,7 @@
|
|
| 1 |
-
|
| 2 |
-
|
| 3 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
Roleplay Lora trained on llama-7b in 4-bit mode.
|
| 2 |
+
|
| 3 |
+
Trained for 3 epochs.
|
| 4 |
+
|
| 5 |
+
uses the https://github.com/teknium1/GPTeacher/tree/main/Roleplay dataset
|
| 6 |
+
|
| 7 |
+
Training in 4bit is very fast.. only took 1/2 hour on a 3090. Eval against the dataset gave a 3.x perpelexity.
|