| Roleplay Lora trained on llama-7b in 4-bit mode. | |
| Trained for 3 epochs. | |
| uses the https://github.com/teknium1/GPTeacher/tree/main/Roleplay dataset | |
| Training in 4bit is very fast.. only took 1/2 hour on a 3090. Eval against the dataset gave a 3.x perpelexity. |