Fine-tuning instructions
#6
by
millionwords
- opened
Hi,
It would be great if you could also add fine-tuning instructions and provide the code and sample data for it.
We want to train it to write creative writing and to edit stories and provide feedback
Hello, @millionwords !
There are no official guide for fine-tuning EXAONE for now.
But, you can still use any of popular fine-tuning frameworks; we recommend TRL(https://github.com/huggingface/trl) or native HF trainer for starter.
Also, please aware that our smallest models (LGAI-EXAONE/EXAONE-Deep-2.4B, LGAI-EXAONE/EXAONE-3.5-2.4B-Instruct) use a slightly different architecture with tied embeddings.