Fine tuning the LLM backbone
#76
by
antogrk
- opened
I'm working on a text-only task (which ultimately will be expanded to a multimodal task in the future). I was wondering if it's possible to fine-tune only the language model that's used as the backbone of the model on only textual data. Also, is it possible to apply LoRA on the LLM and train only the linear layers of it?
It would be very helpful if you could provide a very basic script of how the fine-tuning can be done for a causal language modeling task.
antogrk
changed discussion status to
closed
antogrk
changed discussion status to
open