Post
1605
You can now use DoRA for your embedding layers!
PR: https://github.com/huggingface/peft/pull/2006
I have documented my journey of this specific PR in a blog post for everyone to read. The highlight of the PR was when the first author of DoRA reviewed my code.
Blog Post: https://huggingface.co/blog/ariG23498/peft-dora
Huge thanks to @BenjaminB for all the help I needed.
PR: https://github.com/huggingface/peft/pull/2006
I have documented my journey of this specific PR in a blog post for everyone to read. The highlight of the PR was when the first author of DoRA reviewed my code.
Blog Post: https://huggingface.co/blog/ariG23498/peft-dora
Huge thanks to @BenjaminB for all the help I needed.