what is your "continuous finetuning"
Hi
@rombodawg
Congrats, very strong 72B model!
Rombos-LLM-V2.5-Qwen-72b is a continues finetuned version of Qwen2.5-72B. I noticed recently that the Qwen team did not learn from my methods of continuous finetuning, the great benefits, and no downsides of it. So I took it upon myself to merge the instruct model with the base model myself using the Ties merge method
Is there a place to read about this "continuous finetuning" method you mentioned here? Is it merging the base with instruct or is it any fine-tuning involve?
Thank you and congrats again
@MaziyarPanahi Thank you so much
And thank you @Joseph717171 for linking my write up. Ive been busy and havnt had a chance to look through all my notifications
Thanks for sharing the doc! It is a smart strategy.
Does this strategy apply to full fine-tuned models (without LoRA)?
After merging the models, have you done other alignment on top of the merged model?
Have you tried to merge multiple fine-tuned Qwen2.5-72B-Instruct models (e.g., this model and MaziyarPanahi/calme-2.1-qwen2-72b that is performing very well in IFEval)?
@tanliboy In theory you could merge finetuned models that dont have lora adapters, however im not sure how the performance would compare. (But only if the model is trained on the base model, i cannot promise this would work at all to increase performance if the finetuned model is finetune on the instruct model) The config would look something like this
models:
- model: ./qwen-72b-instruct
parameters:
weight: 1
- model: ./calme-2.1-qwen2-72b
parameters:
weight: 1
merge_method: ties
base_model: ./qwen-72b
parameters:
normalize: true
int8_mask: true
dtype: bfloat16
isn't this how merges describe in merge paper?
Thanks,
@rombodawg
!
@djuna
, would you mind sharing the link of the paper you mentioned?
@tanliboy
I forgot which one but I think in one of these video(from arcee's Chief Evangelist)
https://youtu.be/cvOpX75Kz4M
https://youtu.be/qbAvOgGmFuE