Which version for Lora training?
I read somewhere that the annealed version is for image generation, while the other is for finetuning.
But which one should I use for simple Lora training?
I'm training w annealed rn and I'm getting good results at 300 steps already, ymmv
I'd also like to know what's officially recommended, unfortunately the creator of this model hates documentation
It's quite easy though? Fast generation -> Annealed. More quailty -> default model.
I love for Lodestones to answer , or provide a document I can reference , but I doubt he will.
From spying on Discord I know what separates the V49 -> V50 Annealed training process is:
'Hi-resolution training' ( no further info)
'Annealing' = letting the V50 model train on the entire database of training images ( 5 million or maybe the full 20 million ) one final time at a very low learning rate
Some possible merging of weights / layer switch-a-roo with earlier epochs (not confirmed)
//---//
Without documentation we can't know if any of the steps 1) - 3) would interfere with V50 LoRa training , or V50 checkpoint training
, or V50 higher fidelity training / tinkering => inhibiting the V50 Annealed range of output compared to the V49.
Therefore the safe bet is to train the LoRa on the V49 , in my opinion.
I love for Lodestones to answer , or provide a document I can reference , but I doubt he will.
From spying on Discord I know what separates the V49 -> V50 Annealed training process is:
'Hi-resolution training' ( no further info)
'Annealing' = letting the V50 model train on the entire database of training images ( 5 million or maybe the full 20 million ) one final time at a very low learning rate
Some possible merging of weights / layer switch-a-roo with earlier epochs (not confirmed)
//---//
Without documentation we can't know if any of the steps 1) - 3) would interfere with V50 LoRa training , or V50 checkpoint training
, or V50 higher fidelity training / tinkering => inhibiting the V50 Annealed range of output compared to the V49.
Therefore the safe bet is to train the LoRa on the V49 , in my opinion.
I've trained a few realistic loras on the standard v50 version, and they've turned out great. They've also been less resilient to the weird banding effect that happens with longer prompts in my experiences too.