Virt-io

/

Irene-RP-7B

like 1

Sure, do you have anything in mind?

Looking for merge methods, will let you know if I find a worthy candidate or any other solid idea, but which methods will retain a mix of all models in all layers the best?

I've mostly played with task_arithmetic and slerp.

I've tried dare_ties but it keeps crashing. I only have 16GB of ram >:|

Anyway maybe give this video a watch. It gives a decent overview of merging.

@Lewdiculous

Will be uploading another test model. I lowered Nina-v2-7B and it is extremely unhinged.

@Lewdiculous

Test model here

128k test here

I still need to do some more testing with different configs on the 128k, since it affected the unhingedness of the outputs.

Aighty, bit busy but I'm keeping tabs for when I can get to it.
🚀