Interesting merge
It's pretty similar to Aura Uncensored (my favorite model), but with ResplendentAI/Smarts_Llama3 instead of ResplendentAI/Theory_of_Mind_Llama3, using a more uncensored base than Undi95/Llama-3-Unholy-8B.
That said, could you tell me your setup and how much time do this merges need, please? I should start doing my own, but as of now I only have a 1060 3GB and 16GB RAM, so I need to get a new computer.
Model stock is pretty easy to run, however merging loras to a model is not.
So if I wanted to run OAS over Aura uncensored for example, I could do it if I had 32GB RAM? Buying an extra 16GB RAM is pretty cheap, all things considered xD
That said, could you tell me your setup and how much time do this merges need, please? I should start doing my own, but as of now I only have a 1060 3GB and 16GB RAM, so I need to get a new computer.
As for setup, I run a ryzen 5 5600G @4.55Ghz, 32GB of mismatched ram 😶, rtx2080 (unused for merging)
And still the longest portion of merging is downloading the models 🥲
This is the amount of downloading I've done just merging the recent models.
Last month was about 300GB so about a TB worth 😸 or 7.5 hours of downloading at my network speed (300mbit)
So if I wanted to run OAS over Aura uncensored for example, I could do it if I had 32GB RAM? Buying an extra 16GB RAM is pretty cheap, all things considered xD
You would have to merge it with a oas model which means you lose a good portion of aura and may not get the oas layers.
Which is why i try to replicate sovl best by merging loras similar to the orignal model, It allows me to keep the style of sovl while using a much more uncensored model as a base.
OAS models work by targeting specific layers inside the model, and most models don't specify which layers those are. so when merging it with aura it's extremely hard to guarantee you're including those uncensored layers into the final merge
But you can guarantee you keep those layers when just merging loras in a similar style over the top of the OAS model.
I hope that makes sense?
You would have to merge it with a oas model which means you lose a good portion of aura and may not get the oas layers.
Which is why i try to replicate sovl best by merging loras similar to the orignal model, It allows me to keep the style of sovl while using a much more uncensored model as a base.
OAS models work by targeting specific layers inside the model, and most models don't specify which layers those are. so when merging it with aura it's extremely hard to guarantee you're including those uncensored layers into the final merge
But you can guarantee you keep those layers when just merging loras in a similar style over the top of the OAS model.
I hope that makes sense?
Yeah, thank you for the advice! So if I understood correctly your explanation, it would be better if I did a merge with Undi95/Llama3-Unholy-8B-OAS that already has OAS applied to it than trying to run OAS myself on an existing model.
Can OAS be run on low end hardware?
I think we misunderstood your question.
Can OAS be run on low end hardware?
I think we misunderstood your question.
Yes knowing that would be useful too. I need to buy a new computer this year anyway, so I will have better hardware in a few months probably.
Yeah, thank you for the advice! So if I understood correctly your explanation, it would be better if I did a merge with Undi95/Llama3-Unholy-8B-OAS that already has OAS applied to it than trying to run OAS myself on an existing model.
Yes, merge with an existing OAS model.
I don't have experience running OAS. Last time I skimmed the thread they were using big server GPU's
Can OAS be run on low end hardware?
I think we misunderstood your question.
Yes knowing that would be useful too. I need to buy a new computer this year anyway, so I will have better hardware in a few months probably.
OAS seems to still be quite complicated at this point, i imagine it will get easier.
It also requires model training on a gpu.
You can read into it here.
https://gist.github.com/wassname/42aba7168bb83e278fcfea87e70fa3af
This is the original script, which uses fp16 for model loading, so 24GB of vram would probably be necessary
https://colab.research.google.com/drive/1a-aQvKC9avdZpdyBn4jgRQFObTPy1JZw?usp=sharing#scrollTo=Vnp65Vsg5x-5
Alright, thank you for the info. I thought it would be better running OAS on top of it since Undi95 said that he ran it on top of Lumimaid, having better results than with Unholy, since Lumimaid was already fairly NSFW.
I'm not Undi95, so when I get to it I'll follow your advice and start it easy.
OAS seems to still be quite complicated at this point, i imagine it will get easier.
It also requires model training on a gpu.
You can read into it here.
https://gist.github.com/wassname/42aba7168bb83e278fcfea87e70fa3afThis is the original script, which uses fp16 for model loading, so 24GB of vram would probably be necessary
Wow, 24GB VRAM. I better save for a 4090 then. I'll read those articles, thanks for sharing them.
Alright, thank you for the info. I thought it would be better running OAS on top of it since Undi95 said that he ran it on top of Lumimaid, having better results than with Unholy, since Lumimaid was already fairly NSFW.
I'm not Undi95, so when I get to it I'll follow your advice and start it easy.
Running OAS over the top would be the best way to do it, yes
But I figured it's not really an option for most
Replacing bases would be second best
Take the orignal Aura Uncensored
models:
- model: Undi95/Llama-3-Unholy-8B+ResplendentAI/Theory_of_Mind_Llama3
- model: Undi95/Llama-3-Unholy-8B+ResplendentAI/Aura_Llama3
- model: Undi95/Llama-3-Unholy-8B+ResplendentAI/Luna_Llama3
- model: Undi95/Llama-3-Unholy-8B+ResplendentAI/BlueMoon_Llama3
- model: Undi95/Llama-3-Unholy-8B+ResplendentAI/RP_Format_QuoteAsterisk_Llama3
merge_method: model_stock
base_model: Undi95/Llama-3-Unholy-8B
dtype: float16
and swap models for the OAS version
models:
- model: Undi95/Llama3-Unholy-8B-OAS+ResplendentAI/Theory_of_Mind_Llama3
- model: Undi95/Llama3-Unholy-8B-OAS+ResplendentAI/Aura_Llama3
- model: Undi95/Llama3-Unholy-8B-OAS+ResplendentAI/Luna_Llama3
- model: Undi95/Llama3-Unholy-8B-OAS+ResplendentAI/BlueMoon_Llama3
- model: Undi95/Llama3-Unholy-8B-OAS+ResplendentAI/RP_Format_QuoteAsterisk_Llama3
merge_method: model_stock
base_model: Undi95/Llama3-Unholy-8B-OAS
dtype: float16
I should be able to run the merge if Undi uploads Unholy OAS in fp16 (i have requested the fp16 files already)
You can run the merge using the fp32, it will save as dtype.
However, it will need more ram.
You can run the merge with the fp32
I shall cook it up then!
Take the orignal Aura Uncensored
and swap models for the OAS versionmodels: - model: Undi95/Llama3-Unholy-8B-OAS+ResplendentAI/Theory_of_Mind_Llama3 - model: Undi95/Llama3-Unholy-8B-OAS+ResplendentAI/Aura_Llama3 - model: Undi95/Llama3-Unholy-8B-OAS+ResplendentAI/Luna_Llama3 - model: Undi95/Llama3-Unholy-8B-OAS+ResplendentAI/BlueMoon_Llama3 - model: Undi95/Llama3-Unholy-8B-OAS+ResplendentAI/RP_Format_QuoteAsterisk_Llama3 merge_method: model_stock base_model: Undi95/Llama3-Unholy-8B-OAS dtype: float16I should be able to run the merge if Undi uploads Unholy OAS in fp16 (i have requested the fp16 files already)
That was precisely my original plan B, switching the base to Unholy OAS and see what happens xD
That was precisely my original plan B, switching the base to Unholy OAS and see what happens xD
It'll be up in half an hour to an hour, all is going well so far!
FP32 download took 20 minutes @_@
You are a hero, man.
20 minutes? It took me 3+ hours to download only the FP32. I already downloaded everything just in case I could cook it myself :) Or save it in my exernal HDD waiting for when I could.