Good stuff
I like it so far! Can you share the uncompressed so I can make some chonky quants?
Thanks, will do!
It'll take some time to upload, I'll make gghfez/SmartMaid-123b public once it's finished uploading.
I'll probably end up downloading your quants of it to save my GPUs from being tied up for hours.
Sounds good. I'll be on the lookout.
Awesome work, especially if it helps with the Luminum repetition issue! If anyone could do Exl2 8.0bpw (for 6 GPUs with 24 vram) and 6.0bpw (4 GPUs with 24 vram) that would be greatly appreciated!
That's finally uploaded
https://huggingface.co/gghfez/SmartMaid-123b
Awesome, I'll download and get to cooking.. Based on the other 123B models I have done it will probably be about 12 hours before the 8.0bpw is ready then the others will follow
Fresh from the oven
https://huggingface.co/BigHuggyD/gghfez_SmartMaid-123b_exl2_8.0bpw_h8
7.0bpw is cooking .. then 6.0 and so on and so on