Local Installation Video and Testing - Step by Step
Hi,
Kudos on producing such a sublime model. I did a local installation and testing video :
https://youtu.be/DiMZqWC7-04?si=wbEM5dLgHr3IHAZy
Thanks and regards,
Fahd
Wow, hey great job explaining and showing people all the steps to get this quant running! So happy to share this exciting high quality model with the world and appreciate everyone working together and sharing their notes and information! yay!
I added a link to your video in the ik_llama.cpp discussion on these quants! Thanks again!
Thanks a lot. Stellar job with the model.
Just uploaded a larger one with great (low) perplexity as well as a very small one that might fit into 128GiB RAM + 24GB VRAM though not optimized for offloading extra layers to GPU so probably slower unless you can't fit a bigger one. Kinda cool it still works at such a small size though!
Cheers and keep up the good work yourself!