berkeley-nest

/

Starling-LM-7B-alpha

like 553

Follow

Berkeley-Nest 61

Thank you for the kind word @rjmehta !!!

amazing

This model is outstanding. Thank you for your work.

@TheBloke @banghua Any plans for GPTQ Quantization?

Just finished a little while ago!

Thx @TheBloke !

@rjmehta We would also like to try some quantization. But the current model still has some issues that might need to be fixed first, including

1. Output unnecessary and weird content at the beginning or end, occasionally.

2. Hallucinates a lot.

I hope next version can fix most of 1. But for 2 we can only pray that such small model has memorized good amount of knowledge.

In any case, we'll probably devote our limited computation resource to improve the model first. After we get a satisfying and stable version, we're happy to explore a bit more other possibilities!

Feel free to ping me when your new model(s) are up and I will prioritise their quantisations

Thank you so much @TheBloke ! Will let you know once it's up!!

_