Sub 10B Models, when?
When can we expect to get a much smaller variant of this? It would be awesome to have a much smaller version that can run on consumer devices.
When can we expect to get a much smaller variant of this? It would be awesome to have a much smaller version that can run on consumer devices.
I understand your wish for small models that can run on consumer devices, but keep in mind that this particular model is a multimodal model, so it consists of several models in one. If they were to do the same type of a model but on a much smaller scale for consumer devices, the quality of each model inside of such small model would be much lower. To put it in perspective, if you wanted to use say the chat portion of the model, it would be much smaller than your usual 8B Llama 3.1 model and thus the quality of its output would be nowhere near that small Llama 3.1 model. I want to believe that one day we will see small models of much higher quality than we have now, but I don't think we're there yet.