README.md

Try to get it to answer your questions, if you even can...

A frankenMoE of TinyLlama-1.1B-1T-OpenOrca TinyLlama-1.1B-intermediate-step-1195k-token-2.5T and tiny-llama-1.1b-chat-medical

Most 1.1B models are decoherent and can't even answer simple questions. I found the models that don't fail in this regard, then mashed 32 copies of those 3 models together into a 32x MoE