leonzhou286
/

llama3_8b_instruct_moe

Model card Files Files and versions Community

Llama 3 8b Instruct MOE

Llama 3 8b Instruct base model converted to MOE style by randomly partitioning the FFN layers of each decoder layer into 8 experts of the same size. Weights are directly taken from the llama3 instruct base model.

Downloads last month: 0

Safetensors

Model size

8.03B params

Tensor type

F32

·

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for leonzhou286/llama3_8b_instruct_moe

Base model

meta-llama/Meta-Llama-3-8B-Instruct

Finetuned

(596)

this model