Inference
#2
by
marcel
- opened
I think we can just modify the phi-2 code https://huggingface.co/microsoft/phi-2/blob/main/modeling_phi.py and add MoE
this is what I did. Just waiting for Maxime to modify the tensors name to stick to Phi-2 + config.json to original phi-2 too
then I'll push the hacked modeling_phi.py
Working on it 🫡