Model Details

This is microsoft/Phi-3.5-mini-instruct quantized with AutoRound to 4-bit and symmetric quantization for compatibility with Marlin. The model has been created, tested, and evaluated by The Kaitchup.

Details on quantization process, evaluation, and how to use the model here: Fine-tuning Phi-3.5 MoE and Mini on Your Computer

Developed by: The Kaitchup
Language(s) (NLP): English
License: cc-by-4.0

Downloads last month: 12

Safetensors

Model size

683M params

Tensor type

I32

FP16

Inference Examples

Text Generation

This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.