Model Card for Model ID

[OPEA/Qwen2.5-7B-Instruct-int4-sym-inc]

Uses

import torch
import transformers
model = transformers.AutoModel.from_pretrained("yujiepan/Qwen2.5-7B-Instruct-int4-sym-inc-autogptq", torch_dtype=torch.float16)

Evaluation

model_id	Qwen/Qwen2.5-7B-Instruct	yujiepan/Qwen2.5-7B-Instruct-int4-sym-inc-autogptq
wikitext	9.72	10.01
avg of 8 zero shot tasks	74.82 (0.00%)	74.36 (-0.61%)
mmlu_5shot	74.26 (0.00%)	73.66 (-0.81%)

arc_challenge	52.65	53.07
arc_easy	81.86	80.68
boolq	86.39	86.02
hellaswag	62.02	61.39
lambada_openai	69.73	68.78
piqa	79.54	78.67
sciq	95.70	95.40
winogrande	70.64	70.88