Quantization made by Richard Erkhov.

Llama3.1-60B - GGUF

Model creator: https://huggingface.co/allknowingroger/
Original model: https://huggingface.co/allknowingroger/Llama3.1-60B/

Name	Quant method	Size
Llama3.1-60B.Q2_K.gguf	Q2_K	21.57GB
Llama3.1-60B.IQ3_XS.gguf	IQ3_XS	23.96GB
Llama3.1-60B.IQ3_S.gguf	IQ3_S	25.28GB
Llama3.1-60B.Q3_K_S.gguf	Q3_K_S	25.2GB
Llama3.1-60B.IQ3_M.gguf	IQ3_M	26.09GB
Llama3.1-60B.Q3_K.gguf	Q3_K	28.0GB
Llama3.1-60B.Q3_K_M.gguf	Q3_K_M	28.0GB
Llama3.1-60B.Q3_K_L.gguf	Q3_K_L	30.42GB
Llama3.1-60B.IQ4_XS.gguf	IQ4_XS	31.33GB
Llama3.1-60B.Q4_0.gguf	Q4_0	32.74GB
Llama3.1-60B.IQ4_NL.gguf	IQ4_NL	33.03GB
Llama3.1-60B.Q4_K_S.gguf	Q4_K_S	32.96GB
Llama3.1-60B.Q4_K.gguf	Q4_K	34.73GB
Llama3.1-60B.Q4_K_M.gguf	Q4_K_M	34.73GB
Llama3.1-60B.Q4_1.gguf	Q4_1	36.29GB
Llama3.1-60B.Q5_0.gguf	Q5_0	39.84GB
Llama3.1-60B.Q5_K_S.gguf	Q5_K_S	39.84GB
Llama3.1-60B.Q5_K.gguf	Q5_K	40.86GB
Llama3.1-60B.Q5_K_M.gguf	Q5_K_M	40.86GB
Llama3.1-60B.Q5_1.gguf	Q5_1	43.38GB
Llama3.1-60B.Q6_K.gguf	Q6_K	47.38GB
Llama3.1-60B.Q8_0.gguf	Q8_0	61.36GB

Original model description:

license: apache-2.0 library_name: transformers tags: - mergekit - merge base_model: - mattshumer/Reflection-Llama-3.1-70B model-index: - name: Llama3.1-60B results: - task: type: text-generation name: Text Generation dataset: name: IFEval (0-Shot) type: HuggingFaceH4/ifeval args: num_few_shot: 0 metrics: - type: inst_level_strict_acc and prompt_level_strict_acc value: 18.15 name: strict accuracy source: url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=allknowingroger/Llama3.1-60B name: Open LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: BBH (3-Shot) type: BBH args: num_few_shot: 3 metrics: - type: acc_norm value: 7.78 name: normalized accuracy source: url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=allknowingroger/Llama3.1-60B name: Open LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: MATH Lvl 5 (4-Shot) type: hendrycks/competition_math args: num_few_shot: 4 metrics: - type: exact_match value: 0.0 name: exact match source: url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=allknowingroger/Llama3.1-60B name: Open LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: GPQA (0-shot) type: Idavidrein/gpqa args: num_few_shot: 0 metrics: - type: acc_norm value: 5.93 name: acc_norm source: url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=allknowingroger/Llama3.1-60B name: Open LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: MuSR (0-shot) type: TAUR-Lab/MuSR args: num_few_shot: 0 metrics: - type: acc_norm value: 2.18 name: acc_norm source: url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=allknowingroger/Llama3.1-60B name: Open LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: MMLU-PRO (5-shot) type: TIGER-Lab/MMLU-Pro config: main split: test args: num_few_shot: 5 metrics: - type: acc value: 25.67 name: accuracy source: url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=allknowingroger/Llama3.1-60B name: Open LLM Leaderboard

merge

This is a merge of pre-trained language models created using mergekit.

Merge Details

Merge Method

This model was merged using the passthrough merge method.

Models Merged

The following models were included in the merge:

mattshumer/Reflection-Llama-3.1-70B

Configuration

The following YAML configuration was used to produce this model:

slices:
  - sources:
    - model: mattshumer/Reflection-Llama-3.1-70B
      layer_range: [0, 39]
  - sources:
    - model: mattshumer/Reflection-Llama-3.1-70B
      layer_range: [8, 39]
merge_method: passthrough
dtype: float16

Open LLM Leaderboard Evaluation Results

Detailed results can be found here

Metric	Value
Avg.	9.95
IFEval (0-Shot)	18.15
BBH (3-Shot)	7.78
MATH Lvl 5 (4-Shot)	0.00
GPQA (0-shot)	5.93
MuSR (0-shot)	2.18
MMLU-PRO (5-shot)	25.67