README.md

metadata

language:
  - en
license: apache-2.0
datasets:
  - Open-Orca/OpenOrca
  - teknium/openhermes
  - cognitivecomputations/dolphin
  - jondurbin/airoboros-3.1
  - unalignment/toxic-dpo-v0.1
  - unalignment/spicy-3.1
model-index:
  - name: Hippolyta-7B-bf16
    results:
      - task:
          type: text-generation
          name: Text Generation
        dataset:
          name: AI2 Reasoning Challenge (25-Shot)
          type: ai2_arc
          config: ARC-Challenge
          split: test
          args:
            num_few_shot: 25
        metrics:
          - type: acc_norm
            value: 60.58
            name: normalized accuracy
        source:
          url: >-
            https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Kquant03/Hippolyta-7B-bf16
          name: Open LLM Leaderboard
      - task:
          type: text-generation
          name: Text Generation
        dataset:
          name: HellaSwag (10-Shot)
          type: hellaswag
          split: validation
          args:
            num_few_shot: 10
        metrics:
          - type: acc_norm
            value: 79.98
            name: normalized accuracy
        source:
          url: >-
            https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Kquant03/Hippolyta-7B-bf16
          name: Open LLM Leaderboard
      - task:
          type: text-generation
          name: Text Generation
        dataset:
          name: MMLU (5-Shot)
          type: cais/mmlu
          config: all
          split: test
          args:
            num_few_shot: 5
        metrics:
          - type: acc
            value: 57.71
            name: accuracy
        source:
          url: >-
            https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Kquant03/Hippolyta-7B-bf16
          name: Open LLM Leaderboard
      - task:
          type: text-generation
          name: Text Generation
        dataset:
          name: TruthfulQA (0-shot)
          type: truthful_qa
          config: multiple_choice
          split: validation
          args:
            num_few_shot: 0
        metrics:
          - type: mc2
            value: 55.74
        source:
          url: >-
            https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Kquant03/Hippolyta-7B-bf16
          name: Open LLM Leaderboard
      - task:
          type: text-generation
          name: Text Generation
        dataset:
          name: Winogrande (5-shot)
          type: winogrande
          config: winogrande_xl
          split: validation
          args:
            num_few_shot: 5
        metrics:
          - type: acc
            value: 73.95
            name: accuracy
        source:
          url: >-
            https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Kquant03/Hippolyta-7B-bf16
          name: Open LLM Leaderboard
      - task:
          type: text-generation
          name: Text Generation
        dataset:
          name: GSM8k (5-shot)
          type: gsm8k
          config: main
          split: test
          args:
            num_few_shot: 5
        metrics:
          - type: acc
            value: 1.82
            name: accuracy
        source:
          url: >-
            https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Kquant03/Hippolyta-7B-bf16
          name: Open LLM Leaderboard

The flower of Ares.

GGUF files here

Fine-tuned on mistralai/Mistral-7B-v0.1...my team and I reformatted many different datasets and included a small amount of private stuff to see how much we could improve mistral.

I spoke to it personally for about an hour, and I believe we need to work on our format for the private dataset a bit more, but other than that, it turned out great. I will be uploading it to open llm evaluations, today.

• Uses Mistral prompt template with chat-instruct.

Open LLM Leaderboard Evaluation Results

Detailed results can be found here

Metric	Value
Avg.	54.96
AI2 Reasoning Challenge (25-Shot)	60.58
HellaSwag (10-Shot)	79.98
MMLU (5-Shot)	57.71
TruthfulQA (0-shot)	55.74
Winogrande (5-shot)	73.95
GSM8k (5-shot)	1.82