Ayyystin commited on
Commit
2ce7f47
1 Parent(s): d477bb4

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +158 -0
README.md ADDED
@@ -0,0 +1,158 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: llama3
3
+ tags:
4
+ - uncensored
5
+ - llama3
6
+ - instruct
7
+ - open
8
+ - llama-cpp
9
+ - gguf-my-repo
10
+ base_model: Orenguteng/Llama-3-8B-Lexi-Uncensored
11
+ model-index:
12
+ - name: Llama-3-8B-Lexi-Uncensored
13
+ results:
14
+ - task:
15
+ type: text-generation
16
+ name: Text Generation
17
+ dataset:
18
+ name: AI2 Reasoning Challenge (25-Shot)
19
+ type: ai2_arc
20
+ config: ARC-Challenge
21
+ split: test
22
+ args:
23
+ num_few_shot: 25
24
+ metrics:
25
+ - type: acc_norm
26
+ value: 59.56
27
+ name: normalized accuracy
28
+ source:
29
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Orenguteng/Llama-3-8B-Lexi-Uncensored
30
+ name: Open LLM Leaderboard
31
+ - task:
32
+ type: text-generation
33
+ name: Text Generation
34
+ dataset:
35
+ name: HellaSwag (10-Shot)
36
+ type: hellaswag
37
+ split: validation
38
+ args:
39
+ num_few_shot: 10
40
+ metrics:
41
+ - type: acc_norm
42
+ value: 77.88
43
+ name: normalized accuracy
44
+ source:
45
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Orenguteng/Llama-3-8B-Lexi-Uncensored
46
+ name: Open LLM Leaderboard
47
+ - task:
48
+ type: text-generation
49
+ name: Text Generation
50
+ dataset:
51
+ name: MMLU (5-Shot)
52
+ type: cais/mmlu
53
+ config: all
54
+ split: test
55
+ args:
56
+ num_few_shot: 5
57
+ metrics:
58
+ - type: acc
59
+ value: 67.68
60
+ name: accuracy
61
+ source:
62
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Orenguteng/Llama-3-8B-Lexi-Uncensored
63
+ name: Open LLM Leaderboard
64
+ - task:
65
+ type: text-generation
66
+ name: Text Generation
67
+ dataset:
68
+ name: TruthfulQA (0-shot)
69
+ type: truthful_qa
70
+ config: multiple_choice
71
+ split: validation
72
+ args:
73
+ num_few_shot: 0
74
+ metrics:
75
+ - type: mc2
76
+ value: 47.72
77
+ source:
78
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Orenguteng/Llama-3-8B-Lexi-Uncensored
79
+ name: Open LLM Leaderboard
80
+ - task:
81
+ type: text-generation
82
+ name: Text Generation
83
+ dataset:
84
+ name: Winogrande (5-shot)
85
+ type: winogrande
86
+ config: winogrande_xl
87
+ split: validation
88
+ args:
89
+ num_few_shot: 5
90
+ metrics:
91
+ - type: acc
92
+ value: 75.85
93
+ name: accuracy
94
+ source:
95
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Orenguteng/Llama-3-8B-Lexi-Uncensored
96
+ name: Open LLM Leaderboard
97
+ - task:
98
+ type: text-generation
99
+ name: Text Generation
100
+ dataset:
101
+ name: GSM8k (5-shot)
102
+ type: gsm8k
103
+ config: main
104
+ split: test
105
+ args:
106
+ num_few_shot: 5
107
+ metrics:
108
+ - type: acc
109
+ value: 68.39
110
+ name: accuracy
111
+ source:
112
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Orenguteng/Llama-3-8B-Lexi-Uncensored
113
+ name: Open LLM Leaderboard
114
+ ---
115
+
116
+ # Ayyystin/Llama-3-8B-Lexi-Uncensored-Q4_0-GGUF
117
+ This model was converted to GGUF format from [`Orenguteng/Llama-3-8B-Lexi-Uncensored`](https://huggingface.co/Orenguteng/Llama-3-8B-Lexi-Uncensored) using llama.cpp via the ggml.ai's [GGUF-my-repo](https://huggingface.co/spaces/ggml-org/gguf-my-repo) space.
118
+ Refer to the [original model card](https://huggingface.co/Orenguteng/Llama-3-8B-Lexi-Uncensored) for more details on the model.
119
+
120
+ ## Use with llama.cpp
121
+ Install llama.cpp through brew (works on Mac and Linux)
122
+
123
+ ```bash
124
+ brew install llama.cpp
125
+
126
+ ```
127
+ Invoke the llama.cpp server or the CLI.
128
+
129
+ ### CLI:
130
+ ```bash
131
+ llama --hf-repo Ayyystin/Llama-3-8B-Lexi-Uncensored-Q4_0-GGUF --hf-file llama-3-8b-lexi-uncensored-q4_0.gguf -p "The meaning to life and the universe is"
132
+ ```
133
+
134
+ ### Server:
135
+ ```bash
136
+ llama-server --hf-repo Ayyystin/Llama-3-8B-Lexi-Uncensored-Q4_0-GGUF --hf-file llama-3-8b-lexi-uncensored-q4_0.gguf -c 2048
137
+ ```
138
+
139
+ Note: You can also use this checkpoint directly through the [usage steps](https://github.com/ggerganov/llama.cpp?tab=readme-ov-file#usage) listed in the Llama.cpp repo as well.
140
+
141
+ Step 1: Clone llama.cpp from GitHub.
142
+ ```
143
+ git clone https://github.com/ggerganov/llama.cpp
144
+ ```
145
+
146
+ Step 2: Move into the llama.cpp folder and build it with `LLAMA_CURL=1` flag along with other hardware-specific flags (for ex: LLAMA_CUDA=1 for Nvidia GPUs on Linux).
147
+ ```
148
+ cd llama.cpp && LLAMA_CURL=1 make
149
+ ```
150
+
151
+ Step 3: Run inference through the main binary.
152
+ ```
153
+ ./main --hf-repo Ayyystin/Llama-3-8B-Lexi-Uncensored-Q4_0-GGUF --hf-file llama-3-8b-lexi-uncensored-q4_0.gguf -p "The meaning to life and the universe is"
154
+ ```
155
+ or
156
+ ```
157
+ ./server --hf-repo Ayyystin/Llama-3-8B-Lexi-Uncensored-Q4_0-GGUF --hf-file llama-3-8b-lexi-uncensored-q4_0.gguf -c 2048
158
+ ```