|
|
--- |
|
|
datasets: |
|
|
- c-s-ale/alpaca-gpt4-data |
|
|
- Open-Orca/OpenOrca |
|
|
- Intel/orca_dpo_pairs |
|
|
- allenai/ultrafeedback_binarized_cleaned |
|
|
- HuggingFaceH4/no_robots |
|
|
license: cc-by-nc-4.0 |
|
|
language: |
|
|
- en |
|
|
library_name: ExLlamaV2 |
|
|
pipeline_tag: text-generation |
|
|
tags: |
|
|
- Mistral |
|
|
- SOLAR |
|
|
- Quantized Model |
|
|
- exl2 |
|
|
base_model: |
|
|
- rishiraj/meow |
|
|
--- |
|
|
|
|
|
# exl2 quants for meow |
|
|
|
|
|
This repository includes the quantized models for the [meow](https://huggingface.co/rishiraj/meow) model by [Rishiraj Acharya](https://huggingface.co/rishiraj). meow is a fine-tune of [SOLAR-10.7B-Instruct-v1.0](https://huggingface.co/upstage/SOLAR-10.7B-Instruct-v1.0) with the [no_robots](https://huggingface.co/datasets/HuggingFaceH4/no_robots) dataset. |
|
|
|
|
|
## Current models |
|
|
|
|
|
| exl2 BPW | Model Branch | Model Size | |
|
|
|-|-|-| |
|
|
| 2-Bit | main | N/A | |
|
|
| 4-Bit | 4bit | N/A | |
|
|
| 5-Bit | 5bit | N/A | |
|
|
| 6-Bit | 6bit | 8.23 GB | |
|
|
| 8-Bit | 8bit | N/A | |
|
|
|
|
|
## Where to use |
|
|
|
|
|
There are a couple places you can use an exl2 model, here are a few: |
|
|
|
|
|
- [oobabooga's Text Gen Webui](https://github.com/oobabooga/text-generation-webui) |
|
|
- [tabbyAPI](https://github.com/theroyallab/tabbyAPI) |
|
|
- [ExUI](https://github.com/turboderp/exui) |
|
|
- [KoboldAI](https://github.com/henk717/KoboldAI) (Clone repo, don't use snapshot) |
|
|
|
|
|
# WARNING |
|
|
Model cannot be used commercially due to the Alpaca dataset license. |
|
|
|
