---
datasets:
- mlabonne/alpagasus
language:
- en
pipeline_tag: text-generation
tags:
- llama
- alpaca
- alpagasus
---
# 🦙🕊️ Alpagasus-2-7b

📝 [Paper](https://arxiv.org/abs/2307.08701) | 📄 [Blog](https://lichang-chen.github.io/AlpaGasus/) | 💻 [Code](https://github.com/gpt4life/alpagasus/tree/main) | 🤗 [Model](https://huggingface.co/gpt4life/alpagasus-7b) (unofficial)

This is a `Llama-2-7b-hf` model fine-tuned using QLoRA (4-bit precision) on the [`mlabonne/alpagasus`](https://huggingface.co/datasets/mlabonne/alpagasus) dataset, which is a high-quality subset (9k samples) of the Alpaca dataset (52k samples).

## 🔧 Training

![](https://i.imgur.com/ebwyRbo.png)

It was trained on an RTX 3090 using the [🐜🔧TinyTuner](https://github.com/mlabonne/tinytuner). Parameters:

```yaml
# Dataset
dataset_name: mlabonne/alpagasus
prompt_template: alpaca
max_seq_length: 512
val_set_size: 0.01

# Loading
load_in_8bit: false
load_in_4bit: true
bf16: true
fp16: false
tf32: true

# Lora
adapter: qlora
lora_model_dir:
lora_r: 8
lora_alpha: 16
lora_dropout: 0.1
lora_target_modules:
  - q_proj
  - v_proj
lora_fan_in_fan_out:

# Training
learning_rate: 0.00002
micro_batch_size: 24
gradient_accumulation_steps: 1
num_epochs: 3
lr_scheduler_type: cosine
optim: paged_adamw_32bit
group_by_length: true
warmup_ratio: 0.03
eval_steps: 0.01
save_strategy: epoch
logging_steps: 1
weight_decay: 0
max_grad_norm:
max_steps: -1
gradient_checkpointing: true

# QLoRA
bnb_4bit_compute_dtype: float16
bnb_4bit_quant_type: nf4
bnb_4bit_use_double_quant: false
```

## 💻 Usage

``` python
# pip install transformers accelerate

from transformers import AutoTokenizer
import transformers
import torch

model = "mlabonne/alpagasus-2-7b"
prompt = "What is a large language model?"

tokenizer = AutoTokenizer.from_pretrained(model)
pipeline = transformers.pipeline(
    "text-generation",
    model=model,
    torch_dtype=torch.float16,
    device_map="auto",
)

sequences = pipeline(
    f'### Instruction: {prompt}',
    do_sample=True,
    top_k=10,
    num_return_sequences=1,
    eos_token_id=tokenizer.eos_token_id,
    max_length=200,
)
for seq in sequences:
    print(f"Result: {seq['generated_text']}")
```