---
license: apache-2.0
datasets:
- shibing624/alpaca-zh
language:
- zh
tags:
- LoRA
- LLaMA
- Alpaca
- PEFT
- int8
---

# Model Card for llama-7b-alpaca-zh-20k

<!-- Provide a quick summary of what the model is/does. -->

## Uses

<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->

### Direct Use

<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->

```python
from peft import PeftModel
from transformers import GenerationConfig, LlamaForCausalLM, LlamaTokenizer


max_memory = {i: "15GIB" for i in range(torch.cuda.device_count())}
tokenizer = LlamaTokenizer.from_pretrained(base_model)
model = LlamaForCausalLM.from_pretrained(
    base_model,
    load_in_8bit=True,
    torch_dtype=torch.float16,
    device_map="auto"
    max_memory=max_memory
)
model = PeftModel.from_pretrained(
    model,
    lora_weights,
    torch_dtype=torch.float16,
    max_memory=max_memory
)
```