m-a-p/Infinity-Instruct-3M-0625-Llama3-8B-COIG-P

# Model Card for Infinity-Instruct-3M-0625-Llama3-8B-COIG-P

This repository contains the Infinity-Instruct-3M-0625-Llama3-8B-COIG-P model, a large language model fine-tuned on the COIG-P dataset. COIG-P is a high-quality, large-scale Chinese preference dataset for aligning LLMs with human values. This model is described in the paper [COIG-P: A High-Quality and Large-Scale Chinese Preference Dataset for Alignment with Human Values](https://huggingface.co/papers/2504.05535).

## Model Details

### Model Description

This model was fine-tuned using an LLM-based Chinese preference dataset annotation pipeline to avoid human intervention. The pipeline crawled and filtered 9k high-quality Chinese queries and used 15 powerful LLMs to generate and score chosen-rejected response pairs. The resulting COIG-P dataset contains 101k Chinese preference pairs across 6 domains: Chat, Code, Math, Logic, Novel, and Role. This model is an 8B parameter Llama model.

### Model Sources

- **Repository:** [https://github.com/MAP-Lab/COIG-P](https://github.com/MAP-Lab/COIG-P)
- **Paper:** [https://huggingface.co/papers/2504.05535](https://huggingface.co/papers/2504.05535)

## Uses

### Direct Use

This model can be used directly for text generation tasks, particularly those involving Chinese language and instruction following.

## Bias, Risks, and Limitations

This model, like other LLMs, may exhibit biases present in its training data. It's crucial to be aware of potential biases related to the specific domains and language (Chinese) included in the COIG-P dataset. Further research is needed to fully characterize these biases.

### Recommendations

Users should be mindful of potential biases in the model's outputs and critically evaluate the generated text.

## How to Get Started with the Model

The following code snippet demonstrates how to use the model for text generation:

```python
from transformers import AutoModelForCausalLM, AutoTokenizer, LogitsProcessorList
import torch
device = "cuda" # the device to load the model onto

model = AutoModelForCausalLM.from_pretrained("m-a-p/Infinity-Instruct-3M-0625-Llama3-8B-COIG-P",
    torch_dtype=torch.bfloat16,
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("m-a-p/Infinity-Instruct-3M-0625-Llama3-8B-COIG-P")

prompt = "Give me a short introduction to large language model."
messages = [
    {"role": "user", "content": prompt}
]

text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)
model_inputs = tokenizer([text], return_tensors="pt").to(device)

logits_processor = LogitsProcessorList(
            [
                MinLengthLogitsProcessor(1, eos_token_id=tokenizer.eos_token_id),
                TemperatureLogitsWarper(0.7),
            ]
 )
 
generated_ids = model.generate(
    model_inputs.input_ids,
    logits_processor=logits_processor,
    max_new_tokens=512
)

generated_ids = [
    output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
]

response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
print(response)

Citation

BibTeX:

@misc{pteam2025coigphighqualitylargescalechinese,
      title={COIG-P: A High-Quality and Large-Scale Chinese Preference Dataset for Alignment with Human Values}, 
      author={P Team and Siwei Wu and Jincheng Ren and Xinrun Du and Shuyue Guo and Xingwei Qu and Yiming Liang and Jie Liu and Yunwen Li and Tianyu Zheng and Boyu Feng and Huaqing Yuan and Zenith Wang and Jiaheng Liu and Wenhao Huang and Chenglin Cai and Haoran Que and Jian Yang and Yuelin Bai and Zekun Moore Wang and Zhouliang Yu and Qunshu Lin and Ding Pan and Yuchen Jiang and Tiannan Wang and Wangchunshu Zhou and Shenzhi Wang and Xingyuan Bu and Minghao Liu and Guoyin Wang and Ge Zhang and Chenghua Lin},
      year={2025},
      eprint={2504.05535},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2504.05535}, 
}