A 1 billion parameters model, fine-tuned from Llama-3.2-1B-Instruct

Model Card: Precis-1B-Instruct

Model Details

Model Name: Precis-1B-Instruct
Base Model: LLaMA 3.2 1B Instruct
Parameters: 1.24 billion

Model Description

Precis-1B-Instruct is a fine-tuned language model designed to perform a wide variety of instruction-following tasks. It is based on the LLaMA 3.2 architecture with 1 billion parameters, fine-tuned using the Arthur-LAGACHERIE/EntierInstruct-66k dataset.

This model is intended to provide concise and accurate responses in natural language tasks, making it suitable for applications such as summarization, question answering, math, and dialogue systems.

---

Training Details

Dataset:
The Arthur-LAGACHERIE/EntierInstruct-66k dataset comprises a wide range of instruction-based prompts and outputs, including math, instruction following, and code.

Fine-Tuning Process:
The model was fine-tuned using the Hugging Face peft library with LoRA techniques (SFT).

Limitations and Risks

• Limitations:
  The model may not generalize well to tasks or inputs outside its training distribution. It is designed for English tasks and may perform poorly in other languages.

• Ethical Considerations:
  Ensure the model is not used to generate harmful, misleading, or biased outputs. Users must comply with ethical AI guidelines.

Example Code:

from transformers import AutoModelForCausalLM, AutoTokenizer

# Load the model and tokenizer
model_name = "Arthur-LAGACHERIE/Precis-1B-Instruct"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)

# Generate text
input_text = "What the answer to life."
inputs = tokenizer(input_text, return_tensors="pt")
outputs = model.generate(**inputs, max_length=100)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))