Update README.md
Browse files
README.md
CHANGED
@@ -1,6 +1,6 @@
|
|
1 |
---
|
2 |
language:
|
3 |
-
-
|
4 |
tags:
|
5 |
- instruction-tuning
|
6 |
- text-generation
|
@@ -62,63 +62,6 @@ Erynn excels at a variety of text generation tasks:
|
|
62 |
|
63 |
- **💻 Basic Code Examples**: Generate simple code snippets for common tasks
|
64 |
|
65 |
-
## 📥 Using Erynn
|
66 |
-
|
67 |
-
The model works best with simple prompt formats. Here's how to use it:
|
68 |
-
|
69 |
-
```python
|
70 |
-
import torch
|
71 |
-
from transformers import AutoTokenizer, AutoModelForCausalLM
|
72 |
-
|
73 |
-
# Model paths
|
74 |
-
MODEL_PATH = "erynn/erynn-model"
|
75 |
-
|
76 |
-
def load_model():
|
77 |
-
"""Load the Erynn model and tokenizer."""
|
78 |
-
# Load model with efficient memory usage
|
79 |
-
model = AutoModelForCausalLM.from_pretrained(
|
80 |
-
MODEL_PATH,
|
81 |
-
device_map="auto",
|
82 |
-
torch_dtype=torch.float16,
|
83 |
-
low_cpu_mem_usage=True
|
84 |
-
)
|
85 |
-
|
86 |
-
# Load tokenizer
|
87 |
-
tokenizer = AutoTokenizer.from_pretrained(MODEL_PATH)
|
88 |
-
tokenizer.pad_token = tokenizer.eos_token
|
89 |
-
return model, tokenizer
|
90 |
-
|
91 |
-
def get_response(model, tokenizer, instruction, context=None):
|
92 |
-
"""
|
93 |
-
Generate a response for the given instruction and optional context.
|
94 |
-
Example: get_response(model, tokenizer, "Write an ad for a phone")
|
95 |
-
"""
|
96 |
-
# Build simple prompt
|
97 |
-
prompt = f"Instruction: {instruction}\n"
|
98 |
-
if context and context.strip():
|
99 |
-
prompt += f"Context: {context}\n"
|
100 |
-
prompt += "Response: "
|
101 |
-
|
102 |
-
# Tokenize input
|
103 |
-
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
|
104 |
-
|
105 |
-
# Generate response
|
106 |
-
with torch.no_grad():
|
107 |
-
output = model.generate(
|
108 |
-
input_ids=inputs["input_ids"],
|
109 |
-
attention_mask=inputs["attention_mask"],
|
110 |
-
max_new_tokens=100,
|
111 |
-
temperature=0.7,
|
112 |
-
top_p=0.9,
|
113 |
-
repetition_penalty=1.2,
|
114 |
-
do_sample=True,
|
115 |
-
pad_token_id=tokenizer.eos_token_id
|
116 |
-
)
|
117 |
-
|
118 |
-
# Extract response
|
119 |
-
response = tokenizer.decode(output[0], skip_special_tokens=True)
|
120 |
-
response_start = response.find("Response: ") + len("Response: ")
|
121 |
-
return response[response_start:].strip()
|
122 |
|
123 |
## 📊 Performance Examples
|
124 |
|
@@ -146,9 +89,9 @@ Response:
|
|
146 |
|
147 |
Thanks to advanced quantization techniques, Erynn runs efficiently on standard hardware:
|
148 |
|
149 |
-
- **GPU**: NVIDIA GPU
|
150 |
-
- **CPU**:
|
151 |
-
- **RAM**:
|
152 |
|
153 |
## 🛠️ Limitations
|
154 |
|
|
|
1 |
---
|
2 |
language:
|
3 |
+
- en
|
4 |
tags:
|
5 |
- instruction-tuning
|
6 |
- text-generation
|
|
|
62 |
|
63 |
- **💻 Basic Code Examples**: Generate simple code snippets for common tasks
|
64 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
65 |
|
66 |
## 📊 Performance Examples
|
67 |
|
|
|
89 |
|
90 |
Thanks to advanced quantization techniques, Erynn runs efficiently on standard hardware:
|
91 |
|
92 |
+
- **GPU**: NVIDIA GPU RTX 3050 Tİ 4GB VRAM
|
93 |
+
- **CPU**: Intel i7 12700H
|
94 |
+
- **RAM**: 16GB
|
95 |
|
96 |
## 🛠️ Limitations
|
97 |
|