File size: 4,332 Bytes
8561f04 068a5a6 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 |
---
language: en
license: mit
library_name: transformers
pipeline_tag: text-generation
tags:
- text-generation
- conversational
- survey-response-generation
- synthetic-data
- fine-tuned
- chatbot
---
# aryashah00/survey-finetuned-DeepSeek-R1-Distill-Qwen-1.5B
## Model Description
This model is a fine-tuned version of [deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B) optimized for generating synthetic survey responses across multiple domains. It has been instruction-tuned using a custom dataset of survey responses, with each response reflecting a specific persona.
## Training Data
- **Dataset Size**: ~3,000 examples
- **Domains**: 10 domains including healthcare, education, etc.
- **Format**: ChatML instruction format with system and user prompts
## Training Details
- **Base Model**: [deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B)
- **Training Method**: Parameter-Efficient Fine-Tuning with LoRA
- **LoRA Parameters**: r=16, alpha=32, dropout=0.05
- **Training Setup**:
- Batch Size: 8
- Learning Rate: 0.0002
- Epochs: 5
## Usage
This model is specifically designed for generating synthetic survey responses from different personas. It works best when provided with:
1. A detailed persona description
2. A specific survey question
### Python Example
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
# Load model and tokenizer
model = AutoModelForCausalLM.from_pretrained("aryashah00/survey-finetuned-DeepSeek-R1-Distill-Qwen-1.5B", device_map="auto", trust_remote_code=True)
tokenizer = AutoTokenizer.from_pretrained("aryashah00/survey-finetuned-DeepSeek-R1-Distill-Qwen-1.5B", trust_remote_code=True)
# Define persona and question
persona = "A nurse who educates the child about modern medical treatments and encourages a balanced approach to healthcare"
question = "How often was your pain well controlled during this hospital stay?"
# Prepare prompts
system_prompt = f"You are embodying the following persona: {{persona}}"
user_prompt = f"Survey Question: {{question}}\n\nPlease provide your honest and detailed response to this question."
# Create message format
messages = [
{"role": "system", "content": system_prompt},
{"role": "user", "content": user_prompt}
]
# Apply chat template
input_text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
# Tokenize
input_ids = tokenizer(input_text, return_tensors="pt").input_ids.to(model.device)
# Generate response
import torch
with torch.no_grad():
output_ids = model.generate(
input_ids=input_ids,
max_new_tokens=256,
temperature=0.7,
top_p=0.9,
do_sample=True
)
# Decode
output = tokenizer.decode(output_ids[0], skip_special_tokens=True)
# Extract just the generated response
response_start = output.find(input_text) + len(input_text)
generated_response = output[response_start:].strip()
print(f"Generated response: {{generated_response}}")
```
### Inference API Example
```python
import requests
API_URL = "https://api-inference.huggingface.co/models/aryashah00/survey-finetuned-DeepSeek-R1-Distill-Qwen-1.5B"
headers = {"Authorization": "Bearer YOUR_API_KEY"}
def query(payload):
response = requests.post(API_URL, headers=headers, json=payload)
return response.json()
messages = [
{"role": "system", "content": "You are embodying the following persona: A nurse who educates the child about modern medical treatments and encourages a balanced approach to healthcare"},
{"role": "user", "content": "Survey Question: How often was your pain well controlled during this hospital stay?\n\nPlease provide your honest and detailed response to this question."}
]
output = query({"inputs": messages})
print(output)
```
## Limitations
- The model is optimized for survey response generation and may not perform well on other tasks
- Response quality depends on the clarity and specificity of the persona and question
- The model may occasionally generate responses that don't fully align with the given persona
## License
This model follows the license of the base model [deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B).
|