Update README.md
Browse files
README.md
CHANGED
|
@@ -1,82 +1,90 @@
|
|
| 1 |
-
|
| 2 |
-
|
| 3 |
-
|
| 4 |
-
|
| 5 |
-
|
| 6 |
-
|
| 7 |
-
|
| 8 |
-
|
| 9 |
-
|
| 10 |
-
|
| 11 |
-
|
| 12 |
-
|
| 13 |
-
|
| 14 |
-
|
| 15 |
-
|
| 16 |
-
|
| 17 |
-
|
| 18 |
-
|
| 19 |
-
|
| 20 |
-
|
| 21 |
-
|
| 22 |
-
|
| 23 |
-
|
| 24 |
-
|
| 25 |
-
|
| 26 |
-
|
| 27 |
-
|
| 28 |
-
|
| 29 |
-
|
| 30 |
-
|
| 31 |
-
|
| 32 |
-
|
| 33 |
-
|
| 34 |
-
|
| 35 |
-
|
| 36 |
-
|
| 37 |
-
#
|
| 38 |
-
|
| 39 |
-
|
| 40 |
-
|
| 41 |
-
|
| 42 |
-
|
| 43 |
-
|
| 44 |
-
|
| 45 |
-
|
| 46 |
-
|
| 47 |
-
|
| 48 |
-
|
| 49 |
-
|
| 50 |
-
|
| 51 |
-
|
| 52 |
-
|
| 53 |
-
|
| 54 |
-
|
| 55 |
-
|
| 56 |
-
|
| 57 |
-
|
| 58 |
-
|
| 59 |
-
|
| 60 |
-
|
| 61 |
-
|
| 62 |
-
|
| 63 |
-
|
| 64 |
-
|
| 65 |
-
|
| 66 |
-
|
| 67 |
-
|
| 68 |
-
|
| 69 |
-
|
| 70 |
-
|
| 71 |
-
|
| 72 |
-
|
| 73 |
-
|
| 74 |
-
|
| 75 |
-
|
| 76 |
-
|
| 77 |
-
|
| 78 |
-
|
| 79 |
-
|
| 80 |
-
|
| 81 |
-
|
| 82 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: mit
|
| 3 |
+
datasets:
|
| 4 |
+
- wangkevin02/LMSYS-USP
|
| 5 |
+
language:
|
| 6 |
+
- en
|
| 7 |
+
base_model:
|
| 8 |
+
- meta-llama/Meta-Llama-3-8B
|
| 9 |
+
---
|
| 10 |
+
# User Simulator Model
|
| 11 |
+
|
| 12 |
+
## Overview
|
| 13 |
+
|
| 14 |
+
Our User Simulator with Implicit Profiles (USP) replicates human-like conversational behavior in interactions with large language models (LLMs). By emulating diverse user dynamics based on predefined profiles, it reconstructs realistic user-LLM dialogues, leveraging the LLaMA-3-base-8B architecture with **Conditional Supervised Fine-Tuning (SFT)** and **Reinforcement Learning with Cycle Consistency (RLCC)**. For a detailed methodology and insights, refer to [Our Paper](#).
|
| 15 |
+
|
| 16 |
+
## Constraints
|
| 17 |
+
|
| 18 |
+
1. **Maximum Context Length**: Supports up to **4,096 tokens**. Exceeding this may degrade performance; keep inputs within this limit for best results.
|
| 19 |
+
|
| 20 |
+
1. **Language Limitation**: Optimized for English. Non-English performance may vary due to limited training data.
|
| 21 |
+
|
| 22 |
+
|
| 23 |
+
## Sample Implementation
|
| 24 |
+
|
| 25 |
+
Below is a concise example of using the User Simulator to generate user utterances, ideal for recreating scenarios or evaluating multi-turn dialogues based on specific profiles.
|
| 26 |
+
|
| 27 |
+
```python
|
| 28 |
+
import torch
|
| 29 |
+
from transformers import AutoTokenizer, AutoModelForCausalLM
|
| 30 |
+
# Define the model path
|
| 31 |
+
model_path = "/your/path/to/USP"
|
| 32 |
+
|
| 33 |
+
# Initialize tokenizer with specific configurations
|
| 34 |
+
tokenizer = AutoTokenizer.from_pretrained(model_path, padding_side="left", trust_remote_code=True)
|
| 35 |
+
tokenizer.pad_token = tokenizer.eos_token
|
| 36 |
+
|
| 37 |
+
# Define model configuration
|
| 38 |
+
model_config = {
|
| 39 |
+
"pretrained_model_name_or_path": model_path,
|
| 40 |
+
"trust_remote_code": True,
|
| 41 |
+
"torch_dtype": torch.bfloat16,
|
| 42 |
+
"device_map": "cuda",
|
| 43 |
+
"attn_implementation": "flash_attention_2"
|
| 44 |
+
}
|
| 45 |
+
|
| 46 |
+
# Load the pre-trained model with specified configurations
|
| 47 |
+
model = AutoModelForCausalLM.from_pretrained(**model_config)
|
| 48 |
+
model.eval()
|
| 49 |
+
|
| 50 |
+
# Define conversation messages
|
| 51 |
+
messages = [
|
| 52 |
+
[
|
| 53 |
+
{"role": "system", "content": "You are engaging in a conversation with an AI assistant. Your profile is: \nYou have a knack for planning exciting adventures, particularly when it comes to exploring new destinations on a budget. Your current focus is on organizing a cost-effective vacation to a warm, beach-filled location. You're actively seeking recommendations for such a getaway, with a particular interest in places like Phuket, Thailand. You're keen on discovering the must-visit spots in Phuket without breaking the bank, and you're looking for advice on how to make the most of your trip within your budget constraints. Your love for travel is evident in your habit of meticulously planning vacations in advance, ensuring you maximize both the experience and the value for money.\n\nYour personality shines through in your conscientious approach to planning, where every detail is considered and nothing is left to chance. You're open-minded and adventurous, always eager to dive into new experiences and embrace what each destination has to offer. Your inquisitive nature means you're always asking questions, seeking out the best advice to enhance your journeys. You communicate with an informal and friendly style, making it easy for others to share their knowledge and insights with you. This combination of traits makes you not only a savvy traveler but also a delightful companion on any adventure.\n You can say anything you want, either based on the profile or something brand new.\n\n"},
|
| 54 |
+
],
|
| 55 |
+
[
|
| 56 |
+
{"role": "system", "content": "You are engaging in a conversation with an AI assistant. Your profile is: \nYou have a knack for planning exciting adventures, particularly when it comes to exploring new destinations on a budget. Your current focus is on organizing a cost-effective vacation to a warm, beach-filled location. You're actively seeking recommendations for such a getaway, with a particular interest in places like Phuket, Thailand. You're keen on discovering the must-visit spots in Phuket without breaking the bank, and you're looking for advice on how to make the most of your trip within your budget constraints. Your love for travel is evident in your habit of meticulously planning vacations in advance, ensuring you maximize both the experience and the value for money.\n\nYour personality shines through in your conscientious approach to planning, where every detail is considered and nothing is left to chance. You're open-minded and adventurous, always eager to dive into new experiences and embrace what each destination has to offer. Your inquisitive nature means you're always asking questions, seeking out the best advice to enhance your journeys. You communicate with an informal and friendly style, making it easy for others to share their knowledge and insights with you. This combination of traits makes you not only a savvy traveler but also a delightful companion on any adventure.\n You can say anything you want, either based on the profile or something brand new.\n\n"},
|
| 57 |
+
{"role": "user", "content": "I want to go on vacation to a warm place. Do you have any recommendations?"},
|
| 58 |
+
{"role": "assistant", "content": "Sure! If you like beaches, Maldives or Bali are great options. If you're into culture, consider Tuscany in Italy or Santorini in Greece. Which type of destination do you prefer?"},
|
| 59 |
+
]
|
| 60 |
+
|
| 61 |
+
]
|
| 62 |
+
|
| 63 |
+
|
| 64 |
+
def generated_user_utt(msgs, model, tokenizer):
|
| 65 |
+
with torch.no_grad():
|
| 66 |
+
input_text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
|
| 67 |
+
inputs = tokenizer.batch_encode_plus(input_text, return_tensors="pt", truncation=True, padding=True, max_length=4096).to(model.device)
|
| 68 |
+
input_ids = inputs["input_ids"]
|
| 69 |
+
outputs = model.generate(
|
| 70 |
+
**inputs,
|
| 71 |
+
pad_token_id=tokenizer.eos_token_id,
|
| 72 |
+
eos_token_id=tokenizer.eos_token_id,
|
| 73 |
+
max_new_tokens=4096,
|
| 74 |
+
repetition_penalty=1.2)
|
| 75 |
+
generated_ids = [
|
| 76 |
+
output[len(input_):]
|
| 77 |
+
for input_, output in zip(input_ids, outputs)
|
| 78 |
+
]
|
| 79 |
+
response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)
|
| 80 |
+
return response
|
| 81 |
+
|
| 82 |
+
# Generate responses for the provided messages
|
| 83 |
+
responses = generated_user_utt(messages, model, tokenizer)
|
| 84 |
+
print(f"Generated response: {responses}")
|
| 85 |
+
|
| 86 |
+
# >>> Expected Output:
|
| 87 |
+
# >>> ['I am thinking about going away for vacation', 'How about phucket thailand']
|
| 88 |
+
# >>> The first output initiates a topic based on the system-defined user profile.
|
| 89 |
+
# >>> The second output extends the conversation based on the given profile and context to mimic the user's behavior.
|
| 90 |
+
```
|