lunahr commited on
Commit
3229381
·
verified ·
1 Parent(s): a37047e

use chat completions instead of base model syntax

Browse files
Files changed (1) hide show
  1. README.md +16 -15
README.md CHANGED
@@ -194,12 +194,12 @@ Phi-4-mini-instruct is also available in [Azure AI Studio]()
194
 
195
  #### Example
196
 
197
- After obtaining the Phi-4-mini-instruct model checkpoints, users can use this sample code for inference.
198
 
199
  ```python
200
  import torch
201
- from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline
202
-
203
  torch.random.manual_seed(0)
204
 
205
  model_path = "microsoft/Phi-4-mini-instruct"
@@ -211,29 +211,30 @@ model = AutoModelForCausalLM.from_pretrained(
211
  trust_remote_code=True,
212
  )
213
  tokenizer = AutoTokenizer.from_pretrained(model_path)
214
-
215
  messages = [
216
  {"role": "system", "content": "You are a helpful AI assistant."},
217
  {"role": "user", "content": "Can you provide ways to eat combinations of bananas and dragonfruits?"},
218
  {"role": "assistant", "content": "Sure! Here are some ways to eat bananas and dragonfruits together: 1. Banana and dragonfruit smoothie: Blend bananas and dragonfruits together with some milk and honey. 2. Banana and dragonfruit salad: Mix sliced bananas and dragonfruits together with some lemon juice and honey."},
219
  {"role": "user", "content": "What about solving an 2x + 3 = 7 equation?"},
220
  ]
221
-
222
- pipe = pipeline(
223
- "text-generation",
224
- model=model,
225
- tokenizer=tokenizer,
226
- )
227
-
228
  generation_args = {
229
  "max_new_tokens": 500,
230
- "return_full_text": False,
231
  "temperature": 0.0,
232
  "do_sample": False,
233
  }
234
-
235
- output = pipe(messages, **generation_args)
236
- print(output[0]['generated_text'])
 
 
 
 
237
  ```
238
 
239
  ## Responsible AI Considerations
 
194
 
195
  #### Example
196
 
197
+ After obtaining the Phi-4-mini-instruct chat models, users can use this sample code for inference.
198
 
199
  ```python
200
  import torch
201
+ from transformers import AutoModelForCausalLM, AutoTokenizer
202
+
203
  torch.random.manual_seed(0)
204
 
205
  model_path = "microsoft/Phi-4-mini-instruct"
 
211
  trust_remote_code=True,
212
  )
213
  tokenizer = AutoTokenizer.from_pretrained(model_path)
214
+
215
  messages = [
216
  {"role": "system", "content": "You are a helpful AI assistant."},
217
  {"role": "user", "content": "Can you provide ways to eat combinations of bananas and dragonfruits?"},
218
  {"role": "assistant", "content": "Sure! Here are some ways to eat bananas and dragonfruits together: 1. Banana and dragonfruit smoothie: Blend bananas and dragonfruits together with some milk and honey. 2. Banana and dragonfruit salad: Mix sliced bananas and dragonfruits together with some lemon juice and honey."},
219
  {"role": "user", "content": "What about solving an 2x + 3 = 7 equation?"},
220
  ]
221
+
222
+ # Prepare the input for the model
223
+ input_ids = tokenizer.apply_chat_template(messages, return_tensors="pt").to(model.device)
224
+
225
+ # Generate the response
 
 
226
  generation_args = {
227
  "max_new_tokens": 500,
 
228
  "temperature": 0.0,
229
  "do_sample": False,
230
  }
231
+
232
+ output_ids = model.generate(input_ids, **generation_args)
233
+
234
+ # Decode the generated tokens to text
235
+ response = tokenizer.decode(output_ids[0], skip_special_tokens=True)
236
+
237
+ print(response)
238
  ```
239
 
240
  ## Responsible AI Considerations