Text Generation
Transformers
Safetensors
English
llama
text-generation-inference
phi3
phi
conversational
Instructions to use prithivMLmods/Phi-4-QwQ with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use prithivMLmods/Phi-4-QwQ with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="prithivMLmods/Phi-4-QwQ") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("prithivMLmods/Phi-4-QwQ") model = AutoModelForCausalLM.from_pretrained("prithivMLmods/Phi-4-QwQ", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use prithivMLmods/Phi-4-QwQ with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "prithivMLmods/Phi-4-QwQ" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "prithivMLmods/Phi-4-QwQ", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/prithivMLmods/Phi-4-QwQ
- SGLang
How to use prithivMLmods/Phi-4-QwQ with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "prithivMLmods/Phi-4-QwQ" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "prithivMLmods/Phi-4-QwQ", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "prithivMLmods/Phi-4-QwQ" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "prithivMLmods/Phi-4-QwQ", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use prithivMLmods/Phi-4-QwQ with Docker Model Runner:
docker model run hf.co/prithivMLmods/Phi-4-QwQ
Create README.md
Browse files
README.md
ADDED
|
@@ -0,0 +1,127 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: mit
|
| 3 |
+
language:
|
| 4 |
+
- en
|
| 5 |
+
base_model:
|
| 6 |
+
- microsoft/phi-4
|
| 7 |
+
pipeline_tag: text-generation
|
| 8 |
+
library_name: transformers
|
| 9 |
+
tags:
|
| 10 |
+
- text-generation-inference
|
| 11 |
+
- llama
|
| 12 |
+
- phi3
|
| 13 |
+
- phi
|
| 14 |
+
---
|
| 15 |
+

|
| 16 |
+
|
| 17 |
+
# **Phi-4-QwQ [ Responsible Problem Solving & Advanced Reasoning ]**
|
| 18 |
+
|
| 19 |
+
`[Phi-4-QwQ finetuned]` from Microsoft's Phi-4 is a state-of-the-art open model developed with a focus on **responsible problem solving** and **advanced reasoning capabilities**. Built upon a diverse blend of synthetic datasets, carefully filtered public domain websites, and high-quality academic books and Q&A datasets, Phi-4-QwQ ensures that small, capable models are trained with datasets of exceptional depth and precision.
|
| 20 |
+
|
| 21 |
+
Phi-4-QwQ adopts a robust **safety post-training approach** using open-source and in-house synthetic datasets. This involves a combination of **SFT (Supervised Fine-Tuning)** and iterative **DPO (Direct Preference Optimization)** techniques, ensuring helpful and harmless outputs across various safety categories.
|
| 22 |
+
|
| 23 |
+
---
|
| 24 |
+
|
| 25 |
+
# **Dataset Info**
|
| 26 |
+
|
| 27 |
+
Phi-4-QwQ is fine-tuned on a carefully curated synthetic dataset generated using an advanced pipeline optimized for **Chain of Thought (CoT)** reasoning and **Responsible Problem Breakdown (RPB)** methodologies. This ensures that the model excels at:
|
| 28 |
+
|
| 29 |
+
- **Logical reasoning**
|
| 30 |
+
- **Step-by-step problem-solving**
|
| 31 |
+
- **Breaking down complex tasks into manageable parts**
|
| 32 |
+
|
| 33 |
+
The dataset also emphasizes responsible decision-making and fairness in generating solutions.
|
| 34 |
+
|
| 35 |
+
---
|
| 36 |
+
|
| 37 |
+
# **Run with Transformers**
|
| 38 |
+
|
| 39 |
+
```python
|
| 40 |
+
# pip install accelerate
|
| 41 |
+
from transformers import AutoTokenizer, AutoModelForCausalLM
|
| 42 |
+
import torch
|
| 43 |
+
|
| 44 |
+
tokenizer = AutoTokenizer.from_pretrained("prithivMLmods/Phi-4-QwQ")
|
| 45 |
+
model = AutoModelForCausalLM.from_pretrained(
|
| 46 |
+
"prithivMLmods/Phi-4-QwQ",
|
| 47 |
+
device_map="auto",
|
| 48 |
+
torch_dtype=torch.bfloat16,
|
| 49 |
+
)
|
| 50 |
+
|
| 51 |
+
input_text = "Explain the concept of black holes."
|
| 52 |
+
input_ids = tokenizer(input_text, return_tensors="pt").to("cuda")
|
| 53 |
+
|
| 54 |
+
outputs = model.generate(**input_ids, max_new_tokens=64)
|
| 55 |
+
print(tokenizer.decode(outputs[0]))
|
| 56 |
+
```
|
| 57 |
+
|
| 58 |
+
For chat-style interactions, use `tokenizer.apply_chat_template`:
|
| 59 |
+
|
| 60 |
+
```python
|
| 61 |
+
messages = [
|
| 62 |
+
{"role": "user", "content": "Explain the concept of black holes."},
|
| 63 |
+
]
|
| 64 |
+
input_ids = tokenizer.apply_chat_template(messages, return_tensors="pt", return_dict=True).to("cuda")
|
| 65 |
+
|
| 66 |
+
outputs = model.generate(**input_ids, max_new_tokens=256)
|
| 67 |
+
print(tokenizer.decode(outputs[0]))
|
| 68 |
+
```
|
| 69 |
+
|
| 70 |
+
# **Intended Use**
|
| 71 |
+
|
| 72 |
+
Phi-4-QwQ is tailored for a wide range of applications, especially those involving **advanced reasoning**, **multilingual capabilities**, and **responsible problem-solving**. Its primary use cases include:
|
| 73 |
+
|
| 74 |
+
1. **Responsible Problem Solving**
|
| 75 |
+
- Breaking down complex problems into logical, actionable steps.
|
| 76 |
+
- Offering ethical, well-rounded solutions in academic and professional contexts.
|
| 77 |
+
|
| 78 |
+
2. **Advanced Reasoning Tasks**
|
| 79 |
+
- Excelling in mathematics, logic, and scientific reasoning.
|
| 80 |
+
- Providing detailed explanations and systematic answers.
|
| 81 |
+
|
| 82 |
+
3. **Content Generation**
|
| 83 |
+
- Assisting in generating high-quality content for various domains, including creative writing and technical documentation.
|
| 84 |
+
- Supporting marketers, writers, and educators with detailed and well-structured outputs.
|
| 85 |
+
|
| 86 |
+
4. **Educational Support**
|
| 87 |
+
- Acting as a virtual tutor for students by generating practice questions, answers, and detailed explanations.
|
| 88 |
+
- Helping educators design learning material that promotes critical thinking and step-by-step problem-solving.
|
| 89 |
+
|
| 90 |
+
5. **Customer Support & Dialogue Systems**
|
| 91 |
+
- Enabling chatbots and virtual assistants to provide accurate, helpful, and responsible responses.
|
| 92 |
+
- Enhancing customer service with reasoning-driven automation.
|
| 93 |
+
|
| 94 |
+
6. **Multilingual Capabilities**
|
| 95 |
+
- Supporting multilingual communication and content generation while maintaining contextual accuracy.
|
| 96 |
+
- Assisting in translations with a focus on retaining meaning and nuance.
|
| 97 |
+
|
| 98 |
+
7. **Safety-Critical Applications**
|
| 99 |
+
- Ensuring safe and harmless outputs, making it suitable for sensitive domains.
|
| 100 |
+
- Providing aligned interactions with human oversight for critical systems.
|
| 101 |
+
|
| 102 |
+
---
|
| 103 |
+
|
| 104 |
+
# **Limitations**
|
| 105 |
+
|
| 106 |
+
Despite its strengths, Phi-4-QwQ has some limitations that users should be aware of:
|
| 107 |
+
|
| 108 |
+
1. **Bias and Fairness**
|
| 109 |
+
- While great effort has been made to minimize biases, users should critically assess the model’s output in sensitive scenarios to avoid unintended bias.
|
| 110 |
+
|
| 111 |
+
2. **Contextual Interpretation**
|
| 112 |
+
- The model may occasionally misinterpret highly nuanced prompts or ambiguous contexts, leading to suboptimal responses.
|
| 113 |
+
|
| 114 |
+
3. **Knowledge Cutoff**
|
| 115 |
+
- Phi-4-QwQ’s knowledge is static and based on the data available at the time of training. It does not include real-time updates or information on recent developments.
|
| 116 |
+
|
| 117 |
+
4. **Safety and Harmlessness**
|
| 118 |
+
- Despite post-training safety alignment, inappropriate or harmful outputs may still occur. Continuous monitoring and human oversight are advised when using the model in critical contexts.
|
| 119 |
+
|
| 120 |
+
5. **Computational Requirements**
|
| 121 |
+
- Deploying Phi-4-QwQ efficiently may require substantial computational resources, particularly for large-scale deployments or real-time applications.
|
| 122 |
+
|
| 123 |
+
6. **Ethical Considerations**
|
| 124 |
+
- Users are responsible for ensuring that the model is not employed for malicious purposes, such as spreading misinformation, generating harmful content, or facilitating unethical behavior.
|
| 125 |
+
|
| 126 |
+
7. **Domain-Specific Expertise**
|
| 127 |
+
- While the model is versatile, it may not perform optimally in highly specialized domains (e.g., law, medicine, finance) without further domain-specific fine-tuning.
|