Update README.md

README.md

@@ -1,77 +1,81 @@
 ---
 library_name: transformers
-tags: []
+license: mit
+base_model: deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B
+tags:
+- generated_from_trainer
+- conversational
+- instruction-tuned
+- smoltalk
+datasets:
+- HuggingFaceTB/smoltalk
+language:
+- en
 ---
 
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
+# Model Card for DeepSeek-R1-SmolTalk
 
+This model is a fine-tuned version of [`deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B`](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B) on the [SmolTalk dataset](https://huggingface.co/datasets/HuggingFaceTB/smoltalk). It is optimized for small-scale, friendly, and engaging instruction-following dialogue.
 
 ## Model Details
 
 ### Model Description
 
-<!-- Provide a longer summary of what this model is. -->
-
-This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
+This model builds on DeepSeek's distilled Qwen-1.5B architecture and is trained for conversational tasks using the SmolTalk dataset. The goal is to create a lightweight, instruction-following model suitable for use in chatbots or lightweight assistants with limited hardware resources.
 
-<!-- Provide the basic links for the model. -->
+- **Model type:** Instruction-tuned causal decoder (chat)
+- **Language(s):** English
+- **License:** MIT
+- **Finetuned from model:** deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B
 
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
 
 ## Uses
 
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
 ### Direct Use
 
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+This model can be used as a lightweight assistant or chatbot in applications such as:
 
-[More Information Needed]
+- Embedded conversational interfaces
+- Educational or toy assistants
+- Small devices or local applications
 
-### Downstream Use [optional]
+### Downstream Use
 
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
+The model can be further fine-tuned or integrated into larger conversational systems, especially where resource efficiency is crucial.
 
 ### Out-of-Scope Use
 
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
+- Not suitable for tasks requiring deep factual accuracy or reasoning
+- Should not be used for sensitive or high-stakes decision making
+- Not designed for multilingual use
 
 ## Bias, Risks, and Limitations
 
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+Due to the small model size and dataset limitations:
 
-[More Information Needed]
+- May produce generic or incorrect outputs
+- Can reflect biases present in the training dataset
+- Not guaranteed to be safe for all user demographics or use cases
 
 ### Recommendations
 
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+- Use in controlled or sandboxed environments
+- Consider integrating content moderation or rule-based filtering
+- Do not deploy in contexts requiring factual correctness or ethical judgment
 
 ## How to Get Started with the Model
 
-Use the code below to get started with the model.
+```Python
+from transformers import AutoModelForCausalLM, AutoTokenizer
 
-[More Information Needed]
+model = AutoModelForCausalLM.from_pretrained("your-username/your-model-id")
+tokenizer = AutoTokenizer.from_pretrained("your-username/your-model-id")
+
+input_text = "Hi there! What can you do?"
+inputs = tokenizer(input_text, return_tensors="pt")
+outputs = model.generate(**inputs, max_new_tokens=100)
+print(tokenizer.decode(outputs[0], skip_special_tokens=True))
+```
 
 ## Training Details
 
@@ -79,7 +83,7 @@ Use the code below to get started with the model.
 
 <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
 
-[More Information Needed]
+Used [SmolTalk dataset](https://huggingface.co/datasets/HuggingFaceTB/smoltalk), a dataset of lightweight, instruction-style conversations. The dataset is designed to help models learn concise, friendly, and helpful interactions.
 
 ### Training Procedure
 
@@ -87,12 +91,32 @@ Use the code below to get started with the model.
 
 #### Preprocessing [optional]
 
-[More Information Needed]
+Used the DeepSeek tokenizer
+
+#### LoRA Configuration
 
+- rank: 6
+- alpha: 12
+- dropout: 0.05
+- bias: none
+- target: linear
 
 #### Training Hyperparameters
 
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+The following hyperparameters were used during training:
+
+- learning_rate: 2e-04
+- train_batch_size: 2
+- eval_batch_size: 2
+- seed: 42
+- gradient_accumulation_steps: 2
+- gradient_clipping: 0.3
+- total_train_batch_size: 128
+- optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED
+- lr_scheduler_type: constant
+- lr_scheduler_warmup_ratio: 0.03
+- num_epochs: 1
+- mixed_precision_training: bf16
 
 #### Speeds, Sizes, Times [optional]
 
@@ -196,4 +220,6 @@ Carbon emissions can be estimated using the [Machine Learning Impact calculator]
 
 ## Model Card Contact
 
-[More Information Needed]
+[More Information Needed]
+
+fill this model card