Updated readme

README.md

@@ -1,7 +1,7 @@
 ---
 base_model: unsloth/llama-3.2-3b-instruct-unsloth-bnb-4bit
 library_name: peft
-pipeline_tag: text-generation
+pipeline_tag: summarization
 tags:
 - base_model:adapter:unsloth/llama-3.2-3b-instruct-unsloth-bnb-4bit
 - lora
@@ -9,202 +9,191 @@ tags:
 - transformers
 - trl
 - unsloth
+license: apache-2.0
+language:
+- en
 ---
 
-# Model Card for Model ID
+# STL Phone Summarizer
 
-<!-- Provide a quick summary of what the model is/does. -->
+A conversational LLM for summarizing phone specifications into concise, appealing descriptions for e-commerce.  
+**Model:** LoRA fine-tuned Llama-3.2  
+**Repo:** [`masabhuq/stl_phone_summarizer`](https://huggingface.co/masabhuq/stl_phone_summarizer)
 
+---
 
+## Installation
 
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
+```bash
+pip install unsloth torch
+```
 
-[More Information Needed]
+---
 
-### Compute Infrastructure
+## Usage
+
+### 1. Load Model and Tokenizer
+
+```python
+from unsloth import FastLanguageModel
+from unsloth.chat_templates import get_chat_template
+
+model, tokenizer = FastLanguageModel.from_pretrained(
+    "masabhuq/stl_phone_summarizer",
+    max_seq_length=2048,
+    dtype=None,          # Auto-detect (bfloat16 if supported)
+    load_in_4bit=True,   # 4-bit quantization for memory efficiency
+)
+FastLanguageModel.for_inference(model)
+```
+
+### 2. Apply the Chat Template
+
+```python
+tokenizer = get_chat_template(
+    tokenizer,
+    chat_template="llama-3.2",
+    map_eos_token=True,
+)
+```
+
+### 3. Prepare the Input
+
+```python
+system_prompt = (
+    "You are an expert at summarizing phone specifications into short, appealing key descriptions for an e-commerce site. "
+    "Always output in exactly this format:\n"
+    "Display: [concise display summary]\n"
+    "Processor: [processor name]\n"
+    "Camera: [camera highlights]\n"
+    "Battery: [battery capacity and charging]\n"
+    "Others: [comma-separated unique features]. "
+    "Focus on desirable aspects like high refresh rates, zoom capabilities, fast charging, and unique features such as water resistance or special sensors. "
+    "Do not include complicated keyboards that don't make sense on their own. "
+    "Do not include words that are too technical to understand for someone who is not highly tech savvy. "
+    "Output should be within 280 characters. Don't include anything like IPDC or IP64 or any such features in the result. Words starting with IP are not to be considered display feature."
+)
+
+specs = "Build: Glass front (Gorilla Glass 5), silicone polymer back (eco leather), plastic frame\nWeight: 178 g ..."
+
+prompt = [
+    {"role": "system", "content": system_prompt},
+    {"role": "user", "content": specs}
+]
+
+formatted_prompt = tokenizer.apply_chat_template(
+    prompt,
+    tokenize=False,
+    add_generation_prompt=True,
+)
+```
+
+### 4. Tokenize and Generate
+
+```python
+import torch
+
+inputs = tokenizer(formatted_prompt, return_tensors="pt").to("cuda")
+outputs = model.generate(
+    **inputs,
+    max_new_tokens=256,
+    do_sample=True,
+    temperature=0.7,
+    top_p=0.9,
+)
+```
+
+### 5. Post-process Output
+
+```python
+generated_text = tokenizer.decode(outputs[0], skip_special_tokens=False)
+# Extract the last paragraph and clean up
+paragraphs = generated_text.strip().split("\n\n")
+last_paragraph = paragraphs[-1]
+clean_last_paragraph = last_paragraph.split("<|eot_id|>")[0].strip()
+print(clean_last_paragraph)
+```
+### 6. Clean Up
+
+Free GPU memory after inference:
+
+```python
+model.cpu()
+torch.cuda.empty_cache()
+```
 
-[More Information Needed]
+---
+## Hardware Requirements
 
-#### Hardware
+- **GPU**: CUDA-compatible GPU with ~4-6GB VRAM for 4-bit inference.
+- **CPU**: Optional for offloading model after inference (`model.cpu()`).
+- **RAM**: ~8GB system RAM for smooth operation with dataset processing.
 
-[More Information Needed]
 
-#### Software
+---
 
-[More Information Needed]
+## Notes
 
-## Citation [optional]
+- **Chat Template:** The tokenizer is uploaded without a chat template. Always apply the template at runtime as shown above.
+- **System Prompt:** Adjust the system prompt for your use case.
+- **Output Format:** The model is trained to output in a strict format for easy parsing.
+- **Memory Management**: Use `model.cpu()` and `torch.cuda.empty_cache()` to free GPU memory after inference, especially on low-VRAM GPUs.
+- **Inference Parameters**: Adjust `temperature` and `top_p` for more or less creative outputs, and `max_new_tokens` for longer or shorter summaries.
 
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+---
+## Model Details
 
-**BibTeX:**
+- **Base Model**: `unsloth/Llama-3.2-3B-Instruct-bnb-4bit`
+- **Fine-Tuning**: LoRA adapters with rank `r=16`, targeting modules: `["q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj"]`.
+- **Quantization**: 4-bit for memory efficiency (~4-6GB VRAM).
+- **Training Data**: A dataset of phone specifications (`specs`) paired with concise summaries (`output`) in the format shown above.
+- **Training Setup**: Fine-tuned with `trl.SFTTrainer`, `train_on_responses_only` to focus on assistant responses, and Llama-3.2 chat template for single-turn interactions.
+- **Output Constraints**: Summaries are limited to 280 characters, focusing on user-friendly features and avoiding technical terms like "IP68" or "IPDC".
 
-[More Information Needed]
+---
 
-**APA:**
+## Dataset
 
-[More Information Needed]
+The model was trained on a custom dataset (`specs_list.json`) containing pairs of detailed phone specifications and their corresponding summaries. Each entry includes:
+- `specs`: Detailed technical specs (e.g., display size, chipset, camera details).
+- `output`: A concise summary in the format:
+  ```
+  Display: [summary]
+  Processor: [name]
+  Camera: [highlights]
+  Battery: [capacity and charging]
+  Others: [features]
+  ```
+The dataset emphasizes consumer-friendly features like high refresh rates, fast charging, and water resistance, avoiding overly technical terms.
 
-## Glossary [optional]
+---
 
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+## License
 
-[More Information Needed]
+This model is licensed under the [Apache 2.0 License](LICENSE). See the `LICENSE` file in the repository for details.
 
-## More Information [optional]
+---
 
-[More Information Needed]
+## Citation
 
-## Model Card Authors [optional]
+If you use this model, please cite the repository:
 
-[More Information Needed]
+```bibtex
+@misc{stl_phone_summarizer,
+  author = {masabhuq},
+  title = {STL Phone Summarizer: A Fine-Tuned Llama-3.2 Model for Phone Specification Summaries},
+  year = {2025},
+  publisher = {Hugging Face},
+  howpublished = {\url{https://huggingface.co/masabhuq/stl_phone_summarizer}}
+}
+```
+### 6. Clean Up
 
-## Model Card Contact
+Free GPU memory after inference:
 
-[More Information Needed]
-### Framework versions
+```python
+model.cpu()
+torch.cuda.empty_cache()
+```
 
-- PEFT 0.17.1
+---