Update README.md

README.md

@@ -2,7 +2,7 @@
 license: gemma
 language: en
 library_name: transformers
-base_model: google/gemma-3-1b-it
+base_model: unsloth/gemma-3-1b-it
 datasets:
 - iamtarun/code_instructions_120k_alpaca
 tags:
@@ -11,18 +11,19 @@ tags:
 - instruction-following
 - code
 - alpaca
+- unsloth
 ---
 
 # Gemma-3-1B-Code-Alpaca-FT
 
-This is a fine-tuned version of Google's `google/gemma-3-1b-it` model. It was fine-tuned by [prathmesh_nik](https://huggingface.co/prathmesh_nik) on the [`iamtarun/code_instructions_120k_alpaca`](https://huggingface.co/datasets/iamtarun/code_instructions_120k_alpaca) dataset to enhance its ability to follow instructions related to code generation and general programming tasks.
+This is a fine-tuned version of the `unsloth/gemma-3-1b-it` model. It was fine-tuned by [prathmesh-nik](https://huggingface.co/prathmesh-nik) on the [`iamtarun/code_instructions_120k_alpaca`](https://huggingface.co/datasets/iamtarun/code_instructions_120k_alpaca) dataset to enhance its ability to follow instructions related to code generation and general programming tasks.
 
 ## Model Details
 
 ### Technical Specifications
 
-- **Context Window:** 8192 tokens
-- **Max New Tokens:** 256 tokens (default generation limit)
+- **Context Window:** The model's configuration (`config.json`) specifies a theoretical maximum context of 32,768 tokens. However, it is based on `unsloth/gemma-3-1b-it`, which is optimized for an **8192 token** context. Furthermore, this fine-tune was performed with a `max_seq_length` of only **2048 tokens**. For best results, it is recommended to use this model with sequences close to the 2048 token training length.
+- **Max New Tokens:** This is a parameter set during inference, not an intrinsic property of the model. The example code uses a default, but you can set this value higher or lower based on your needs (up to the context window limit).
 
 ### Intended Use
 
@@ -63,7 +64,11 @@ print(tokenizer.decode(outputs[0], skip_special_tokens=False))
 
 ### Limitations and Bias
 
-This model inherits the limitations and biases of the base `google/gemma-3-1b-it` model and the `iamtarun/code_instructions_120k_alpaca` dataset. As a model fine-tuned on a specific dataset, it may not be as effective for general-purpose tasks and may not always provide correct answers. It may generate incorrect or insecure code and should not be used for mission-critical applications without human oversight. Users are encouraged to implement their own safety and evaluation procedures.
+This model inherits the limitations and biases of the base `unsloth/gemma-3-1b-it` model and the training dataset.
+
+- **Short Context Training:** Since the model was fine-tuned on sequences of only 2048 tokens, its performance on tasks requiring a very long context (i.e., approaching the 8192 token limit) may be degraded.
+- **Quantization-Aware Training:** The adapters were trained on a 4-bit model. While the final model is full-precision, this process can introduce subtle differences compared to training in full precision.
+- **General Limitations:** As a model fine-tuned on a specific dataset, it may not be as effective for general-purpose tasks and may not always provide correct answers. It may generate incorrect or insecure code and should not be used for mission-critical applications without human oversight.
 
 ## Fine-tuning Details
 
@@ -73,7 +78,17 @@ This model was fine-tuned on the [`iamtarun/code_instructions_120k_alpaca`](http
 
 ### Training Procedure
 
-The model was fine-tuned using the LoRA (Low-Rank Adaptation) methodology for efficient parameter tuning. The training was performed using the [Unsloth](https://github.com/unslothai/unsloth) library to optimize for memory usage and training speed.
+The model was fine-tuned using the LoRA (Low-Rank Adaptation) methodology. Key aspects of the training include:
+
+- **Unsloth Optimization:** The training was performed using the [Unsloth](https://github.com/unslothai/unsloth) library, which enables significantly faster training and lower memory usage.
+- **4-bit Quantization:** For maximum efficiency, the LoRA adapters were trained on a 4-bit quantized version of the base model. The final merged model is in full precision (`bfloat16`).
+- **Parameter-Efficient Fine-Tuning (PEFT):** Only a small fraction of the model's parameters (the LoRA adapters) were trained, making the process highly efficient.
+
+## Repository and Training Code
+
+This model was fine-tuned by [prathmesh_nik](https://huggingface.co/prathmesh-nik). The complete source code for the fine-tuning process, including data preparation, training, and merging scripts, is available on GitHub.
+
+- **GitHub Repository:** [https://github.com/prathmeshnik]
 
 ## Original Model Citation