---
datasets:
- flytech/python-codes-25k
language:
- en
- tr
base_model:
- Qwen/Qwen3-0.6B
pipeline_tag: text-generation
library_name: transformers
tags:
- code
- text-generation-inference
---

# haydarkadioglu/Qwen3-0.6B-lora-python-expert-fine-tuned

Qwen 0.6B LoRA fine-tuned for Python expert tasks

## Training Notebook (Google Colab)

You can reproduce the fine-tuning process or adapt it for your own dataset using the Colab notebook:  
👉 [Open in Google Colab](https://colab.research.google.com/drive/17mU5LFWT6JQ5uDI8FGGugyEkKnykw4Xj?usp=sharing)

## Model Details
- **Model type:** Qwen 0.6B LoRA
- **Base model:** Qwen/Qwen-0.6B
- **Fine-tuned by:** @haydarkadioglu
- **Language(s):** English, Python


## Intended Use
- **Primary use case:** Code generation, Python expert help
- **Not suitable for:** General conversation, non-Python coding tasks

## Training Details
- **Dataset:** flytech/python-codes-25k
- **Steps / Epochs:** 3 epochs, batch size 8
- **Hardware:** A100 GPU / Colab T4
- **Fine-tuning method:** LoRA / PEFT

## Evaluation
| Step         | Training Loss |
| ------------ | ------------- |
| 100          | 1.8288        |
| 500          | 1.7133        |
| 1000         | 1.5976        |
| 1500         | 1.6438        |
| 2000         | 1.5797        |
| 2500         | 1.5619        |
| 3000         | 1.6235        |
| Final (3102) | **1.6443**    |

Final Results:
Training loss (avg): 1.64
Steps/sec: 0.645
Samples/sec: 10.3
FLOPs: 5.31e15

## Limitations
- The model might produce incorrect or insecure code.
- Not guaranteed to follow PEP8.
- May hallucinate libraries or functions.

## Example Usage
```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "haydarkadioglu/Qwen3-0.6B-lora-python-expert-fine-tuned"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)

prompt = "Write a Python function, this function should return prime numbers between 0-100"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=200)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))