Upload folder using huggingface_hub

README.md

@@ -0,0 +1,131 @@
+---
+library_name: peft
+license: gemma
+base_model: google/gemma-3-1b-it
+tags:
+- generated_from_trainer
+model-index:
+- name: mnt/d/druban/axolotl/try2
+  results: []
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+[<img src="https://raw.githubusercontent.com/axolotl-ai-cloud/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/axolotl-ai-cloud/axolotl)
+<details><summary>See axolotl config</summary>
+
+axolotl version: `0.8.0`
+```yaml
+base_model: google/gemma-3-1b-it
+model_type: gemma
+tokenizer_type: AutoTokenizer
+trust_remote_code: true
+
+load_in_4bit: true
+adapter: qlora
+
+lora_r: 8                          # 💡 уменьшено для маленького датасета
+lora_alpha: 16                    # 💡 пропорционально уменьшено
+lora_dropout: 0.05
+lora_target_modules:
+  - q_proj
+  - k_proj
+  - v_proj
+  - o_proj
+  - gate_proj
+  - up_proj
+  - down_proj
+
+
+deepspeed: /mnt/d/druban/axolotl/try1/zero1.json
+
+datasets:
+  - path: /mnt/d/druban/axolotl/try1/data/finetuning-dataset-chatml.jsonl
+    type: chat_template
+    field_messages: conversations
+
+val_set_size: 0.1                 # 💡 10% для более надёжной валидации
+
+output_dir: /mnt/d/druban/axolotl/try2
+
+sequence_len: 2048
+sample_packing: true
+eval_sample_packing: false
+
+gradient_checkpointing: true
+
+batch_size: 2
+micro_batch_size: 1
+
+num_epochs: 10                    # 💡 увеличено с 3 до 10 для глубокого дообучения
+
+optimizer: adamw_bnb_8bit
+learning_rate: 5e-5               # 💡 чуть выше для малых данных
+lr_scheduler: cosine
+warmup_steps: 10
+
+wandb_project: gemma-finetune
+```
+
+</details><br>
+
+# mnt/d/druban/axolotl/try2
+
+This model is a fine-tuned version of [google/gemma-3-1b-it](https://huggingface.co/google/gemma-3-1b-it) on the /mnt/d/druban/axolotl/try1/data/finetuning-dataset-chatml.jsonl dataset.
+It achieves the following results on the evaluation set:
+- Loss: 4.5024
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 5e-05
+- train_batch_size: 1
+- eval_batch_size: 1
+- seed: 42
+- distributed_type: multi-GPU
+- num_devices: 2
+- total_train_batch_size: 2
+- total_eval_batch_size: 2
+- optimizer: Use OptimizerNames.ADAMW_BNB with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
+- lr_scheduler_type: cosine
+- lr_scheduler_warmup_steps: 10
+- num_epochs: 10.0
+
+### Training results
+
+| Training Loss | Epoch | Step | Validation Loss |
+|:-------------:|:-----:|:----:|:---------------:|
+| 6.4781        | 1.0   | 7    | 13.0395         |
+| 5.5099        | 2.0   | 14   | 9.8655          |
+| 4.5869        | 3.0   | 21   | 7.0045          |
+| 4.1804        | 4.0   | 28   | 5.8688          |
+| 4.0683        | 5.0   | 35   | 5.1847          |
+| 4.0524        | 6.0   | 42   | 4.7762          |
+| 3.9197        | 7.0   | 49   | 4.6118          |
+| 3.6931        | 8.0   | 56   | 4.5488          |
+| 3.995         | 9.0   | 63   | 4.5053          |
+| 3.7989        | 10.0  | 70   | 4.5024          |
+
+
+### Framework versions
+
+- PEFT 0.15.1
+- Transformers 4.51.0
+- Pytorch 2.5.1+cu121
+- Datasets 3.5.0
+- Tokenizers 0.21.1