Model save

README.md

@@ -1,9 +1,9 @@
 ---
-library_name: peft
 license: apache-2.0
-base_model: mistralai/Mistral-7B-Instruct-v0.2
+library_name: peft
 tags:
 - generated_from_trainer
+base_model: mistralai/Mistral-7B-Instruct-v0.2
 model-index:
 - name: mistral-magyar-portas-lora
   results: []
@@ -15,6 +15,8 @@ should probably proofread and complete it, then remove this comment. -->
 # mistral-magyar-portas-lora
 
 This model is a fine-tuned version of [mistralai/Mistral-7B-Instruct-v0.2](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2) on the None dataset.
+It achieves the following results on the evaluation set:
+- Loss: 0.0117
 
 ## Model description
 
@@ -33,22 +35,34 @@ More information needed
 ### Training hyperparameters
 
 The following hyperparameters were used during training:
-- learning_rate: 0.0001
+- learning_rate: 0.0002
 - train_batch_size: 2
 - eval_batch_size: 2
 - seed: 42
 - gradient_accumulation_steps: 4
 - total_train_batch_size: 8
-- optimizer: Use OptimizerNames.PAGED_ADAMW_8BIT with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: cosine
 - lr_scheduler_warmup_steps: 100
-- num_epochs: 5
-- mixed_precision_training: Native AMP
+- num_epochs: 3
+
+### Training results
+
+| Training Loss | Epoch  | Step | Validation Loss |
+|:-------------:|:------:|:----:|:---------------:|
+| 0.0323        | 0.3918 | 500  | 0.0634          |
+| 0.0123        | 0.7837 | 1000 | 0.0131          |
+| 0.0094        | 1.1755 | 1500 | 0.0121          |
+| 0.0093        | 1.5674 | 2000 | 0.0120          |
+| 0.0088        | 1.9592 | 2500 | 0.0111          |
+| 0.0065        | 2.3511 | 3000 | 0.0119          |
+| 0.0064        | 2.7429 | 3500 | 0.0117          |
+
 
 ### Framework versions
 
-- PEFT 0.15.2
-- Transformers 4.52.4
-- Pytorch 2.6.0+cu124
-- Datasets 2.14.4
-- Tokenizers 0.21.1
+- PEFT 0.7.1
+- Transformers 4.40.2
+- Pytorch 2.1.1+cu121
+- Datasets 2.16.1
+- Tokenizers 0.19.1

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:abdb2545f9019d29f61988b8a09742663e67a63341580409226d2612798d0fc8
+oid sha256:0feff8954fcd1be4a316b874df88395f71c75cb719a3bcf6b84d441a3cabcf8e
 size 335605144