---
library_name: peft
license: mit
base_model: Salesforce/blip2-flan-t5-xl
tags:
- generated_from_trainer
datasets:
- arrow
metrics:
- f1
model-index:
- name: blip2_lora_vqa_model
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# blip2_lora_vqa_model

This model is a fine-tuned version of [Salesforce/blip2-flan-t5-xl](https://huggingface.co/Salesforce/blip2-flan-t5-xl) on the arrow dataset.
It achieves the following results on the evaluation set:
- Loss: 0.0612
- Exact: 74.8351
- F1: 78.3950
- Total: 1061
- Hasans Exact: 74.8351
- Hasans F1: 78.3950
- Hasans Total: 1061
- Best Exact: 74.8351
- Best Exact Thresh: 0.0
- Best F1: 78.3950
- Best F1 Thresh: 0.0

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.002
- train_batch_size: 64
- eval_batch_size: 64
- seed: 42
- optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: linear
- num_epochs: 10

### Training results

| Training Loss | Epoch | Step | Validation Loss | Exact   | F1      | Total | Hasans Exact | Hasans F1 | Hasans Total | Best Exact | Best Exact Thresh | Best F1 | Best F1 Thresh |
|:-------------:|:-----:|:----:|:---------------:|:-------:|:-------:|:-----:|:------------:|:---------:|:------------:|:----------:|:-----------------:|:-------:|:--------------:|
| No log        | 1.0   | 77   | 0.1769          | 57.3044 | 62.1696 | 1061  | 57.3044      | 62.1696   | 1061         | 57.3044    | 0.0               | 62.1696 | 0.0            |
| 1.8287        | 2.0   | 154  | 0.1209          | 63.8077 | 68.4910 | 1061  | 63.8077      | 68.4910   | 1061         | 63.8077    | 0.0               | 68.4910 | 0.0            |
| 0.1517        | 3.0   | 231  | 0.0959          | 65.0330 | 69.5182 | 1061  | 65.0330      | 69.5182   | 1061         | 65.0330    | 0.0               | 69.5182 | 0.0            |
| 0.1145        | 4.0   | 308  | 0.0863          | 67.9548 | 72.6603 | 1061  | 67.9548      | 72.6603   | 1061         | 67.9548    | 0.0               | 72.6603 | 0.0            |
| 0.1145        | 5.0   | 385  | 0.0788          | 70.8765 | 74.6248 | 1061  | 70.8765      | 74.6248   | 1061         | 70.8765    | 0.0               | 74.6248 | 0.0            |
| 0.0946        | 6.0   | 462  | 0.0697          | 73.1385 | 76.8465 | 1061  | 73.1385      | 76.8465   | 1061         | 73.1385    | 0.0               | 76.8465 | 0.0            |
| 0.0837        | 7.0   | 539  | 0.0724          | 72.3845 | 76.2186 | 1061  | 72.3845      | 76.2186   | 1061         | 72.3845    | 0.0               | 76.2186 | 0.0            |
| 0.069         | 8.0   | 616  | 0.0644          | 74.3638 | 77.8250 | 1061  | 74.3638      | 77.8250   | 1061         | 74.3638    | 0.0               | 77.8250 | 0.0            |
| 0.069         | 9.0   | 693  | 0.0632          | 73.7041 | 77.3865 | 1061  | 73.7041      | 77.3865   | 1061         | 73.7041    | 0.0               | 77.3865 | 0.0            |
| 0.0652        | 10.0  | 770  | 0.0612          | 74.8351 | 78.3950 | 1061  | 74.8351      | 78.3950   | 1061         | 74.8351    | 0.0               | 78.3950 | 0.0            |


### Framework versions

- PEFT 0.13.2
- Transformers 4.46.3
- Pytorch 2.4.1+cu121
- Datasets 3.1.0
- Tokenizers 0.20.3