File size: 2,550 Bytes
437b087 7eb2829 437b087 7eb2829 437b087 7eb2829 437b087 7eb2829 437b087 7eb2829 437b087 7eb2829 437b087 7eb2829 437b087 7eb2829 437b087 7eb2829 437b087 7eb2829 437b087 7eb2829 437b087 7eb2829 437b087 7eb2829 437b087 7eb2829 437b087 7eb2829 437b087 7eb2829 437b087 7eb2829 437b087 7eb2829 437b087 7eb2829 437b087 7eb2829 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 |
---
library_name: transformers
tags: []
---
---
# phi2-2b-absa: Fine-Tuned Aspect-Based Sentiment Analysis Model
## Model Description
The **phi2-2b-absa** model is a fine-tuned aspect-based sentiment analysis (ABSA) model based on the Microsoft Phi-2 model. It has been trained on the **semeval2016-full-absa-reviews-english-translated-resampled** dataset. The model predicts sentiments towards different aspects mentioned in a given sentence.
## Fine-Tuning Details
The fine tuning can be revisited on [Google Colab](https://colab.research.google.com/drive/1n3ykETLpHQPXwPhUcOe-z9cG3ThrDkSi?usp=sharing).
### Dataset
- **Name:** semeval2016-full-absa-reviews-english-translated-resampled
- **Description:** Annotated dataset for ABSA containing sentences, aspects, sentiments, and additional contextual text. It is split into train and test sets.
### Model Architecture
- **Base Model:** Microsoft Phi-2
- **Fine-Tuned Model:** phi2-2b-absa
### Fine-Tuning Parameters
- **LoRA Attention Dimension (lora_r):** 64
- **LoRA Scaling Parameter (lora_alpha):** 16
- **LoRA Dropout Probability (lora_dropout):** 0.1
### BitsAndBytes Quantization
- **Activate 4-bit Precision:** True
- **Compute Dtype for 4-bit Models:** float16
- **Quantization Type:** nf4
### Training Parameters
- **Number of Training Epochs:** 1
- **Batch Size per GPU for Training:** 4
- **Batch Size per GPU for Evaluation:** 4
- **Gradient Accumulation Steps:** 1
- **Learning Rate:** 2e-4
- **Weight Decay:** 0.001
- **Optimizer:** PagedAdamW (32-bit)
- **Learning Rate Scheduler:** Cosine
### SFT Parameters
- **Maximum Sequence Length:** None
- **Packing:** False
## How to Use
```
from transformers import AutoTokenizer, pipeline
import torch
model = "Alpaca69B/llama-2-7b-absa-semeval-2016"
tokenizer = AutoTokenizer.from_pretrained(model)
pipeline = pipeline(
"text-generation",
model=model,
tokenizer=tokenizer,
torch_dtype=torch.float16,
device="auto",
)
input_sentence = "the first thing that attracts attention is the warm reception and the smiling receptionists."
sequences = pipeline(
f'### Human: {input_sentence} ### Assistant: aspect:',
do_sample=True,
top_k=10,
num_return_sequences=1,
eos_token_id=tokenizer.eos_token_id,
max_length=200,
)
sequences[0]['generated_text']
```
Testing can be seen on [Google Colab](https://colab.research.google.com/drive/1eKdZYYWiivyeCQDsocGBstVODMLZyT-_?usp=sharing)
## Acknowledgments
- The fine-tuning process and model development were performed by Ben Kampmann.
--- |