cja5553/covid-vaccine-sentiments-BERTweet-large

Benchmark result for paper titled "Towards Robust Sentiment Analysis of Temporally-Sensitive Policy-Related Online Text", accepted into ACL-SRW.

Classification of sentiments towards COVID-vaccines from Tweets (now known as X).

This model classifies sentiments towards COVID Vaccines into "Negative" (1), "Neutral" (2) and "Positive" (3).

Training Dataset

This was fine-tuned on the COVID Vaccine Twitter Dataset with vinai/bertweet-large serving as the base model.

The COVID Vaccine Twitter Dataset contains 6,000 tweets annotated with sentiment labels positive, negative or neutral toward COVID-19 vaccines. The tweets were collected during the initial months following the vaccine's release, spanning December 2020 through April 2021.

3,000 data points were sampled through continous-time series clustering and were used to finetune the data in order to mimic the constraints of annotation and finetuning in policy-related studies. (To understand why we do that, refer to our paper at the reference below)

How to use model

Here is some source code to get you started on using the model to classify the sentiments.

import torch
from transformers import AutoModelForSequenceClassification, AutoTokenizer, pipeline
from tqdm import tqdm
import pandas as pd
def classify_tweets(df, text_col, model, tokenizer):
    df[text_col]=df[text_col].astype(str)
    device = 0 if torch.cuda.is_available() else -1  # Use GPU if available
    classifier = pipeline(
        "text-classification",
        model=model,
        tokenizer=tokenizer,
        device=device,
        truncation=True,  # Ensures inputs don't exceed max length
        max_length=512,   # Manually set to avoid exceeding model's limit
        padding="max_length"  # Ensures all inputs have the same length
    )
    
    outcomes, probs, pred_labels = [], [], []
    for text in tqdm(df[text_col]):  # Fixed tqdm syntax
        preds = classifier(text, return_all_scores=True)
        outcomes.append(preds)
        
        # Extract probabilities and predicted label
        label_scores = {entry['label']: entry['score'] for entry in preds[0]}
        probs.append(list(label_scores.values()))
        pred_labels.append(max(label_scores, key=label_scores.get))
    df["predicted_label"] = pred_labels
    return df
model_name="cja5553/covid-vaccine-sentiments-BERTweet-large"
label2id={1:"Negative",2:"Neutral",3:"Positive"}
id2label = {v: k for k, v in label2id.items()}
model = AutoModelForSequenceClassification.from_pretrained(model_name, num_labels=len(label2id), 
                                                           label2id=label2id,id2label=id2label,
                                                           use_auth_token=False).to("cuda")
tokenizer = AutoTokenizer.from_pretrained(model_name,use_auth_token=False)
text_col="text" # change text column accordingly
df_with_classification=classify_tweets(df, text_col, model, tokenizer)

Citation

If you find this model useful, please cite the following paper:

@inproceedings{
author={Charles Alba, Benjamin C Warner, Akshar Saxena, Jiaxin Huang, Ruopeng An},
title={Towards Robust Sentiment Analysis of Temporally-Sensitive Policy-Related Online Text},
year={2025},
booktitle={Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (ACL), Volume 4: Student Research Workshop.},
url={https://aclanthology.org/2025.acl-srw.70/}}

Code

Code used to train these models are available on GitHub at github.com/cja5553/ctscams

Questions?

contact me at [email protected]