Flan-T5-XL Fine-Tuned for Medication Mention Extraction

Model Description

This is a fine-tuned version of the google/flan-t5-xl model for the automatic extraction of medication mentions from social media text, specifically tweets. The model reformulates named entity recognition (NER) as a sequence-to-sequence generation task, directly outputting a structured list of medications mentioned in the input text.

Training Data

The model was fine-tuned using the publicly available datasets from:

BioCreative VII Shared Task 3
#SMM4H 2018 Task 1 (with span-level annotations)

In total, the training set included 98,610 tweets, with approximately 5% containing medication mentions.

Intended Use

Extraction of medication mentions from social media data (primarily Twitter).
Suitable for applications in digital epidemiology, pharmacovigilance, and health-related large-scale analysis of social media data.

How to Use

from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

tokenizer = AutoTokenizer.from_pretrained("guilopgar/flan-t5-xl-medication-ner")
model = AutoModelForSeq2SeqLM.from_pretrained("guilopgar/flan-t5-xl-medication-ner")

input_text = ("You are given a tweet followed by a specific question asking about the content of the tweet. "
               "Your objective is to identify and list any drug names, medications, or dietary supplements mentioned "
               "in the tweet. If one or more are mentioned, list each distinctly, separated by a comma. "
               "If none are mentioned, return an empty list [].\n\n"
               "Input: Tweet: Benadryl, bedtime snack, and New Girl. The party is getting real.\n"
               "Question: What are the drugs, medications or dietary supplements mentioned in the tweet?\n"
               "Output:")

inputs = tokenizer(input_text, return_tensors="pt")
outputs = model.generate(**inputs)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Citation

If you use this model, please cite our work:

@article{Lopez-Garcia2025.05.16.25327791,
   author = {Lopez-Garcia, Guillermo and Xu, Dongfang and Gonzalez-Hernandez, Graciela},
   title = {Detecting Medication Mentions in Social Media Data Using Large Language Models},
   year = {2025},
   doi = {10.1101/2025.05.16.25327791},
   publisher = {Cold Spring Harbor Laboratory Press},
   URL = {https://www.medrxiv.org/content/early/2025/05/18/2025.05.16.25327791},
   journal = {medRxiv}
}