ModernBERT Fine-tuned for Named Entity Recognition
This model is a fine-tuned version of answerdotai/ModernBERT-base on the CoNLL-2003 Named Entity Recognition dataset.
Model Details
Model Description
- Developed by: Jayesh Thakare
- Model type: Token Classification (Named Entity Recognition)
- Language(s): English
- License: Apache 2.0
- Finetuned from model: answerdotai/ModernBERT-base
Model Sources
- Repository: joe-xhedi/ModernBERT-NER
- Base Model: answerdotai/ModernBERT-base
- Dataset: CoNLL-2003
Uses
Direct Use
This model can be used for Named Entity Recognition in English text, identifying the following entity types:
- PER: Person
- ORG: Organization
- LOC: Location
- MISC: Miscellaneous
Usage
from transformers import AutoTokenizer, AutoModelForTokenClassification, pipeline
# Load model and tokenizer
tokenizer = AutoTokenizer.from_pretrained("joe-xhedi/ModernBERT-NER")
model = AutoModelForTokenClassification.from_pretrained("joe-xhedi/ModernBERT-NER")
# Create NER pipeline
ner_pipeline = pipeline("ner",
model=model,
tokenizer=tokenizer,
aggregation_strategy="simple")
# Example usage
text = "John Doe works at OpenAI in San Francisco."
results = ner_pipeline(text)
print(results)
Training Details
Training Data
The model was fine-tuned on the CoNLL-2003 Named Entity Recognition dataset, which contains:
- Training examples: ~14,000 sentences
- Validation examples: ~3,200 sentences
- Test examples: ~3,400 sentences
Training Procedure
Training Hyperparameters
- Training regime: Mixed precision (fp16)
- Learning rate: 2e-05
- Training epochs: 3
- Batch size: 16
- Weight decay: 0.01
- Warmup steps: 500
Evaluation Strategy
- Evaluation steps: 500
- Save steps: 500
- Logging steps: 100
Evaluation
Testing Data, Factors & Metrics
Testing Data
The model was evaluated on the CoNLL-2003 test dataset.
Metrics
The model was evaluated using seqeval metrics:
- Accuracy: Token-level accuracy
- F1 Score: Entity-level F1 score (macro-averaged)
- Precision: Entity-level precision
- Recall: Entity-level recall
Results
Metric | Value |
---|---|
Accuracy | 0.9892527549550251 |
F1 Score | 0.9363408521303258 |
Precision | N/A |
Recall | N/A |
Entity Labels
The model recognizes the following entity types:
- B-PER: B-PER
- I-PER: I-PER
- B-ORG: B-ORG
- I-ORG: I-ORG
- B-LOC: B-LOC
- I-LOC: I-LOC
- B-MISC: B-MISC
- I-MISC: I-MISC
Environmental Impact
Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).
Technical Specifications
Model Architecture and Objective
- Base Architecture: ModernBERT (BERT-like transformer)
- Objective: Token Classification with Cross-Entropy Loss
- Number of Parameters: ~110M (inherited from ModernBERT-base)
- Number of Labels: 9
Compute Infrastructure
Hardware
- GPU: T4
Software
- Framework: PyTorch + Transformers
- Training Library: Hugging Face Transformers
- Evaluation Library: seqeval
Citation
If you use this model, please cite:
@misc{ModernBERT-NER,
title={ModernBERT Fine-tuned for Named Entity Recognition},
author={Jayesh Thakare},
year={2025},
url={https://huggingface.co/joe-xhedi/ModernBERT-NER}
}
- Downloads last month
- 9
Inference Providers
NEW
This model isn't deployed by any Inference Provider.
🙋
Ask for provider support
Model tree for joe-xhedi/ModernBERT-NER
Base model
answerdotai/ModernBERT-baseDataset used to train joe-xhedi/ModernBERT-NER
Evaluation results
- F1 on CoNLL-2003self-reported0.936
- Accuracy on CoNLL-2003self-reported0.989