Hi! 👋

Thanks for your interest in NeuroBERT-Mini. Here’s a quick overview to help you get started:

Using NeuroBERT-Mini for Conversational AI 🤖
NeuroBERT-Mini is primarily designed as a lightweight BERT model that works great for masked language modeling and token classification (like NER). While it’s not a conversational model out-of-the-box (like GPT or DialoGPT), you can definitely use it as a foundational encoder for conversational tasks such as intent detection, slot filling, or building a dialogue system pipeline.

Supported Transformers Pipelines
Besides the usual "fill-mask" pipeline, NeuroBERT-Mini can be fine-tuned and used with these common pipelines:

Token Classification (NER, slot filling)
Question Answering (extractive QA)
Text Classification (intent detection, sentiment analysis)

It’s flexible depending on how you fine-tune it.

Quick Example: NER Pipeline with NeuroBERT-Mini
from transformers import AutoTokenizer, AutoModelForTokenClassification, pipeline

model_name = "boltuix/NeuroBERT-Mini"
tokenizer = AutoTokenizer.from_pretrained(model_name)

Define the number of labels based on the NER training data

In the previous NER training block, label_list had 3 labels: ["O", "DEVICE", "ACTION"]

your_num_labels = 3

model = AutoModelForTokenClassification.from_pretrained(model_name, num_labels=your_num_labels)

ner_pipeline = pipeline("token-classification", model=model, tokenizer=tokenizer, aggregation_strategy="simple")

text = "Turn on the living room lights"
entities = ner_pipeline(text)
print(entities)

For detailed fine-tuning and usage instructions, check out these links:
🔗 NeuroBERT-Mini Fine-Tuning Guide
https://huggingface.co/boltuix/NeuroBERT-Mini#fine-tuning-guide

🔗 BERT-Mini Blog & Resources
https://www.boltuix.com/2025/05/bert-mini.html
Feel free to reach out if you want help with specific tasks or custom datasets! 😊

Thanks! I'll give it a try with the Question Answering pipeline