Update README.md

README.md

@@ -38,7 +38,7 @@ language:
 pipeline_tag: text-classification
 ---
 
-# ModernBERT Topics Classification Model (davanstrien/modernbert-topics-1m)
+# ModernBERT-web-topics-1m)
 
 ## Model Description
 
@@ -183,6 +183,92 @@ predicted_label = model.config.id2label[prediction]
 print(f"Predicted topic: {predicted_label}")
 ```
 
+### Efficient Inference with vLLM
+This model is compatible with vLLM for efficient, large-scale inference. vLLM is a high-performance inference engine that can significantly accelerate inference for ModernBERT classifiers.
+Installation
+
+To use vLLM with this model, install the latest version that supports ModernBERT (support was added in April 2025):
+
+#### Basic Usage
+
+Here's how to load and use the model with vLLM:
+
+```python
+from vllm import LLM
+import torch
+import torch.nn.functional as F
+
+# Load the model with vLLM
+llm = LLM(model="davanstrien/modernbert-topics-1m", task="classify")
+
+# Single prediction
+text = "This article discusses various approaches to content categorization using machine learning"
+outputs = llm.classify(text)
+
+# Process outputs
+logits = torch.tensor(outputs[0].outputs.probs)
+probabilities = F.softmax(logits, dim=0)
+top_idx = torch.argmax(probabilities).item()
+top_prob = probabilities[top_idx].item()
+
+# Get label mapping from model config
+import httpx
+from huggingface_hub import hf_hub_url
+from toolz import keymap
+
+id2label = (
+    httpx.get(
+        hf_hub_url(
+            "davanstrien/modernbert-topics-1m", 
+            filename="config.json"
+        )
+    )
+    .json()
+    .get("id2label")
+)
+id2label = keymap(int, id2label)
+
+# Get predicted label
+predicted_label = id2label.get(top_idx)
+print(f"Predicted topic: {predicted_label}")
+print(f"Confidence: {top_prob:.4f}")
+```
+
+#### Batch Processing for Large Datasets
+
+For large datasets, vLLM can process thousands of examples efficiently:
+
+```python
+from toolz import partition_all
+from tqdm.auto import tqdm
+
+# Load your dataset (could be from Hugging Face, Pandas, etc.)
+# Example with documents list
+documents = ["Document 1 content", "Document 2 content", ..., "Document N content"]
+
+# Process in batches for very large datasets
+batch_size = 10000
+all_results = []
+
+for batch in tqdm(list(partition_all(batch_size, documents))):
+    all_results.extend(llm.classify(batch))
+
+# Helper function to extract labels and confidence scores
+def get_top_label(output, label_map):
+    logits = torch.tensor(output.outputs.probs)
+    probs = F.softmax(logits, dim=0)
+    top_idx = torch.argmax(probs).item()
+    top_prob = probs[top_idx].item()
+    return label_map.get(top_idx), top_prob
+
+# Process all results
+predictions = [get_top_label(output, id2label) for output in all_results]
+labels = [pred[0] for pred in predictions]
+confidence_scores = [pred[1] for pred in predictions]
+```
+
+
+
 ## Ethical Considerations and Biases
 
 - This model may inherit biases present in the training data, potentially leading to inconsistent classification across different demographic or cultural contexts.