Update README.md (#2)
Browse files- Update README.md (688ed07803d61e03beb199a7c548894148633be3)
README.md
CHANGED
|
@@ -7,39 +7,39 @@ base_model:
|
|
| 7 |
pipeline_tag: image-text-to-text
|
| 8 |
library_name: transformers
|
| 9 |
tags:
|
| 10 |
-
- Highlights
|
| 11 |
- Generation
|
| 12 |
- OCR
|
| 13 |
- KIE
|
|
|
|
| 14 |
---
|
| 15 |
|
| 16 |
-

|
|
|
|
|
|
|
| 17 |
|
| 18 |
+
# **WASP-2B-VL-Highlights**
|
|
|
|
| 19 |
|
| 20 |
+
> \[!Note]
|
| 21 |
+
> The **WASP-2B-VL-Highlights** model is a fine-tuned version of *Qwen2-VL-2B-Instruct*, specifically optimized for **image highlights extraction**, **messy handwriting recognition**, **Optical Character Recognition (OCR)**, **English language understanding**, and **math problem solving with LaTeX formatting**. This model uses a conversational visual-language interface to effectively handle multi-modal tasks.
|
| 22 |
+
|
| 23 |
+
[](https://colab.research.google.com/#fileId=https%3A//huggingface.co/prithivMLmods/WASP-2B-VL-Highlights/blob/main/Callisto_OCR3_2B_Instruct.ipynb)
|
| 24 |
|
| 25 |
# **Key Enhancements:**
|
| 26 |
|
| 27 |
+
* **State-of-the-art image comprehension** across varying resolutions and aspect ratios:
|
| 28 |
+
WASP-2B-VL-Highlights delivers top-tier performance on benchmarks such as MathVista, DocVQA, RealWorldQA, and MTVQA.
|
| 29 |
|
| 30 |
+
* **Image Highlighting Expertise**:
|
| 31 |
Specially tuned to **identify and summarize key visual elements** in an image — ideal for **creating visual highlights**, annotations, and summaries.
|
| 32 |
|
| 33 |
+
* **Handwriting OCR Enhanced**:
|
| 34 |
Recognizes **messy and complex handwritten notes** with precision, perfect for digitizing real-world documents.
|
| 35 |
|
| 36 |
+
* **Video Content Understanding**:
|
| 37 |
+
Capable of processing videos longer than 20 minutes for **context-aware Q\&A, transcription**, and **highlight extraction**.
|
| 38 |
|
| 39 |
+
* **Multi-device Integration**:
|
| 40 |
Can be used as an intelligent agent for mobile phones, robots, and other devices — able to **understand visual scenes and execute actions**.
|
| 41 |
|
| 42 |
+
* **Multilingual OCR Support**:
|
| 43 |
In addition to English and Chinese, supports OCR for European languages, Japanese, Korean, Arabic, and Vietnamese.
|
| 44 |
|
| 45 |
# **Run with Transformers🤗**
|
|
|
|
| 76 |
|
| 77 |
# Define model options
|
| 78 |
MODEL_OPTIONS = {
|
| 79 |
+
"Needle-2B-VL-Highlights": "prithivMLmods/WASP-2B-VL-Highlights",
|
| 80 |
}
|
| 81 |
|
| 82 |
# Preload models and processors into CUDA
|
|
|
|
| 288 |
model_choice = gr.Dropdown(
|
| 289 |
label="Model Selection",
|
| 290 |
choices=list(MODEL_OPTIONS.keys()),
|
| 291 |
+
value="WASP-2B-VL-Highlights"
|
| 292 |
)
|
| 293 |
input_media = gr.File(
|
| 294 |
label="Upload Image", type="filepath"
|