Minibase commited on
Commit
88f0131
·
verified ·
1 Parent(s): 35a6bef

Upload benchmarks.txt with huggingface_hub

Browse files
Files changed (1) hide show
  1. benchmarks.txt +48 -0
benchmarks.txt ADDED
@@ -0,0 +1,48 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # NER Benchmark Results
2
+ **Model:** Minibase-NER-Standard
3
+ **Dataset:** ner_benchmark_dataset.jsonl
4
+ **Sample Size:** 100
5
+ **Date:** 2025-10-07T13:41:36.866891
6
+
7
+ ## Overall Performance
8
+
9
+ | Metric | Score | Description |
10
+ |--------|-------|-------------|
11
+ | F1 Score | 0.951 | Overall NER performance (harmonic mean of precision and recall) |
12
+ | Precision | 0.915 | Accuracy of entity predictions |
13
+ | Recall | 1.000 | Ability to find all entities |
14
+ | Average Latency | 323.3ms | Response time performance |
15
+
16
+ ## Entity Type Performance
17
+
18
+ | Entity Type | Accuracy | Correct/Total |
19
+ |-------------|----------|---------------|
20
+ | PERSON | 1.000 | 100/100 |
21
+ | ORG | 1.000 | 100/100 |
22
+ | LOC | 0.660 | 66/100 |
23
+ | MISC | 1.000 | 34/34 |
24
+
25
+ ## Key Improvements
26
+
27
+ - **BIO Tagging**: Model outputs entities in BIO (Beginning-Inside-Outside) format
28
+ - **Multiple Entity Types**: Supports PERSON, ORG, LOC, and MISC entities
29
+ - **Entity-Level Evaluation**: Metrics calculated at entity level rather than token level
30
+ - **Comprehensive Coverage**: Evaluates across different text domains
31
+
32
+ ## Example Results
33
+
34
+ ### Example 1
35
+ **Input:** John Smith works at Google in New York and uses Python programming language....
36
+ **Predicted:** { "PER": ["John Smith"], "ORG": ["Google"], "LOC": ["New York"], "MISC": ["Python"] }...
37
+ **F1 Score:** 0.857
38
+
39
+ ### Example 2
40
+ **Input:** Microsoft Corporation announced that Satya Nadella will visit London next week....
41
+ **Predicted:** { "PER": ["Satya Nadella"], "ORG": ["Microsoft Corporation"], "LOC": ["London"], "MISC": [] }...
42
+ **F1 Score:** 1.000
43
+
44
+ ### Example 3
45
+ **Input:** The University of Cambridge is located in the United Kingdom and was founded by King Henry III....
46
+ **Predicted:** { "PER": ["King Henry III"], "ORG": ["University of Cambridge"], "LOC": ["United Kingdom"], "MISC": [] }...
47
+ **F1 Score:** 1.000
48
+