Mitchins commited on
Commit
2ee5f3c
·
verified ·
1 Parent(s): 2667b42

Upload folder using huggingface_hub

Browse files
README.md CHANGED
@@ -153,6 +153,58 @@ result = classifier("Captain Torres must infiltrate enemy lines while battling h
153
  print(result)
154
  ```
155
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
156
  ## Limitations
157
 
158
  - **Domain:** Optimized for character descriptions in narrative fiction
 
153
  print(result)
154
  ```
155
 
156
+ ## Example Classifications
157
+
158
+ Here are sample classifications showing the model's predictions with confidence scores:
159
+
160
+ ### NONE
161
+
162
+ **Simple Example:**
163
+ > *"Margaret runs the village bakery, making fresh bread every morning at 5 AM for the past thirty years."*
164
+
165
+ **Prediction:** NONE ✅ (confidence: 0.997)
166
+
167
+ **Nuanced Example:**
168
+ > *"Dr. Harrison performs routine medical check-ups with methodical precision, maintaining professional distance while patients share their deepest fears about mortality."*
169
+
170
+ **Prediction:** NONE ⚠️ (confidence: 0.581)
171
+
172
+ ### INTERNAL
173
+
174
+ **Simple Example:**
175
+ > *"Emma struggles with overwhelming anxiety after her father's harsh criticism, questioning her self-worth and abilities."*
176
+
177
+ **Prediction:** INTERNAL ✅ (confidence: 0.983)
178
+
179
+ **Nuanced Example:**
180
+ > *"The renowned pianist Clara finds herself paralyzed by perfectionism, her childhood trauma surfacing as she prepares for the performance that could define her legacy."*
181
+
182
+ **Prediction:** INTERNAL ✅ (confidence: 0.733)
183
+
184
+ ### EXTERNAL
185
+
186
+ **Simple Example:**
187
+ > *"Knight Roderick embarks on a dangerous quest to retrieve the stolen crown from the dragon's lair."*
188
+
189
+ **Prediction:** EXTERNAL ✅ (confidence: 0.717)
190
+
191
+ **Nuanced Example:**
192
+ > *"Master thief Elias infiltrates the heavily guarded fortress, disabling security systems and evading patrol routes, each obstacle requiring new techniques and tools to reach the vault."*
193
+
194
+ **Prediction:** EXTERNAL ✅ (confidence: 0.711)
195
+
196
+ ### BOTH
197
+
198
+ **Simple Example:**
199
+ > *"Sarah must rescue her kidnapped daughter from the terrorist compound while confronting her own paralyzing guilt about being an absent mother."*
200
+
201
+ **Prediction:** BOTH ⚠️ (confidence: 0.578)
202
+
203
+ **Nuanced Example:**
204
+ > *"Archaeologist Sophia discovers an ancient artifact that could rewrite history, but must confront her own ethical boundaries and childhood abandonment issues as powerful forces try to silence her."*
205
+
206
+ **Prediction:** BOTH ✅ (confidence: 0.926)
207
+
208
  ## Limitations
209
 
210
  - **Domain:** Optimized for character descriptions in narrative fiction
generate_readme_examples.py ADDED
@@ -0,0 +1,160 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ #!/usr/bin/env python3
2
+ """
3
+ Generate 8 synthetic examples for README with model predictions
4
+ 2 examples per class: simple + nuanced
5
+ """
6
+
7
+ import torch
8
+ from transformers import DebertaV2Tokenizer, DebertaV2ForSequenceClassification
9
+
10
+ def get_examples():
11
+ """Define 8 synthetic examples: 2 per class (simple + nuanced)"""
12
+ return [
13
+ # NONE - Simple
14
+ {
15
+ "text": "Margaret runs the village bakery, making fresh bread every morning at 5 AM for the past thirty years.",
16
+ "expected": "NONE",
17
+ "type": "simple"
18
+ },
19
+ # NONE - Nuanced
20
+ {
21
+ "text": "Dr. Harrison performs routine medical check-ups with methodical precision, maintaining professional distance while patients share their deepest fears about mortality.",
22
+ "expected": "NONE",
23
+ "type": "nuanced"
24
+ },
25
+ # INTERNAL - Simple
26
+ {
27
+ "text": "Emma struggles with overwhelming anxiety after her father's harsh criticism, questioning her self-worth and abilities.",
28
+ "expected": "INTERNAL",
29
+ "type": "simple"
30
+ },
31
+ # INTERNAL - Nuanced
32
+ {
33
+ "text": "The renowned pianist Clara finds herself paralyzed by perfectionism, her childhood trauma surfacing as she prepares for the performance that could define her legacy.",
34
+ "expected": "INTERNAL",
35
+ "type": "nuanced"
36
+ },
37
+ # EXTERNAL - Simple
38
+ {
39
+ "text": "Knight Roderick embarks on a dangerous quest to retrieve the stolen crown from the dragon's lair.",
40
+ "expected": "EXTERNAL",
41
+ "type": "simple"
42
+ },
43
+ # EXTERNAL - Nuanced
44
+ {
45
+ "text": "Master thief Elias infiltrates the heavily guarded fortress, disabling security systems and evading patrol routes, each obstacle requiring new techniques and tools to reach the vault.",
46
+ "expected": "EXTERNAL",
47
+ "type": "nuanced"
48
+ },
49
+ # BOTH - Simple
50
+ {
51
+ "text": "Sarah must rescue her kidnapped daughter from the terrorist compound while confronting her own paralyzing guilt about being an absent mother.",
52
+ "expected": "BOTH",
53
+ "type": "simple"
54
+ },
55
+ # BOTH - Nuanced
56
+ {
57
+ "text": "Archaeologist Sophia discovers an ancient artifact that could rewrite history, but must confront her own ethical boundaries and childhood abandonment issues as powerful forces try to silence her.",
58
+ "expected": "BOTH",
59
+ "type": "nuanced"
60
+ }
61
+ ]
62
+
63
+ def predict_examples():
64
+ """Run predictions on all examples"""
65
+ print("Loading model...")
66
+ tokenizer = DebertaV2Tokenizer.from_pretrained('.')
67
+ model = DebertaV2ForSequenceClassification.from_pretrained('.')
68
+
69
+ device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
70
+ model.to(device)
71
+ model.eval()
72
+
73
+ class_names = ['NONE', 'INTERNAL', 'EXTERNAL', 'BOTH']
74
+ examples = get_examples()
75
+
76
+ results = []
77
+
78
+ print(f"Running predictions on {len(examples)} examples...\n")
79
+
80
+ for i, example in enumerate(examples, 1):
81
+ text = example['text']
82
+ expected = example['expected']
83
+ example_type = example['type']
84
+
85
+ # Predict
86
+ inputs = tokenizer(text, return_tensors="pt", truncation=True, padding=True, max_length=512)
87
+ inputs = {k: v.to(device) for k, v in inputs.items()}
88
+
89
+ with torch.no_grad():
90
+ outputs = model(**inputs)
91
+ probabilities = torch.softmax(outputs.logits, dim=-1)
92
+ predicted_idx = torch.argmax(probabilities, dim=-1).item()
93
+ confidence = probabilities[0][predicted_idx].item()
94
+
95
+ predicted = class_names[predicted_idx]
96
+ is_correct = predicted == expected
97
+
98
+ result = {
99
+ 'text': text,
100
+ 'expected': expected,
101
+ 'predicted': predicted,
102
+ 'confidence': confidence,
103
+ 'correct': is_correct,
104
+ 'type': example_type
105
+ }
106
+
107
+ results.append(result)
108
+
109
+ status = "✅" if is_correct else "❌"
110
+ print(f"{status} Example {i} ({expected} - {example_type})")
111
+ print(f" Predicted: {predicted} (confidence: {confidence:.3f})")
112
+ print(f" Text: {text[:80]}...")
113
+ print()
114
+
115
+ return results
116
+
117
+ def format_for_readme(results):
118
+ """Format results for README inclusion"""
119
+
120
+ # Group by class
121
+ by_class = {}
122
+ for result in results:
123
+ expected = result['expected']
124
+ if expected not in by_class:
125
+ by_class[expected] = []
126
+ by_class[expected].append(result)
127
+
128
+ readme_content = """
129
+ ## Example Classifications
130
+
131
+ Here are sample classifications showing the model's predictions with confidence scores:
132
+
133
+ """
134
+
135
+ for class_name in ['NONE', 'INTERNAL', 'EXTERNAL', 'BOTH']:
136
+ if class_name in by_class:
137
+ readme_content += f"### {class_name}\n\n"
138
+
139
+ for result in by_class[class_name]:
140
+ confidence_icon = "✅" if result['confidence'] > 0.7 else "⚠️" if result['confidence'] > 0.5 else "❌"
141
+
142
+ readme_content += f"**{result['type'].title()} Example:**\n"
143
+ readme_content += f"> *\"{result['text']}\"*\n\n"
144
+ readme_content += f"**Prediction:** {result['predicted']} {confidence_icon} (confidence: {result['confidence']:.3f})\n\n"
145
+
146
+ return readme_content
147
+
148
+ if __name__ == "__main__":
149
+ results = predict_examples()
150
+ readme_section = format_for_readme(results)
151
+
152
+ print("README Section:")
153
+ print("=" * 50)
154
+ print(readme_section)
155
+
156
+ # Save to file
157
+ with open('readme_examples_section.txt', 'w') as f:
158
+ f.write(readme_section)
159
+
160
+ print("Saved README section to 'readme_examples_section.txt'")
readme_examples_section.txt ADDED
@@ -0,0 +1,53 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+
2
+ ## Example Classifications
3
+
4
+ Here are sample classifications showing the model's predictions with confidence scores:
5
+
6
+ ### NONE
7
+
8
+ **Simple Example:**
9
+ > *"Margaret runs the village bakery, making fresh bread every morning at 5 AM for the past thirty years."*
10
+
11
+ **Prediction:** NONE ✅ (confidence: 0.997)
12
+
13
+ **Nuanced Example:**
14
+ > *"Dr. Harrison performs routine medical check-ups with methodical precision, maintaining professional distance while patients share their deepest fears about mortality."*
15
+
16
+ **Prediction:** NONE ⚠️ (confidence: 0.581)
17
+
18
+ ### INTERNAL
19
+
20
+ **Simple Example:**
21
+ > *"Emma struggles with overwhelming anxiety after her father's harsh criticism, questioning her self-worth and abilities."*
22
+
23
+ **Prediction:** INTERNAL ✅ (confidence: 0.983)
24
+
25
+ **Nuanced Example:**
26
+ > *"The renowned pianist Clara finds herself paralyzed by perfectionism, her childhood trauma surfacing as she prepares for the performance that could define her legacy."*
27
+
28
+ **Prediction:** INTERNAL ✅ (confidence: 0.733)
29
+
30
+ ### EXTERNAL
31
+
32
+ **Simple Example:**
33
+ > *"Knight Roderick embarks on a dangerous quest to retrieve the stolen crown from the dragon's lair."*
34
+
35
+ **Prediction:** EXTERNAL ✅ (confidence: 0.717)
36
+
37
+ **Nuanced Example:**
38
+ > *"Master thief Elias infiltrates the heavily guarded fortress, disabling security systems and evading patrol routes, each obstacle requiring new techniques and tools to reach the vault."*
39
+
40
+ **Prediction:** EXTERNAL ✅ (confidence: 0.711)
41
+
42
+ ### BOTH
43
+
44
+ **Simple Example:**
45
+ > *"Sarah must rescue her kidnapped daughter from the terrorist compound while confronting her own paralyzing guilt about being an absent mother."*
46
+
47
+ **Prediction:** BOTH ⚠️ (confidence: 0.578)
48
+
49
+ **Nuanced Example:**
50
+ > *"Archaeologist Sophia discovers an ancient artifact that could rewrite history, but must confront her own ethical boundaries and childhood abandonment issues as powerful forces try to silence her."*
51
+
52
+ **Prediction:** BOTH ✅ (confidence: 0.926)
53
+