PSewmuthu commited on
Commit
662ce5e
Β·
1 Parent(s): 8675e7f

Add README file

Browse files
Files changed (1) hide show
  1. README.md +220 -0
README.md CHANGED
@@ -1,3 +1,223 @@
1
  ---
 
2
  license: apache-2.0
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ language: en
3
  license: apache-2.0
4
+ library_name: tensorflow
5
+ tags:
6
+ - tensorflow
7
+ - keras
8
+ - emotion-recognition
9
+ - resnet50
10
+ - ckplus
11
+ - rafdb
12
+ - fine-tuning
13
+ - computer-vision
14
+ - deep-learning
15
+ - facial-expression
16
+ - affective-computing
17
+ - tflite
18
+ model-index:
19
+ - name: emotion_resnet50_model
20
+ results:
21
+ - task:
22
+ type: image-classification
23
+ dataset:
24
+ type: dataset
25
+ name: CK+ & RAF-DB
26
+ metrics:
27
+ - name: accuracy
28
+ type: float
29
+ value: 0.8165
30
+ inference: "Supports TensorFlow and TensorFlow Lite inference"
31
  ---
32
+
33
+ # 🧠 Emotion Recognition Model – ResNet50 (Fine-Tuned on CK+ and RAF-DB)
34
+
35
+ ## πŸ“˜ Overview
36
+
37
+ This repository presents a **fine-tuned ResNet50-based Emotion Recognition model** trained on the **CK+** and **RAF-DB** facial expression datasets. The model classifies facial emotions into seven categories and provides high accuracy and generalization through a two-phase fine-tuning process. Both **TensorFlow** and **TensorFlow Lite** versions are available for deployment on cloud and edge devices.
38
+
39
+ This model forms a key component of an **AI-powered, emotion-aware sign language translation system**, enabling real-time emotion recognition and integration into affective computing pipelines.
40
+
41
+ ---
42
+
43
+ ## 🧩 Model Architecture
44
+
45
+ The architecture is built on **ResNet50**, pre-trained on ImageNet, and fine-tuned in two stages:
46
+
47
+ 1. **Stage 1 – Frozen Base Training (10 Epochs):**
48
+
49
+ - Convolutional layers frozen.
50
+ - Only top dense layers trained to learn task-specific emotion representations.
51
+
52
+ 2. **Stage 2 – Unfrozen Base Fine-Tuning (30 Epochs):**
53
+
54
+ - Entire base unfrozen with a lower learning rate.
55
+ - Enables deep adaptation to emotion-related facial features.
56
+
57
+ ---
58
+
59
+ ## πŸ“Š Datasets
60
+
61
+ 1. **[CK+ Dataset (Kaggle)](https://www.kaggle.com/datasets/shareef0612/ckdataset)**
62
+ 2. **[RAF-DB Dataset (Kaggle)](https://www.kaggle.com/datasets/shuvoalok/raf-db-dataset)**
63
+
64
+ ### Dataset Preparation
65
+
66
+ - Combined CK+ and RAF-DB datasets to increase emotion diversity.
67
+ - Applied **extensive data augmentation** (rotation, flips, brightness, zoom, and shifts).
68
+ - Ensured **balanced class distribution** post-augmentation.
69
+
70
+ ---
71
+
72
+ ## βš™οΈ Training Configuration
73
+
74
+ | Parameter | Description |
75
+ | ----------------- | ---------------------------------- |
76
+ | **Base Model** | ResNet50 (Pre-trained on ImageNet) |
77
+ | **Optimizer** | Adam |
78
+ | **Learning Rate** | 1e-4 (unfrozen phase) |
79
+ | **Loss Function** | Sparse Categorical Crossentropy |
80
+ | **Batch Size** | 32 |
81
+ | **Epochs** | 40 (10 + 30) |
82
+ | **Image Size** | 224x224 |
83
+
84
+ ---
85
+
86
+ ## πŸ“ˆ Performance Summary
87
+
88
+ | Metric | Training | Validation | Testing |
89
+ | ------------ | -------- | ---------- | ------- |
90
+ | **Accuracy** | 98.43% | 86.05% | 81.65% |
91
+ | **Loss** | 0.0610 | 0.7748 | 1.1962 |
92
+
93
+ ### Classification Report
94
+
95
+ | Class | Precision | Recall | F1-Score |
96
+ | ----- | --------- | ------ | -------- |
97
+ | 0 | 0.74 | 0.69 | 0.72 |
98
+ | 1 | 0.53 | 0.49 | 0.51 |
99
+ | 2 | 0.88 | 0.31 | 0.46 |
100
+ | 3 | 0.93 | 0.90 | 0.91 |
101
+ | 4 | 0.74 | 0.87 | 0.80 |
102
+ | 5 | 0.77 | 0.75 | 0.76 |
103
+ | 6 | 0.82 | 0.83 | 0.82 |
104
+
105
+ **Overall Accuracy:** 81.65%
106
+ **Weighted F1-Score:** 0.81
107
+
108
+ ---
109
+
110
+ ## πŸ–ΌοΈ Visualizations
111
+
112
+ ### 1. Training Accuracy and Loss
113
+
114
+ - **Graph 1:** Training and Validation Accuracy vs Epochs
115
+ ![Training Accuracy](images/Training_Accuracy.png)
116
+
117
+ - **Graph 2:** Training and Validation Loss vs Epochs
118
+ ![Training Loss](images/Training_Loss.png)
119
+
120
+ ### 2. Dataset Distributions
121
+
122
+ - **Graph 3:** Original Dataset Class Distribution
123
+ ![Original Dataset Distribution](images/Original_Dataset_Distribution.png)
124
+
125
+ - **Graph 4:** Balanced (Augmented) Dataset Class Distribution
126
+ ![Balanced Dataset Distribution](images/Balanced_Dataset_Distribution.png)
127
+
128
+ ### 3. Evaluation Visuals
129
+
130
+ - **Graph 5:** Multi-Class ROC Curves (AUC per class)
131
+ ![ROC Curves](images/ROC_Curves.png)
132
+
133
+ - **Graph 6:** Confusion Matrix (Heatmap)
134
+ ![Confusion Matrix](images/Confusion_Matrix.png)
135
+
136
+ - **Graph 7:** Sample Test Results (5 predictions per class)
137
+ ![Sample Test Results](images/Sample_Test_Results.png)
138
+
139
+ These visualizations demonstrate consistent learning, balanced data, and strong class-wise recognition.
140
+
141
+ ---
142
+
143
+ ## 🧩 Model Files
144
+
145
+ | File | Description |
146
+ | ----------------------------------- | -------------------------------------------------------------- |
147
+ | `emotion_resnet50_model.h5` | Full TensorFlow model (β‰ˆ273 MB) |
148
+ | `emotion_resnet50_optimized.tflite` | Optimized TensorFlow Lite model (β‰ˆ23 MB) for mobile deployment |
149
+
150
+ ---
151
+
152
+ ## 🧰 Inference Example
153
+
154
+ ```python
155
+ import tensorflow as tf
156
+ from tensorflow.keras.preprocessing import image
157
+ import numpy as np
158
+
159
+ # Load original model
160
+ model_path = 'emotion_resnet50_model.h5'
161
+ model = tf.keras.models.load_model(model_path)
162
+
163
+ # Prepare input
164
+ img = image.load_img('test_face.jpg', target_size=(224, 224))
165
+ input_data = np.expand_dims(image.img_to_array(img) / 255.0, axis=0)
166
+
167
+ # Run inference
168
+ pred = model.predict(input_data)
169
+ classes = ['Angry', 'Disgust', 'Fear', 'Happy', 'Neutral', 'Sad', 'Surprise']
170
+ print("Original Model Prediction:", classes[np.argmax(pred)])
171
+
172
+ # Load and run TFLite optimized model
173
+ tflite_model_path = 'emotion_resnet50_optimized.tflite'
174
+ interpreter = tf.lite.Interpreter(model_path=tflite_model_path)
175
+ interpreter.allocate_tensors()
176
+ input_index = interpreter.get_input_details()[0]['index']
177
+ output_index = interpreter.get_output_details()[0]['index']
178
+ interpreter.set_tensor(input_index, input_data.astype(np.float32))
179
+ interpreter.invoke()
180
+ output = interpreter.get_tensor(output_index)
181
+ print("TFLite Model Prediction:", classes[np.argmax(output)])
182
+ ```
183
+
184
+ ---
185
+
186
+ ## πŸš€ Key Features
187
+
188
+ - Dual-dataset fine-tuning (CK+ + RAF-DB)
189
+ - Data-balanced training via augmentation
190
+ - High accuracy with stable generalization
191
+ - Lightweight TensorFlow Lite version for edge devices
192
+ - Ideal for integration in emotion-aware systems
193
+
194
+ ---
195
+
196
+ ## 🏷️ Tags
197
+
198
+ `emotion-recognition` `resnet50` `facial-expression` `deep-learning` `tensorflow` `tflite` `ckplus` `rafdb` `computer-vision` `affective-computing` `fine-tuning`
199
+
200
+ ---
201
+
202
+ ## πŸ“„ Citation
203
+
204
+ ```bibtex
205
+ @misc{pasindu_sewmuthu_abewickrama_singhe_2025,
206
+ author = { Pasindu Sewmuthu Abewickrama Singhe },
207
+ title = { resnet50-emotion-recognition-ckplus-rafdb (Revision 3216038) },
208
+ year = 2025,
209
+ url = { https://huggingface.co/PSewmuthu/resnet50-emotion-recognition-ckplus-rafdb },
210
+ doi = { 10.57967/hf/6653 },
211
+ publisher = { Hugging Face }
212
+ }
213
+ ```
214
+
215
+ ---
216
+
217
+ ## πŸ‘€ Author & Model Info
218
+
219
+ **Author:** P.S. Abewickrama Singhe
220
+ **Developed with:** TensorFlow + Keras
221
+ **License:** Apache-2.0
222
+ **Date:** October 2025
223
+ **Email:** [[email protected]](mailto:[email protected])