einrafh commited on
Commit
0cd2d07
·
1 Parent(s): 2a2ce08

docs: Update model card with latest evaluation results

Browse files
Files changed (1) hide show
  1. README.md +26 -17
README.md CHANGED
@@ -13,31 +13,32 @@ pipeline_tag: image-classification
13
 
14
  # VeriChain Deepfake Detection Model - ViT
15
 
16
- This repository contains the Vision Transformer (ViT) model fine-tuned for the task of Deepfake detection, developed as part of the VeriChain project.
17
 
18
- The model is trained to classify an image into one of three categories: **Real**, **AI-Generated**, or **Deepfake**. The model files are located in the `vit-deepfake-model/` directory.
19
 
20
- ## Model Details
21
 
22
- - **Base Model:** `google/vit-base-patch16-224-in21k`
23
- - **Fine-tuning Dataset:** [einrafh/verichain-deepfake-data](https://huggingface.co/datasets/einrafh/verichain-deepfake-data)
24
- - **Framework:** PyTorch
25
- - **Architecture:** A Vision Transformer with a classification head fine-tuned for 3 labels.
26
 
27
- ## How to Use
 
 
28
 
29
- This model can be used for inference directly with the `pipeline` function from the `transformers` library.
 
 
30
 
31
  ```python
32
  from transformers import pipeline
33
  from PIL import Image
34
 
35
  # Load the image classification pipeline with your model
36
- # 'subfolder' points to the directory containing the model files
37
  classifier = pipeline(
38
  "image-classification",
39
  model="einrafh/verichain-deepfake-models",
40
- subfolder="vit-deepfake-model"
41
  )
42
 
43
  # Load an image you want to classify
@@ -50,24 +51,32 @@ except FileNotFoundError:
50
  print("Please provide a valid path to an image file.")
51
 
52
  # Example Output:
53
- # [{'label': 'Deepfake', 'score': 0.9978}, {'label': 'AI Generated', 'score': 0.0012}, {'label': 'Real', 'score': 0.0010}]
54
  ```
55
 
56
  ## Evaluation Results
57
 
58
- The model was evaluated on a held-out test set of 2,000 images, achieving excellent performance.
59
 
60
  | Metric | Score |
61
  |----------------------|---------|
62
- | **Test Accuracy** | **0.9985** |
63
- | **F1-Score (Macro)** | **0.9985** |
64
- | Test Loss | 0.01906 |
 
 
 
 
 
 
 
 
65
 
66
  ### Confusion Matrix
67
 
68
  The confusion matrix below shows the model's high precision and recall across all classes, with very few misclassifications.
69
 
70
- ![Confusion Matrix](https://huggingface.co/einrafh/verichain-deepfake-models/resolve/main/vit-deepfake-model/confusion_matrix.png)
71
 
72
  ## Citation
73
 
 
13
 
14
  # VeriChain Deepfake Detection Model - ViT
15
 
16
+ This repository contains the artifacts for the Vision Transformer (ViT) model fine-tuned for the task of Deepfake detection, developed as part of the VeriChain project.
17
 
18
+ The model is trained to classify an image into one of three categories: **Real**, **AI-Generated**, or **Deepfake**.
19
 
20
+ ## Repository Structure
21
 
22
+ The model artifacts in this repository are organized as follows:
 
 
 
23
 
24
+ - **/models/vit-deepfake-model/**: Contains the final, fine-tuned PyTorch model files, ready to be loaded with the `transformers` library.
25
+ - **/models/onnx/**: Contains the model converted to the ONNX format, optimized for production deployment and inference.
26
+ - **/assets/**: Contains visual assets for documentation, such as the confusion matrix.
27
 
28
+ ## How to Use (PyTorch Model)
29
+
30
+ You can use the fine-tuned PyTorch model directly with the `pipeline` function from the `transformers` library. Make sure to specify the correct `subfolder`.
31
 
32
  ```python
33
  from transformers import pipeline
34
  from PIL import Image
35
 
36
  # Load the image classification pipeline with your model
37
+ # The 'subfolder' parameter points to the directory containing the model files
38
  classifier = pipeline(
39
  "image-classification",
40
  model="einrafh/verichain-deepfake-models",
41
+ subfolder="models/vit-deepfake-model"
42
  )
43
 
44
  # Load an image you want to classify
 
51
  print("Please provide a valid path to an image file.")
52
 
53
  # Example Output:
54
+ # [{'label': 'Deepfake', 'score': 0.9985}, {'label': 'AI Generated', 'score': 0.0010}, {'label': 'Real', 'score': 0.0005}]
55
  ```
56
 
57
  ## Evaluation Results
58
 
59
+ The model was evaluated on a held-out test set of 2,000 images, achieving near-perfect performance.
60
 
61
  | Metric | Score |
62
  |----------------------|---------|
63
+ | **Test Accuracy** | **0.9990** |
64
+ | **F1-Score (Macro)** | **0.9990** |
65
+ | Test Loss | 0.0202 |
66
+
67
+ ### Classification Report
68
+
69
+ | Class | Precision | Recall | F1-Score |
70
+ |----------------|-----------|--------|----------|
71
+ | AI Generated | 1.0000 | 0.9970 | 0.9985 |
72
+ | Deepfake | 0.9970 | 1.0000 | 0.9985 |
73
+ | Real | 1.0000 | 1.0000 | 1.0000 |
74
 
75
  ### Confusion Matrix
76
 
77
  The confusion matrix below shows the model's high precision and recall across all classes, with very few misclassifications.
78
 
79
+ ![Confusion Matrix](https://huggingface.co/einrafh/verichain-deepfake-models/resolve/main/assets/confusion_matrix.png)
80
 
81
  ## Citation
82