--- license: mit base_model: - timm/eva02_base_patch14_448.mim_in22k_ft_in22k_in1k pipeline_tag: image-classification tags: - pytorch - transformers --- # EVA-based Fast NSFW Image Classifier ## Table of Contents - [Model Description](#model-description) - [Try it Online!](#try-it-online-) - [Model Performance Comparison](#model-performance-comparison) - [Global Performance](#global-performance) - [Accuracy by AI Content](#accuracy-by-ai-content) - [AI-Generated Content](#ai-generated-content) - [Non-AI-Generated Content](#non-ai-generated-content) - [Usage](#usage) - [Quick Start via pip](#quick-start-via-pip) - [Quick Start with Pipeline](#quick-start-with-pipeline) - [Avoid installation of pip dependency](#avoid-installation-of-pip-dependency) - [Training](#training) - [Speed and Memory Metrics](#speed-and-memory-metrics) ## Model Description This model is a vision transformer based on the **EVA architecture**, fine-tuned for **NSFW content classification**. It has been trained to detect **four categories** (neutral, low, medium, high) of visual content using **100,000 synthetically labeled images**. The model can be used as a **binary (true/false) classifier if desired, or you can obtain the full output probabilities.**. It **outperforms other excellent publicly available models** such as [Falconsai/nsfw_image_detection](https://huggingface.co/Falconsai/nsfw_image_detection) or [AdamCodd/vit-base-nsfw-detector](https://huggingface.co/AdamCodd/vit-base-nsfw-detector) in our internal benchmarks adding the enrichment of being able to select the NSFW level that suits your use case. ## Try it Online! 🚀 You can try this model directly in your browser through our [Hugging Face Space](https://huggingface.co/spaces/ccabrerafreepik/nsfw_image_detector). Upload any image and get instant NSFW classification results without any installation required. ## Model Performance Comparison ### Global Performance | Category | Freepik | Falconsai | Adamcodd | |----------|-------------|------------------|----------------| | High | 99.54% | 97.92% | 98.62% | | Medium | 97.02% | 78.54% | 91.65% | | Low | 98.31% | 31.25% | 89.66% | | Neutral | 99.87% | 99.27% | 98.37% | In the table below, the results are obtained as follows: * For the **Falconsai and AdamCodd** models: * A prediction is considered correct if the image is labeled "low", "medium", or "high" and the model returns true. * If the label is "neutral", the correct output should be false. * For the **Freepik model**: * If the image label is "low", "medium", or "high", the model should return at least "low". * If the label is "neutral", the correct output should be "neutral". **Conclusions:** * Our model **outperforms AdamCodd and Falconsai in accuracy**. It is entirely fair to compare them on the "high" and "neutral" labels. * Our model **offers greater granularity**. It is not only suitable for detecting "high" and "neutral" content, but also performs excellently at identifying "low" and "medium" NSFW content. * Falconsai may classify some "medium" and "low" images as not NSFW but mark others as safe for work(SFW), which could lead to unexpected results. * AdamCodd classifies both "low" and "medium" categories as NSFW, which may not be desirable depending on your use case. Furthermore, a 10% of images in low and medium are considered SFW. ### Accuracy by AI Content We have created a **manually labeled dataset** with careful attention to **avoiding biases** (gender, ethnicity, etc.). While the sample size is relatively small, it provides meaningful insights into model performance across different scenarios, which was very useful in the training process to avoid biases. The following tables show detection accuracy percentages across different NSFW categories and content types: #### AI-Generated Content | Category | Freepik Model | Falconsai Model | Adamcodd Model | |----------|-------------|------------------|----------------| | High | 100.00% | 84.00% | 92.00% | | Medium | 96.15% | 69.23% | 96.00% | | Low | 100.00% | 35.71% | 92.86% | | Neutral | 100.00% | 100.00% | 66.67% | **Conclusions:** * **Avoid using Falconsai for AI-generated content** to prevent prediction errors. * **Our model is the best option to detect NSFW content in AI-generated content**. ## Usage ### Quick Start via pip ```sh pip install nsfw-image-detector ``` ```python from PIL import Image from nsfw_image_detector import NSFWDetector import torch # Initialize the detector detector = NSFWDetector(dtype=torch.bfloat16, device="cuda") # Load and classify an image image = Image.open("your_image") # Check if the image contains NSFW content sentivity level medium or higher is_nsfw = detector.is_nsfw(image, "medium") # Get probability scores for all categories probabilities = detector.predict_proba(image) print(f"Is NSFW: {is_nsfw}") print(f"Probabilities: {probabilities}") ``` Example output: ```python Is NSFW: False Probabilities: [ {: 0.00372314453125, : 0.1884765625, : 0.234375, : 0.765625} ] ``` ### Quick Start with Pipeline ```python from transformers import pipeline from PIL import Image # Create classifier pipeline classifier = pipeline( "image-classification", model="Freepik/nsfw_image_detector", device=0 # Use GPU (0) or CPU (-1) ) # Load and classify an image image = Image.open("path/to/your/image.jpg") predictions = classifier(image) print(predictions) ``` Example output: ```python [ {'label': 'neutral', 'score': 0.92}, {'label': 'low', 'score': 0.05}, {'label': 'medium', 'score': 0.02}, {'label': 'high', 'score': 0.01} ] ``` The model supports efficient batch processing for multiple images: ```python images = [Image.open(path) for path in ["image1.jpg", "image2.jpg", "image3.jpg"]] predictions = classifier(images) ``` **Note**: If the intention is to use the model in production review [Speed and Memory Metrics](#speed-and-memory-metrics) section before using this approach. ### Avoid installation of pip dependency The following example demonstrates how to **customize the NSFW detection label**, it is very similar to the code in [PyPy](https://pypi.org/project/nsfw-image-detector/0.1.0/). This code returns True if the NSFW level is 'medium' or higher: ```python from transformers import AutoModelForImageClassification import torch from PIL import Image from typing import List, Dict import torch.nn.functional as F from timm.data.transforms_factory import create_transform from torchvision.transforms import Compose from timm.data import resolve_data_config from timm.models import get_pretrained_cfg device = "cuda" if torch.cuda.is_available() else "cpu" # Load model and processor model = AutoModelForImageClassification.from_pretrained("Freepik/nsfw_image_detector", torch_dtype = torch.bfloat16).to(device) # Load original processor (faster for tensors) cfg = get_pretrained_cfg("eva02_base_patch14_448.mim_in22k_ft_in22k_in1k") processor: Compose = create_transform(**resolve_data_config(cfg.__dict__)) def predict_batch_values(model, processor: Compose, img_batch: List[Image.Image] | torch.Tensor) -> List[Dict[str, float]]: """ Process a batch of images and return prediction scores for each NSFW category """ idx_to_label = {0: 'neutral', 1: 'low', 2: 'medium', 3: 'high'} # Prepare batch inputs = torch.stack([processor(img) for img in img_batch]) output = [] with torch.inference_mode(): logits = model(inputs).logits batch_probs = F.log_softmax(logits, dim=-1) batch_probs = torch.exp(batch_probs).cpu() for i in range(len(batch_probs)): element_probs = batch_probs[i] output_img = {} danger_cum_sum = 0 for j in range(len(element_probs) - 1, -1, -1): danger_cum_sum += element_probs[j] if j == 0: danger_cum_sum = element_probs[j] output_img[idx_to_label[j]] = danger_cum_sum.item() output.append(output_img) return output def prediction(model, processor, img_batch: List[Image.Image], class_to_predict: str, threshold: float=0.5) -> List[bool]: """ Predict if images meet or exceed a specific NSFW threshold """ if class_to_predict not in ["low", "medium", "high"]: raise ValueError("class_to_predict must be one of: low, medium, high") if not 0 <= threshold <= 1: raise ValueError("threshold must be between 0 and 1") output = predict_batch_values(model, processor, img_batch) return [output[i][class_to_predict] >= threshold for i in range(len(output))] # Example usage image = Image.open("path/to/your/image.jpg") print(predict_batch_values(model, processor, [image])) print(prediction(model, processor, [image], "medium")) # Options: low, medium, high ``` Example output: ```python [ {'label': 'neutral', 'score': 0.92}, {'label': 'low', 'score': 0.08}, {'label': 'medium', 'score': 0.03}, {'label': 'high', 'score': 0.01} ] [False] ``` **Note**: The sum is higher than one because the prediction is the cumulative sum of all labels equal to or higher than your selected label, except neutral. For instance, if you select 'medium', it is the sum of 'medium' and 'high'. In our opinion, this approach is more effective than selecting only the highest probability label. ## Training * **100,000 images** were used during training. * The model were trained for **3 epochs on 3 NVIDIA GeForce RTX 3090** * The model were trained using two sets, training and validation. * There are **no images with a cosine similarity higher than 0.92** to avoid duplicates and biases between training and validation. The model used for deduplication is "openai/clip-vit-base-patch32" * A **custom loss** was created to minimize predictions that are lower than the true class. For instance, it is very rare for an image labeled as 'high' to be predicted as 'neutral' (this only happens 0.46% of the time). ## Speed and Memory Metrics | Batch Size | Avg by batch (ms) | VRAM (MB) | Optimizations | |------------|------------------|------------|---------------| | 1 | 28 | 540 | BF16 using PIL images | | 4 | 110 | 640 | BF16 using PIL images | | 16 | 412 | 1144 | BF16 using PIL images | | 1 | 10 | 540 | BF16 using torch tensor | | 4 | 33 | 640 | BF16 using torch tensor | | 16 | 102 | 1144 | BF16 using torch tensor | **Notes:** * The model has been trained in bf16 so it is **recommended to use it in bf16**. * **Using torch tensor**: The speed using torch tensor is not achieved using pipeline. Avoid pipeline use in production. * Measurements taken on **NVIDIA RTX 3090**, expect better metrics in more powerful servers. * Throughput increases with larger batch sizes due to better GPU utilization. Consider your use case when selecting batch size. * Optimizations listed are suggestions that could further improve performance. * **Using torch tensors is specially indicated** in cases such as use the model for **text to image models or similar** because the output is already in tensor format. ## License This project is licensed under the MIT License - Copyright 2025 Freepik Company S.L. ## Citation If you use this model in your research or project, please cite it as: ```bibtex @software{freepik2025nsfw, title={EVA-based Fast NSFW Image Classifier}, author={Freepik Company S.L.}, year={2025}, publisher={Hugging Face}, url = {https://huggingface.co/Freepik/nsfw_image_detector}, organization = {Freepik Company S.L.} } ``` ## Acknowledgements This model is based on the EVA architecture ([timm/eva02_base_patch14_448.mim_in22k_ft_in22k_in1k](https://huggingface.co/timm/eva02_base_patch14_448.mim_in22k_ft_in22k_in1k)), as described in the following paper: EVA-02: A Visual Representation for Neon Genesis - https://arxiv.org/abs/2303.11331 ```bibtex @article{EVA02, title={EVA-02: A Visual Representation for Neon Genesis}, author={Fang, Yuxin and Sun, Quan and Wang, Xinggang and Huang, Tiejun and Wang, Xinlong and Cao, Yue}, journal={arXiv preprint arXiv:2303.11331}, year={2023} } ```