Papers
arxiv:2607.13250

AffectFlow-DINO: Uncertainty-Aware Multi-Task Affect Estimation via Conditional Rectified Flow

Published on Jul 14
· Submitted by
Salah Eddine Bekhouche
on Jul 16
Authors:
,
,
,

Abstract

We present AffectFlow-DINO, a multi-task learning system for the 11th ABAW challenge that extends a standard deterministic architecture with a conditional rectified-flow head to model the inherent ambiguity of in-the-wild facial behavior. Instead of predicting a single affect estimate, the model learns a conditional generative distribution, enabling uncertainty-aware one-to-many predictions through Monte Carlo sampling. The system jointly estimates continuous valence-arousal, classifies eight facial expressions, and detects twelve Action Units from static face images. Built on a frozen DINOv3 ViT-S/16 backbone, extensive ablation studies show that rectified-flow decoding consistently improves deterministic prediction, particularly for valence-arousal estimation (CCC-V +0.058). We further show that post-hoc threshold calibration effectively recovers performance on severely imbalanced rare classes (e.g., Fear: 3.8% rightarrow 33.1%) without retraining. Combined with backbone fine-tuning and flow retuning, the final model achieves P_{MTL=1.177}, substantially outperforming the official challenge baseline of P_{MTL}=0.45.

Community

Paper submitter

AffectFlow-DINO models in-the-wild facial affect as a conditional distribution p(y|x), not a single point estimate.

Built on DINOv3, it jointly predicts valence–arousal, 8 expressions, and 12 Action Units, and adds a conditional rectified-flow head for uncertainty-aware, one-to-many predictions on ambiguous faces.

On the 11th ABAW MTL validation set we reach P_MTL = 1.177 (vs. official baseline 0.450), with ablations showing when flow decoding helps and how simple post-hoc calibration recovers rare classes (e.g. Fear F1 3.8% → 33.1%).

Code, pretrained weights, and inference scripts are open:
• Paper: https://arxiv.org/abs/2607.13250
• Code: https://github.com/Bekhouche/AffectFlow-DINO
• Models: https://huggingface.co/Bekhouche/AffectFlow-DINO

Try it in one line:
python inference.py --model finetune-flow-retune-b10 --image face.jpg

This is an automated message from the Librarian Bot. I found the following papers similar to this paper.

The following papers were recommended by the Semantic Scholar API

Please give a thumbs up to this comment if you found it helpful!

If you want recommendations for any Paper on Hugging Face checkout this Space

You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: @librarian-bot recommend

Sign up or log in to comment

Get this paper in your agent:

hf papers read 2607.13250
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 1

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2607.13250 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2607.13250 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.