EPFL-VILAB/VPE-ViPer

ViPer: Visual Personalization of Generative Models via Individual Preference Learning

Tuning-free framework for personalized image generation

Website | Paper | GitHub | BibTeX

We introduce ViPer, a method that personalizes the output of generative models to align with different users’ visual preferences for the same prompt. This is done via a one-time capture of the user’s general preferences and conditioning the generative model on them without the need for engineering detailed prompts.

Installation

For install instructions, please see https://github.com/EPFL-VILAB/ViPer.

Usage

This model can be loaded from Hugging Face Hub as follows:

from transformers import AutoModelForVision2Seq
from peft import PeftModel

model = AutoModelForVision2Seq.from_pretrained("HuggingFaceM4/idefics2-8b")
model = PeftModel.from_pretrained(model, "EPFL-VILAB/VPE-ViPer")

Please see https://github.com/EPFL-VILAB/ViPer for more detailed instructions.

For more examples and interactive demos, please see our website and Hugging Face Space.

Citation

If you find this repository helpful, please consider citing our work:

@article{ViPer,
  title={{ViPer}: Visual Personalization of Generative Models via Individual Preference Learning},
  author={Sogand Salehi and Mahdi Shafiei and Teresa Yeo and Roman Bachmann and Amir Zamir},
  journal={arXiv preprint arXiv:2407.17365},
  year={2024},
}

License

Licensed under the Apache License, Version 2.0. See LICENSE for details.