Update README.md

README.md

@@ -8,7 +8,153 @@ tags:
 - model_hub_mixin
 - pytorch_model_hub_mixin
 ---
+- Library: https://github.com/nicolas-dufour/CAD 
+<div align="center">
 
-This model has been pushed to the Hub using the [PytorchModelHubMixin](https://huggingface.co/docs/huggingface_hub/package_reference/mixins#huggingface_hub.PyTorchModelHubMixin) integration:
-- Library: https://github.com/nicolas-dufour/CAD
-- Docs: [More Information Needed]
+# Don’t drop your samples! Coherence-aware training benefits Conditional diffusion
+
+<a href="https://nicolas-dufour.github.io/" >Nicolas Dufour</a>, <a href="https://scholar.google.com/citations?user=n_C2h-QAAAAJ&hl=fr&oi=ao" >Victor Besnier</a>, <a href="https://vicky.kalogeiton.info/" >Vicky Kalogeiton</a>, <a href="https://davidpicard.github.io/" >David Picard</a>
+
+
+![CAD](https://nicolas-dufour.github.io/assets/publications/cad_cvpr/varying_coherence.png)
+
+</div>
+This repo has the code for the paper "Dont Drop Your samples: Coherence-aware training benefits Condition Diffusion" accepted at CVPR 2024 as a Highlight.
+
+The core idea is that diffusion model is usually trained on noisy data. The usual solution is to filter massive datapools. We propose a new training method that leverages the coherence of the data to improve the training of diffusion models. We show that this method improves the quality of the generated samples on several datasets.
+
+Project website: [https://nicolas-dufour.github.io/cad](https://nicolas-dufour.github.io/cad)
+
+## Install
+
+To install, first create a conda env with python 3.10
+
+```bash
+conda create -n cad python=3.10
+```
+Activate the env
+
+```bash
+conda activate cad
+```
+
+For inference only,
+
+```bash
+pip install cad-diffusion
+```
+
+## Pretrained models
+
+To use the pretrained model do the following:
+```python
+from cad import CADT2IPipeline
+
+pipe = CADT2IPipeline("nicolas-dufour/CAD_512").to("cuda")
+
+prompt = "An avocado armchair"
+
+image = pipe(prompt, cfg=15)
+```
+
+If you just want to download the models, not the sampling pipeline, you can do:
+
+```python
+from cad import CAD
+
+model = CAD.from_pretrained("nicolas-dufour/CAD_512")
+```
+
+Models are hosted in the hugging face hub. The previous scripts download them automatically, but weights can be found at:
+
+[https://huggingface.co/nicolas-dufour/CAD_256](https://huggingface.co/nicolas-dufour/CAD_256)
+
+[https://huggingface.co/nicolas-dufour/CAD_512](https://huggingface.co/nicolas-dufour/CAD_512)
+
+## Using the Pipeline
+
+The `CADT2IPipeline` class provides a comprehensive interface for generating images from text prompts. Here's a detailed guide on how to use it:
+
+### Basic Usage
+
+```python
+from cad import CADT2IPipeline
+
+# Initialize the pipeline
+pipe = CADT2IPipeline("nicolas-dufour/CAD_512").to("cuda")
+
+# Generate an image from a prompt
+prompt = "An avocado armchair"
+image = pipe(prompt, cfg=15)
+```
+
+### Advanced Configuration
+
+The pipeline can be initialized with several customization options:
+
+```python
+pipe = CADT2IPipeline(
+    model_path="nicolas-dufour/CAD_512",
+    sampler="ddim",                    # Options: "ddim", "ddpm", "dpm", "dpm_2S", "dpm_2M"
+    scheduler="sigmoid",               # Options: "sigmoid", "cosine", "linear"
+    postprocessing="sd_1_5_vae",      # Options: "consistency-decoder", "sd_1_5_vae"
+    scheduler_start=-3,
+    scheduler_end=3,
+    scheduler_tau=1.1,
+    device="cuda"
+)
+```
+
+### Generation Parameters
+
+The pipeline's `__call__` method accepts various parameters to control the generation process:
+
+```python
+image = pipe(
+    cond="A beautiful landscape",          # Text prompt or list of prompts
+    num_samples=4,                         # Number of images to generate
+    cfg=15,                               # Classifier-free guidance scale
+    guidance_type="constant",             # Type of guidance: "constant", "linear"
+    guidance_start_step=0,                # Step to start guidance
+    coherence_value=1.0,                  # Coherence value for sampling
+    uncoherence_value=0.0,                # Uncoherence value for sampling
+    thresholding_type="clamp",           # Type of thresholding: "clamp", "dynamic_thresholding", "per_channel_dynamic_thresholding"
+    clamp_value=1.0,                      # Clamp value for thresholding
+    thresholding_percentile=0.995         # Percentile for thresholding
+)
+```
+
+#### Guidance Types
+- `constant`: Applies uniform guidance throughout the sampling process
+- `linear`: Linearly increases guidance strength from start to end
+- `exponential`: Exponentially increases guidance strength from start to end
+
+#### Thresholding Types
+- `clamp`: Clamps values to a fixed range using `clamp_value`
+- `dynamic`: Dynamically adjusts thresholds based on the batch statistics
+- `percentile`: Uses percentile-based thresholding with `thresholding_percentile`
+
+### Advanced Parameters
+
+For more control over the generation process, you can also specify:
+
+- `x_N`: Initial noise tensor
+- `latents`: Previous latents for continuation
+- `num_steps`: Custom number of sampling steps
+- `sampler`: Custom sampler function
+- `scheduler`: Custom scheduler function
+- `guidance_start_step`: Step to start guidance
+- `generator`: Random number generator for reproducibility
+- `unconfident_prompt`: Custom unconfident prompt text
+
+## Citation
+If you happen to use this repo in your experiments, you can acknowledge us by citing the following paper:
+
+```bibtex
+@article{dufour2024dont,
+  title={Don’t drop your samples! Coherence-aware training benefits Conditional diffusion},
+  author={Nicolas Dufour and Victor Besnier and Vicky Kalogeiton and David Picard},
+  journal={CVPR}
+  year={2024}
+}
+```