Update README.md

README.md

@@ -5,21 +5,45 @@ PyTorch Implementation of [Lumina-t2x](https://arxiv.org/abs/2405.05945)
 We will provide our implementation and pretrained models as open source in this repository recently.
 
 [![arXiv](https://img.shields.io/badge/arXiv-Paper-<COLOR>.svg)](https://arxiv.org/abs/2305.18474)
-[![Hugging Face](https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-blue)](https://huggingface.co/spaces/AIGC-Audio/Lumina-Audio)
+[![Hugging Face](https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-blue)](https://huggingface.co/spaces/AIGC-Audio/Make-An-Audio-3)
 [![GitHub Stars](https://img.shields.io/github/stars/Text-to-Audio/Make-An-Audio-3?style=social)](https://github.com/Text-to-Audio/Make-An-Audio-3)
 
 ## Use pretrained model
 We provide our implementation and pretrained models as open source in this repository.
 
 Visit our [demo page](https://make-an-audio-2.github.io/) for audio samples.
+
+## News
+- June, 2024: **[Make-An-Audio-3 (Lumina-Next)](https://arxiv.org/abs/2405.05945)** released in [Github](https://github.com/Text-to-Audio/Make-An-Audio-3).
+
+[//]: # (- May, 2024: **[Make-An-Audio-2]&#40;https://arxiv.org/abs/2207.06389&#41;** released in [Github]&#40;https://github.com/bytedance/Make-An-Audio-2&#41;.)
+[//]: # (- August, 2023: **[Make-An-Audio]&#40;https://arxiv.org/abs/2301.12661&#41; &#40;ICML 2022&#41;** released in [Github]&#40;https://github.com/Text-to-Audio/Make-An-Audio&#41;. )
+
+## Install dependencies
+
+Note: You may want to adjust the CUDA version [according to your driver version](https://docs.nvidia.com/deploy/cuda-compatibility/#default-to-minor-version).
+
+```bash
+conda create -n Make_An_Audio_3 -y
+conda activate Make_An_Audio_3
+conda install python=3.11 pytorch==2.1.0 torchvision==0.16.0 torchaudio==2.1.0 pytorch-cuda=12.1 -c pytorch -c nvidia -y
+pip install -r requirements.txt
+pip install flash-attn --no-build-isolation
+Install [nvidia apex](https://github.com/nvidia/apex) (optional)
+```
+
 ## Quick Started
 ### Pretrained Models
-Simply download the weights from [![Hugging Face](https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-blue)](https://huggingface.co/Alpha-VLLM/Lumina-T2Music).  
-- Text Encoder: [FLAN-T5-Large](https://huggingface.co/google/flan-t5-large)
-- VAE: Make-An-Audio 2, finetuned from [Make an Audio](https://github.com/Text-to-Audio/Make-An-Audio)
-- Decoder: [Vocoder](https://github.com/NVIDIA/BigVGAN)
-- `Music` Checkpoints: [huggingface](https://huggingface.co/Alpha-VLLM/Lumina-T2Music), `Audio` Checkpoints: [huggingface]()
 
+Simply download the 500M weights from [![Hugging Face](https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-blue)](https://huggingface.co/spaces/AIGC-Audio/Make-An-Audio-3/tree/main/useful_ckpts)
+
+ Model     | Pretraining Data   |  Path  
+|-----------|--------------------|--------------------------------------------------------------------------------
+| M (160M)  | AudioCaption       |[Here](https://huggingface.co/spaces/AIGC-Audio/Make-An-Audio-3/tree/main/useful_ckpts)
+| L (520M)  | AudioCaption       |[TBD]
+| XL (750M) | AudioCaption       |[TBD]
+| 3B        | AudioCaption       |[TBD]
+    
 ### Generate audio/music from text
 ```
 python3 scripts/txt2audio_for_2cap_flow.py 
@@ -38,7 +62,7 @@ python3 scripts/txt2audio_for_2cap_flow.py
 ### Generate audio/music from video
 ```
 python3 scripts/video2audio_flow.py 
---outdir output_dir -r  checkpoints_last.ckpt  -b configs/txt2audio-cfm1-cfg-LargeDiT3.yaml --scale 3.0 
+--outdir output_dir -r  checkpoints_last.ckpt  -b configs/video2audio-cfm1-cfg-LargeDiT1-moe.yaml --scale 3.0 
 --vocoder-ckpt useful_ckpts/bigvnat --test-dataset vggsound 
 ```
 
@@ -86,7 +110,7 @@ python main.py --base configs/research/text2audio/text2audio-ConcatDiT-ae1dnat_S
 ```
 
 ## Evaluation
-Please refer to [Make-An-Audio](https://github.com/Text-to-Audio/Make-An-Audio?tab=readme-ov-file#evaluation)
+Please refer to [Make-An-Audio](https://github.com/Text-to-Audio/Make-An-Audio?tab=readme-ov-file#evaluation).
 
 
 ## Acknowledgements