Audio-Text-to-Text
PEFT
Safetensors
English
mistral-lmm
dorienh commited on
Commit
b1a9a2b
·
verified ·
1 Parent(s): 4f94bec

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -2
README.md CHANGED
@@ -14,7 +14,7 @@ metrics:
14
  pipeline_tag: audio-text-to-text
15
  ---
16
 
17
- # Model Card for SonicVerse
18
 
19
  SonicVerse is a multi-task music captioning model that integrates caption generation with auxiliary music feature detection tasks such as key detection, vocals detection, and more. The model directly captures both low-level acoustic details as well as high-level musical attributes through a novel projection-based architecture that transforms audio input into natural language captions while simultaneously detecting music features through dedicated auxiliary heads. Additionally, SonicVerse enables the generation of temporally informed long captions for extended music pieces by chaining outputs from short segments using large language models, providing detailed time-informed descriptions that capture the evolving musical narrative.
20
 
@@ -25,7 +25,7 @@ View **demo** on [our HuggingFace Space](https://huggingface.co/spaces/amaai-lab
25
  **GitHub:** [https://github.com/AMAAI-Lab/SonicVerse](https://github.com/AMAAI-Lab/SonicVerse)
26
 
27
 
28
- ## How to Get Started with the Model
29
 
30
  Use the instructions provided on the [GitHub repository](https://github.com/AMAAI-Lab/SonicVerse) to run inference locally. Alternatively try out the model on the [Spaces demo](https://huggingface.co/spaces/amaai-lab/SonicVerse).
31
 
 
14
  pipeline_tag: audio-text-to-text
15
  ---
16
 
17
+ # SonicVerse: Multi-Task Learning for Music Feature-Informed Captioning
18
 
19
  SonicVerse is a multi-task music captioning model that integrates caption generation with auxiliary music feature detection tasks such as key detection, vocals detection, and more. The model directly captures both low-level acoustic details as well as high-level musical attributes through a novel projection-based architecture that transforms audio input into natural language captions while simultaneously detecting music features through dedicated auxiliary heads. Additionally, SonicVerse enables the generation of temporally informed long captions for extended music pieces by chaining outputs from short segments using large language models, providing detailed time-informed descriptions that capture the evolving musical narrative.
20
 
 
25
  **GitHub:** [https://github.com/AMAAI-Lab/SonicVerse](https://github.com/AMAAI-Lab/SonicVerse)
26
 
27
 
28
+ ## How to Get Started
29
 
30
  Use the instructions provided on the [GitHub repository](https://github.com/AMAAI-Lab/SonicVerse) to run inference locally. Alternatively try out the model on the [Spaces demo](https://huggingface.co/spaces/amaai-lab/SonicVerse).
31