You need to agree to share your Hugging Face information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

Log in or Sign Up to review the conditions and access this model content.

Legato 2

Full-page optical music recognition. A detector finds the systems on a page and a transcriber reads each one in turn, conditioned on the ABC it has already produced for the systems above it, so the notation stays consistent down the page.

Both halves live here: model.safetensors is the transcriber, segmenter.pt the YOLO line detector. They are versioned together because a change to either one changes the result.

Usage

from legato.pipeline import Legato2

omr = Legato2.from_pretrained("guangyangmusic/legato-2", device="cuda:0")
abc = omr("page.png")

Decoding follows the paper by default: greedy (beam size 1), repetition penalty 1.1, maximum length 2048; segmentation uses confidence 0.7 at image size 640. All of these can be overridden per call, for example omr("page.png", beam_size=10, yolo_conf=0.5).

A PDF can be transcribed from the command line, where it is first rendered to one tall image so a multi-page score is read as a single piece:

python scripts/run_legato2.py --image_path score.pdf --model_path guangyangmusic/legato-2

Checkpoint

legato-segment-bytes-tokenizer-lr-3e-4-cosine-e5-b32-v4096: byte-level tokenizer with a 4096-entry vocabulary, learning rate 3e-4 on a cosine schedule, five epochs at batch size 32.

Evaluation

OMR-NED, lower is better. Scored with our musicdiff fork (5.2) on music21 9.9.1 and converter21 4.0.1; the ABC is converted to MusicXML with abc2xml followed by a MuseScore 3.6.2 rewrite, which is part of the measurement and shifts the figures by one to two points if skipped.

Dataset Legato 2 Legato 1 Audiveris Gemini 3.1 Pro
Camera String Quartets 31.6 58.3 69.3 94.8
Rendered String Quartets 17.1 32.6 36.8 93.5
Camera OpenScore Lieder 38.3 49.0 52.1 87.2
Rendered OpenScore Lieder 27.7 34.4 29.9 86.7
IMSLP Piano Scores 36.6 43.8 66.0 89.1
PDMX-Synth test 21.7 27.4 34.7 85.2

License

The two files in this repository are licensed separately:

File License
model.safetensors Llama 3.2 Community License. Its vision encoder is the encoder of Llama 3.2 11B Vision, kept frozen during training.
segmenter.pt AGPL-3.0. It was trained with Ultralytics YOLO.

Built with Llama. Llama 3.2 is licensed under the Llama 3.2 Community License, Copyright ยฉ Meta Platforms, Inc. All Rights Reserved.

Downloads last month
8
Safetensors
Model size
0.9B params
Tensor type
BF16
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support