YuE2: Open Music Generation With an Editable Score

Where Open Music Generation Stands After YuE2

YuE2 is an open-source lyrics-to-song system that does something the commercial tools do not: it shows you the composition before it renders the sound. You give it lyrics and a style prompt, it writes a melody-and-chord plan in ABC notation, and only then does it turn that plan into a full 48 kHz stereo song with vocals and accompaniment. On WildSongBench, a 192-prompt automatic evaluation dated September 12, 2026, its best-of-8 setting posts a SongBench average of 6.9632, the highest observed mean among all 17 evaluated settings, ahead of Mureka 9 and Suno v5. The code is Apache 2.0, the weights are CC BY-NC 4.0, and the whole thing runs on a single 24 GB GPU. The catch, and there is always a catch, is that the headline number comes from picking the best of eight generations, and the single-pass score sits below Suno v5.

Why YuE2 Music Generation Matters Right Now

Here is the thing about most AI music tools. You type a prompt, you get audio, and whatever the model decided about the key, the tempo, the chord progression, that decision is buried in the waveform. You do not like the bridge? Re-roll and pray. That is the entire interaction model.

YuE2, from the Multimodal Art Projection group and a list of institutions that includes HKUST, NYU, Stanford, MBZUAI, Tokenwave.AI, NOIZ and ACE Studio, takes a different route. It is the successor to YuE, the open long-form lyrics-to-song foundation model whose technical report described generating up to five minutes of music with lyrical alignment. The original YuE arrived in January 2025 as the first publicly available open-source lyrics-to-song model claiming quality on par with commercial systems. YuE2 keeps the ambition and changes the interface: the composition is now explicit, readable and editable.

Why should a professional care? Because control is the difference between a toy and a tool. A system that emits a finished mix gives you a demo. A system that emits a score gives you something you can work with.

How the AI Song Generation Pipeline Actually Works

So what is under the hood. One AR-NAR Mixture-of-Transformers backbone does the heavy lifting. It predicts the score and semantic music tokens autoregressively, then generates acoustic latents with flow matching. A VAE decodes those latents into stereo audio. No quantization step. The output is 48 kHz stereo.

The staged Python API exposes the pipeline as four calls: plan(), generate_semantic(), synthesize(), decode(). Creation, covering and editing all run on the same generation checkpoint. They differ only in where the score comes from: YuE2 writes it, SheetSage2 transcribes it from a recording, or you edit it yourself.

Symbolic planning

The model first writes a melody-and-chord plan as an ABC notation score. That score is plain text. You can read it, play it, change the chord under the second chorus, and re-render. The audio follows the edit.

Flow matching

The technique the backbone uses to generate the acoustic latents, the intermediate representation the VAE turns into sound.

PER

Phoneme error rate, the measure of how clearly the generated vocals pronounce the lyrics. Lower is better, and it matters more than people expect, because a beautiful song with garbled words is a garbled song.

Best-of-8 (Bo8)

Generating eight candidates and keeping the best one, selected by SongBench Musicality, then prompt control, then PER. Standard YuE2 selects from just two candidates.

Three modes: create, cover, edit

The cot parameter controls how much planning happens. cot="full" generates the full melody-and-chord plan and is the default for new songs. cot="melody" uses a melody plan with free accompaniment, which is what the project recommends for covers, so the accompaniment can adapt to the new style. cot="off" skips planning and generates directly from lyrics and style. And if you already have a score, you pass it in with abc=... and the model renders your composition instead of its own.

Running YuE2-3B Locally on a Single 24 GB GPU

The requirements are stated plainly: Linux, Python 3.12, an NVIDIA GPU with BF16 support and 24 GB of VRAM. Model files download from Hugging Face on first use. The install is a clone, a virtualenv and a pip install:

A graphics card resting on a grand piano in a dim studio, with sheet music pages fanned out nearby.

git clone https://github.com/multimodal-art-projection/YuE.git, then python -m pip install ., then python examples/generate.py --output outputs/first-song. Open outputs/first-song/audio.flac and you have your first song. The output directory also keeps the score, the semantic tokens, the acoustic latents, the generation settings and the model identities, which is exactly what you want when someone asks “how did you make this.”

Speed is not the problem either. According to the YuE2-3B model card, a 3.6-minute song generates in 71 seconds on an RTX 4090. The separate YuE2-Vae card reports about 3.6 seconds to decode that same song, with full-pipeline peak memory around 11.18 GiB on a 4090. So the 24 GB requirement has real headroom in practice.

WildSongBench: YuE2 vs Suno v5 and v6, With the Actual Numbers

Now the part everyone will quote, and most will quote wrong. WildSongBench is a 192-prompt automatic evaluation. The September 12, 2026 comparison covers 17 settings: public models, proprietary systems, and two YuE2 configurations. Both YuE2 settings use symbolic planning and the benchmark decoder, YuE2-Vae-legacy.

System / setting	SongBench Avg ↑	AudioBox PQ ↑	MuLan ↑	PER ↓
YuE2 (best-of-8)	6.9632	8.2714	0.5051	9.79%
Mureka 9	6.9377	8.0226	0.4394	11.69%
Suno v5	6.8721	8.1698	0.5428	8.10%
YuE2 (standard)	6.7316	8.2598	0.5068	8.44%
Suno v5.5	6.7150	8.1955	0.5089	5.96%
Suno v6	6.5562	8.1296	0.4916	7.58%
YuE 1	4.9165	7.8683	0.2623	36.38%

YuE2 (best-of-8) posts 6.9632 on SongBench Avg, the highest observed mean among all 17 evaluated settings, while standard single-selection YuE2 scores 6.7316, below Suno v5’s 6.8721.WildSongBench results, September 12, 2026

Read the table, not the headline. Three honest observations. First, the gap at the top is small: 6.9632 against 6.9377 for Mureka 9, and the project itself says the small gap between the highest means does not establish statistical significance. Second, the headline number is best-of-8, meaning eight generations were sampled and the best kept. Standard YuE2, selecting from two candidates, lands at 6.7316, inside the proprietary quality range but below Suno v5. Third, rankings vary by metric. Suno v5 still leads on MuLan text alignment (0.5428 to YuE2’s 0.5068), and Suno v4.5 and v5.5 post lower phoneme error rates than either YuE2 setting. The win is concentrated in overall song quality and musicality, not in prompt adherence or pronunciation.

The WildSongBench results page is unusually candid about all of this: the results support frontier quality and competitiveness with Suno v5/v6, and they do not establish universal metric superiority or human preference. When a benchmark page hedges its own headline, believe the hedge.

One more number deserves attention, and it is not at the top of the table. YuE 1 scores 4.9165. YuE2 nearly laps its own predecessor. That is the more striking figure in the whole comparison.

Zero-shot covers: the score is doing the work

On 948 works, full-score YuE2 reaches 0.647 CLEWS mAP for source-identity preservation, compared with 0.006 without a score. Read that again. With the score, 0.647. Without it, basically zero. The symbolic plan is not a gimmick bolted on for marketing. It is the mechanism that makes a cover a cover instead of a hallucination in the same genre. And this uses the general generator, with no cover-specific fine-tuning.

Overhead view of a studio mixing desk with faders, scattered notation sheets, a green GPU card, and coiled cables.

SheetSage2, MERT2 and the Zero-Shot Cover Workflow

YuE2 does not work alone. Two companion models feed it, and both are released alongside it.

SheetSage2 is the audio-to-score transcription model: state of the art on 10 of 13 benchmark metrics, with 82.51% vocal melody pitch-class F1 on RWC-Pop. You point it at a source recording, it produces the melody ABC, you review that score, supply new lyrics or a target style, and YuE2 renders a new interpretation. The cover guide recommends cot="melody" and a score without chord symbols, so the accompaniment can adapt. SheetSage2 runs in a separate environment and loads its MERT2 encoder automatically.

MERT2 is the music understanding model: state of the art on 14 of 15 MARBLE metrics, with 91.72% genre accuracy on GTZAN. It ships in two variants, MERT-v2-FullSong for full songs and MERT-v2-30s for short recordings, and it serves as SheetSage2’s encoder. For generation itself it is optional: YuE2’s pipeline does not require a separate MERT2 download.

Agentic Music Editing With the yue2-music Skill

Because the score is text, an agent can edit it. The repository ships a yue2-music skill under skills/yue2-music/, a SKILL.md package that teaches an agent how to generate songs, transcribe and cover recordings, edit ABC scores, check musical invariants and organize listening comparisons. You install it through your agent’s skill-directory or import mechanism, with the Python runtime installed separately.

The demo the project shows off follows one song, “The Last Train,” through 9 steps and 14 versions, from Mandarin pop to English jazz with new harmony and a saxophone solo. Each version keeps its conversation, score, prompt and lyrics inspectable. The suggested test request is telling: create an English piano-pop song, keep the original audio and score, make a second version with jazz harmony, preserve the vocal melody and lyric order, and deliver both for comparison. That is a workflow, not a prompt.

One honest limitation the project states itself: editing generates a new complete recording. It does not preserve the original waveform outside the edit. You are re-rendering the song from a changed score, not patching the audio.

Licensing: Read This Before You Build on It

Here the picture splits, and the split matters. YuE2’s first-party code, the agent skill and the documentation are Apache 2.0. The model weights are separately licensed under CC BY-NC 4.0. Non-commercial. Third-party components retain their original licenses, and the archived YuE-v1 branch keeps its original license.

So: research, teaching, personal projects, fine. A commercial product on the raw weights, no. That is a deliberate posture, and for a decision-maker it is the line that decides whether YuE2 is a platform or a reference. The editable score and the open benchmark make it valuable either way, but only one of those ways involves revenue.

A holographic waveform floating above a mixing console, with sheet music swirling in the air and a GPU on a stand.

What Is Still Missing, and What Comes Next

The YuE2 technical report is listed as coming soon. For now the project asks to be cited through the MERT and YuE papers. That means the training details, the data provenance and the ablations that would let outsiders properly audit the claims are not yet public in a formal paper. The benchmark is open, the prompts are open, the weights are open, and that is more than the proprietary systems offer, but the full report is the document to wait for.

The other open question is pronunciation. A PER of 8.44% for standard YuE2 against 5.80% for Suno v4.5 says the vocals still lag the best commercial systems on clarity, and best-of-8 actually trades pronunciation away (9.79%) to gain musicality. If the next iteration closes that gap while keeping the editable score, the conversation about open versus closed music generation changes completely.

📊 What YuE2 Means for People Who Build With Music AI

The score is the product: an editable ABC melody-and-chord plan turns music generation from a lottery into a workflow, and the 0.647 vs 0.006 CLEWS mAP cover result proves the score is load-bearing, not decorative.
The benchmark lead is real but narrow: 6.9632 best-of-8 is the highest observed SongBench average, yet standard YuE2 (6.7316) sits below Suno v5, and Suno still leads on text alignment and phoneme error rate.
It runs on one GPU: 24 GB VRAM, Linux, Python 3.12, 71 seconds for a 3.6-minute song on an RTX 4090, no quantization.
The license decides your use case: Apache 2.0 code, CC BY-NC 4.0 weights. Research and teaching yes, commercial deployment of the weights no.
The ecosystem is the moat: MERT2 (14 of 15 MARBLE metrics), SheetSage2 (10 of 13 transcription benchmarks) and the yue2-music agent skill make this a platform for music work, not just a generator.

Additional Resources

This article includes content created with AI.