[Mac] YuE2 Songs in ComfyUI (16GB+ RAM, M1+)

Full songs with vocals from style tags and lyrics, in ComfyUI, on a 16 GB Mac. Covers and song extensions too.

It runs YuE2 through yue2.cpp on Metal instead of PyTorch, so it fits where the official release wants a 24 GB NVIDIA card.

Tested on an M1 MacBook Air 16 GB.

Included:

• YuE2 Song: style, lyrics ([Verse], [Chorus], [Bridge]...), seed.

• YuE2 Render: score, melody and audio in one go. Outputs normal ComfyUI AUDIO.

• YuE2 Plan: the ABC score alone, in seconds. Edit it, feed it back, keep the melody.

• YuE2 Transcribe: the melody of a recording. Cover it with your lyrics and style.

• YuE2 Tokenize: a recording as YuE2 codes. Start a new song from a section of it, or re-render it.

Render times:

                M1 Air 16 GB     M5 Pro 48 GB     M5 Pro 48 GB
Software        nodes, Q8 Metal  nodes, Q8 Metal  official Python
Plan (score)    30 s, 55 s song  14-27 s, 3-5 min 72-113 s, 3-4 min
55 s song       8 min            47 s             -
Full song       3:53 in 54 min   3:14 in 3.5 min  4:02 in 22 min
55 s cover      8 min            56 s             transcribe 1-3 min
Extend 20 s     8.5 min (+55 s)  43 s (+35 s)     -
Re-render 55 s  6 min            37 s             -
Peak memory     11.8-13.5 GB     5.3-8.1 GB       64 GB
Swap            none             none             36 GB
Longest song    3:53 tested      3:53 tested      OOM above ~5:30

Both M5 columns are the same machine, so they compare the software: these nodes render a full song about 5x faster than the official Python, in a twelfth of the memory. The two node columns compare the chips. Extend on the M5 used a 55 s source, which caps the new song at the source length.

Length is the lever: the acoustic stage grows faster than the song. Plan first, render the keepers.

Install:

1. brew install cmake, and xcode-select --install

2. Clone into custom_nodes: github.com/Bambushu/ComfyUI-YuE2-Mac

3. bash ComfyUI-YuE2-Mac/build.sh (about 1 min; ComfyUI-Manager does this for you)

4. Restart ComfyUI, drop in a workflow from this zip.

Models download on the first run (4.3 GB, 1 GB more for the audio nodes) into models/yue2.

Covers: transcribe melody only, set cot to melody, give the lyrics the same sections and roughly the same syllables per line as the original. Only cover recordings you have the rights to.

Each node runs yue2.cpp in its own process: ComfyUI's models are unloaded first and all memory comes back when it ends. Cancel works.

Credits: M-A-P / HKUST (YuE2, SheetSage2, MERT), ServeurpersoCom (yue2.cpp), Mothersuperior (realaudio tokenizer and adapter). Weights CC-BY-NC-4.0, non-commercial. Nodes MIT.