We benchmarked every stem separator we can legally ship

Before shipping a stem separator we had to pick which of our commercially-licensed candidates to default to, and the public comparisons we could find either tested models we are not licensed to ship or reported a single song. So we measured it ourselves.

What we measured, and what we could not

Six models — every entry in our SHIPPABLE registry and nothing else — against 12 tracks from the MUSDB18 test set, scoring median scale-invariant SDR. The corpus is the free 7-second preview tier of the musdb package, and the sweep ran on an arm64 Mac via CoreML rather than the CUDA host we serve from. That affects wall time, not the scores.

Twelve seven-second excerpts is a small corpus. We are reporting it because it is what we ran, not because it settles the question — but the five Inst_HQ variants, which we would expect to score alike, cluster within 0.38 dB of each other on vocals. The model we ship is roughly 5 dB ahead of that entire cluster, a gap over ten times the size of the spread between models we already know are near-identical.

Results

The finding

One model wins, and it wins on vocals. UVR_MDXNET_Main.onnx scores 11.43 dB where every Inst_HQ variant lands between 6.00 and 6.38 — a gap of roughly five decibels, which is not a tuning difference. It gives up between 0.82 and 1.06 dB of instrumental quality to get there (14.15 against a cluster running 14.97 to 15.21), so the total is +4.17 dB ahead of the next best model.

That instrumental cost is not nothing: at 14.15 dB, UVR_MDXNET_Main.onnx is the worst instrumental scorer of the six, giving up as much as 1.06 dB to the best of the Inst_HQ family. If you only ever need an instrumental, one of the other five is the better pick. If you need the vocal, none of that matters — this is the only model worth using.

Why the Inst_HQ family clusters

The five Inst_HQ variants span sums of 21.06 to 21.40 — a spread of 0.34 dB across five supposedly distinct models. That is consistent with them being successive fine-tunes of one architecture rather than materially different approaches, and the 0.34 dB that separates them is small next to the 4.17 dB that separates the winner.

The alignment trap

Separated stems lag the input by 219 samples. Scoring without correcting for that reads roughly -13 dB instead of the true score — enough to make a working separator look broken.

What we changed

Our default had been UVR-MDX-NET-Inst_HQ_3.onnx. On this evidence we changed it to UVR_MDXNET_Main.onnx. Licensing did not enter into it: all six carry the same MIT grant and the same UVR attribution requirement, so this was a quality decision inside a set we had already cleared to ship — not a new licence question.

The full record, including the licence status of every model we considered and rejected, is in docs/model-licensing.md in our API repository.