SI-SDR, and the alignment trap that breaks it

Every separation score we publish is scale-invariant SDR, and two things decide what that number looks like: what "scale-invariant" actually corrects for, and whether the comparison was aligned before scoring. Here is how each one works, and why getting the second one wrong can make a working separator look broken.

What SI-SDR measures

Scale-invariant signal-to-distortion ratio compares a separated stem against the true one and reports, in decibels, how much of what you are hearing is signal rather than error. The scale-invariant part matters: a stem that is perfectly correct but quieter than the reference is still perfectly correct, and the metric should not punish it for its gain. So before scoring, the reference is scaled by whatever single factor best matches the estimate, and only the leftover difference counts as error. Double the estimate and that factor doubles with it, the error doubles with it, and the ratio — the score — does not move.

The trap that makes a good separator look broken

Separated stems come back 219 samples behind the input. That is about five milliseconds — inaudible, and fatal to a naive score. Comparing two nearly identical waveforms that are slightly out of step measures mostly the misalignment.

Scoring without correcting for it reads roughly -13 dB instead of the true score. Not slightly wrong: a working separator reported as producing more error than signal.

Before scoring runs, we find the offset and remove it — best_lag(est, ref) searches for the alignment, align(x, lag) applies it, and only then does si_sdr(est, ref) run. All three are in services/quality.py in our API repository.

Why this matters when comparing services

A separation benchmark that does not say how it aligned, what corpus it used, or whether it reports a mean or a median is not telling you enough to compare against. That applies to ours too, which is why we publish all three.