BPM Detection: 42% to 85% With TempoCNN

The key & BPM finderon this site started life with a single tempo detector that got the exact BPM right on 42% of the GiantSteps tempo set. It now gets 85%. This is the write-up of how it got there — three distinct stages, one dead end, and the part that's still broken. Every number here comes from the same evaluation script, and the script is in the repo, so you can reproduce all of it.

How "accurate" is measured, before any numbers

Tempo accuracy figures are easy to inflate, so the metric comes first. The test set is the GiantSteps tempo dataset — EDM tracks with human-verified BPM labels, and deliberately the hard case, because electronic music is where detectors fail most.

A prediction counts as exactif it's within 3% of the labelled tempo. It counts as octaveif it's within 3% of the label multiplied by a metrical ratio — 2, ½, 1.5, ⅔, 3, ⅓, 4⁄3, or ¾. That's the standard MIREX split: Accuracy 1 is exact, Accuracy 2 is exact-or-octave.

Everything below reports exact. The lenient number is much higher and much less useful: a detector that reads 87 for a 174 BPM drum & bass track is "octave-correct" and, to a DJ, simply wrong. If you only ever see a tempo detector quote one accuracy figure, it's worth asking which one.

Stage 1 — one detector: 42%

The first version used Essentia's RhythmExtractor2013 with the degaramethod. It's fast, it's the default everyone reaches for, and it has one property worth knowing up front: the confidence it returns is always zero. degaradoesn't produce a real confidence, so anything built on it has to invent its own.

More importantly, its errors weren't random. They were almost all the same error: locking onto the half-time pulse of fast genres. Drum & bass at 174 reads as roughly 87. Fast techno folds down the same way. The detector was hearing a real, strong periodicity — it was just the wrong metrical level. That diagnosis shaped everything that came next, because a systematic error is something you can correct for, where random error isn't.

Stage 2 — three detectors and a tuned window: 65%

The next version ran three detectors and made them vote: Essentia degara (most trusted), librosa's tempo estimator on the percussive onset envelope after harmonic/percussive separation (second), and Essentia's Percival estimator (third). The reconciliation isn't a plain average — averaging 87 and 174 gives you 130, which is worse than either. Instead:

• Anchor on the most trusted detector, then find every other reading that is metrically linked to it — within 4% of the anchor times one of the ratios above.
• If the two less-trusted detectors agree with each other and both disagree with the anchor, they outvote it. One confident wrong detector shouldn't win against two that independently agree.
• If the chosen tempo sits outside a preferred window, fold it by a metrical ratio until it lands inside — that's the half-time correction, applied explicitly.

The window itself was not guessed. I swept the lower bound across the GiantSteps set and the scores held flat from about 92 BPM upward, so 95–185 was chosen as a bound that isn't knife-edge. It also happens to match how DJs and Beatport label those tracks — a 174 drum & bass record is catalogued as 174, not 87.

Confidence is derived from agreement rather than borrowed from any one detector: three in agreement scores 95, two scores 88, a folded or range-corrected answer scores 62. That's honest in a way degara's constant zero wasn't.

This got to 65%, and then it hit a ceiling that's worth naming. A voting scheme can only choose among answers the detectors actually produced. Replaying the stored votes with an oracle that picks the best available answer per track shows the hard limit of this approach — and the DSP detectors simply weren't producing the right answer often enough for any voting rule to reach it.

What didn't work: madmom

The obvious next step was madmom, whose RNN beat tracker has been the strong baseline in the tempo literature for years. It didn't make it in. madmom doesn't build against NumPy 2 or current Python, and pinning an old NumPy across a FastAPI service that also runs librosa, Essentia and a transcription stack wasn't a trade worth making for one dependency. It was ruled out on engineering grounds, not accuracy — if you can run it, it's good.

Stage 3 — a pretrained model: 85%

What worked was Schreiber and Müller's TempoCNN, available pretrained through essentia-tensorflow as deeptemp-k16-3. It takes audio at 11,025 Hz and returns a global tempo directly, no onset detection or periodicity search involved. Exact accuracy on the same set went from 65% to 85%.

One design decision is worth stating because it's counter-intuitive: when TempoCNN is present, its answer is used outright. It is notblended into the vote with the DSP detectors. I tried that — replaying stored votes under every combination rule — and every policy that mixed it with the DSP readings scored lower than trusting it alone. The consensus machinery from Stage 2 is still there, but only as the fallback when the model isn't.

The cost is real. essentia-tensorflow adds roughly 500 MB to the container image over plain essentia, and the model is loaded lazily on first use so it doesn't slow startup. And there's a trap: plain essentiasilently lacks the TempoCNN algorithm. The import fails, the engine catches it as non-fatal and falls back to the DSP path, and nothing errors. You can believe you've shipped TempoCNN and be running the 65% path. That fallback is deliberate — it's better than a 500 on every request — but it means the accuracy number has to be checked on the deployed image, not just in development.

Results

Same dataset, same 3% tolerance, same script for every row. The jump from 65% to 85% is the model; the jump from 42% to 65% is entirely the voting and the window, with no change in the underlying detectors.

The part that's still broken: key detection

Tempo is the success story. Key is not, and it would be dishonest to bury that. Key detection uses Essentia's KeyExtractor on the harmonic component of the signal with the bgate key profile. That choice was measured — six profiles across three input signals on 40 GiantSteps tracks, where bgate on the harmonic component scored 20 of 40 and the more commonly recommended edma profile scored 16 of 40 on the same input. So the best configuration is right about half the time.

The reason there's no Stage 3 for key is that there is no pretrained key model in Essentia to reach for the way there was for tempo. The current mitigation is a second opinion from a librosa profile match, with confidence penalised when the two disagree — which improves calibration but not the underlying accuracy.

The lenient figure, where a relative major/minor is accepted as correct, is considerably higher. But it's the wrong figure for this use case: on the Camelot wheel, A minor and C major are different positions, and a DJ mixing harmonically will hear the difference. So the honest number is about 50%, and that is the open problem. If you know of a pretrained key-estimation model that runs without adding another gigabyte of dependencies, I would genuinely like to hear about it.

Reproducing it

Two scripts in the backend repo do all of this. eval_keybpm.py runs the engine over a CSV of file paths and ground-truth labels and prints per-track hits plus the exact and exact-or-octave summaries. tune_bpm_policy.py then replays the stored per-detector votes under different selection rules — single detector, priority order, consensus, window folding, oracle — without touching audio again, so you can compare policies in seconds and pick one from data before changing the engine. That replay is how the "don't blend TempoCNN" decision was made.

For the user-facing version of why a BPM sometimes reads as half or double, and why confidence varies between tracks, see how key and BPM detection works. To run the detector on your own audio, the Key & BPM Finder is free and needs no account.