How Music Recognition Works, and Why It Fails in DJ Mixes

The method behind the best known recognition app is public. In 2003 Avery Wang of Shazam Entertainment presented An Industrial-Strength Audio Search Algorithm at ISMIR, the music information retrieval conference. The paper says Shazam started in 2000, that the first service let a caller sample up to 15 seconds of music over a mobile phone and get the title back by text message, and that the deployed database then held over 1.8 million tracks.

The core idea is stated in one line: both the database tracks and the unknown sample are "subjected to the same analysis", and the fingerprints from the sample are matched against those of the database. Nothing in that process understands melody, lyrics or genre. It compares one physical recording with another.

So the database must already contain the exact recording. The paper's index is a sorted list of records, each holding a hash, a time offset and a track ID, built by analyzing every track in advance. Wang writes that the algorithm was designed to recognize sound files already present in the database and "is not expected to generalize to live recordings". It is also "very sensitive to which particular version of a track has been sampled": given several performances of one song, it picks the right one even when a human ear cannot tell them apart. A version that was never delivered and indexed does not match at all, which explains most of what goes wrong in a DJ set.

Catalogs differ between services and change over time, so a track missing from one app can be present in another. That is the practical reason to try a second app, covered in what to do when Shazam can't identify a song in a mix.

The first step turns audio into a spectrogram: a picture of how much energy sits at each frequency at each moment. Wang's team tested several features that might survive a phone microphone, a voice codec and a noisy room, and settled on spectrogram peaks. In the paper's words, a time-frequency point is a candidate peak if it has more energy than all its neighbors in a region around it.

Peaks are then thinned so the whole strip of audio gets reasonably even coverage, and the loudest peaks in each area are kept because they are the most likely to survive distortion. After that the amplitude is thrown away. What remains is a sparse scatter of dots, each one only a time and a frequency. Wang calls these constellation maps because the plots resemble a star field.

Two properties follow, both from the paper:

• Noise adds dots; it rarely moves them. A peak that is louder than its surroundings usually stays a peak when pub chatter is added on top, so the original pattern is still present among the extra points.
• EQ mostly leaves the dots in place. A peak in the spectrum is generally still a peak at the same coordinates after filtering, so bass or treble changes do little harm.

Wang describes matching as sliding a transparent sheet holding the sample's dots over a long strip chart holding the song's dots. At the right position, a significant number of points coincide.

A single dot carries little information, so searching by dots alone is slow. The paper's answer is combinatorial hashing. Each peak is treated as an anchor point with a target zone just after it, and the anchor is paired with the points inside that zone. Every pair yields two frequencies and the time difference between them, packed into a 32-bit number: the hash. It is stored with the track ID and the time at which the anchor occurs in the track.

With ten pairs per anchor, Wang estimates roughly ten times the storage in exchange for roughly 10,000 times the search speed, at a small cost in detection probability.

Matching then works in these steps, as the paper describes them:

1. The sample is fingerprinted the same way, producing hashes with their times inside the sample.
2. Every sample hash is looked up in the database, and each hit is filed under the track it came from.
3. For each candidate track, the system subtracts the sample time from the database time for every hit.
4. A true match produces many hits with the same difference, because the sample is one continuous piece of the track. A wrong track produces scattered chance hits.
5. The score is the number of hits in the biggest cluster, and a threshold chosen for an acceptable false-positive rate decides whether to report the track.

This is why recognition survives rough conditions. The paper reports that a 15 second sample can be identified when only about 1 to 2 percent of its hashes survive. In its test against 10,000 tracks, with noise recorded in a pub, the recognition rate fell to 50 percent when the noise was about 9 dB louder than the music for 15 second samples, 6 dB for 10 second samples and 3 dB for 5 second samples.

A DJ mix is a harder input than a radio broadcast. Researchers at Johannes Kepler University Linz tested this in Landmark-Based Audio Fingerprinting for DJ Mix Monitoring, presented at ISMIR in 2016. They list the modifications a DJ introduces: pitch shifting, tempo changes, cross-fading and beat-matching, plus effects, often in combination. On a set of real club recordings, the three fingerprinting systems they tested assigned between 25 and 74 percent of the detectable seconds to the correct track, and the authors conclude that identification on DJ mixes is a challenging problem.

Wang's paper reports a property it calls transparency: several tracks mixed together can each be identified. So an overlap is not automatically fatal. But during a long blend each track is noise to the other, at close to equal level, and the paper's own figures show short samples failing first as noise rises. A sample taken mid-transition may return either track or nothing, and you may not know which of the two you were hearing.

Each hash is built from two frequencies and a time gap, and the matching step assumes the sample and the database copy run at the same speed; the paper sets the slope of the match line to 1.0. Its robustness tests cover noise and phone compression, not speed change. DJs change speed constantly. AlphaTheta's help center lists selectable tempo ranges of plus or minus 6, 10, 16 or 100 percent on one of its controllers. As Digital DJ Tips' keymixing guide puts it, speeding up or slowing down a track changes its pitch, and with it the key. That moves every peak up or down in frequency and squeezes or stretches every time gap, so the hashes no longer equal the stored ones.

Key lock, which Pioneer DJ and AlphaTheta call Master Tempo, holds the pitch while the tempo moves. The time gaps still change, and the audio is reprocessed: AlphaTheta's own note on Master Tempo says key control processing is used to keep the key constant, "so the sound changes". The 2016 paper describes systems built to tolerate this, including one evaluated on queries changed by up to 30 percent in pitch or time scale, but the basic pair-of-peaks design was not made for it. How far any consumer app has moved beyond the 2003 design is not something its maker documents in detail, so treat a small tempo change as a risk, not a guaranteed failure.

The remaining failures are database problems, not signal problems.

• Edits and re-edits. A DJ's private edit rearranges or extends a release. Stretches that are untouched may still match the original; reworked stretches will not. Expect the app to name the source track, not the edit.
• Bootlegs and mashups. An unofficial remix is a new recording that was never delivered to a catalog. A mashup can return the instrumental, the vocal track or nothing; Wang notes that reported false positives often turned out to be real cases of sampling, where one record contains a piece of another.
• Unreleased music. Promos, dubplates and a producer's own unfinished tracks are in no database. Mixmag's 2017 feature on unreleased tracks says it plainly: "Shazam won't work in these instances". DJ Mag's 2025 feature on the track ID debate traces the habit back to DJs blacking out white labels with marker pens, and quotes a DJ on the pull of playing a tune nobody else has.
• Live recordings. A phone video from the dance floor adds crowd, room echo and a clipping microphone. Fingerprinting tolerates a lot of noise, but each of those removes more hashes, on top of the DJ's tempo change. A live band or a live PA is a new performance and, per the paper, is not expected to match at all.

For what "ID" means in a tracklist and what to do about tracks like these, see what a track ID is.

Singing at your phone is a different problem, because your voice shares no spectrogram peaks with the record. SoundHound offers it; its App Store listing says you can press the button and "sing or hum the tune". Google added hum to search in October 2020. Its launch post asks for 10 to 15 seconds of humming, whistling or singing and explains that its models turn the audio into "a number-based sequence representing the song's melody", stripping away instruments and the quality of the voice.

Google Research described the machine learning behind hum to search in November 2020. A neural network is trained on pairs of hummed and recorded audio so that two pieces of audio with the same melody produce embeddings that sit close together, even with different accompaniment and different voices. At the time of that post the system covered over half a million songs.

• Fingerprinting needs the recording itself playing, and answers which exact version it is.
• Humming search needs only a tune you can carry, and answers which song it probably is. It offers likely matches, not a precise version, and it needs a melody. A rolling techno track with no tune to hum gives it very little to work with.

Put together, the method tells you how to use it. Sample where one track plays alone, well after a transition or in a breakdown. Feed the app clean audio from the source, not a phone speaker across a loud room. Give it the longest listen it allows. If it fails, try a second app with a different catalog, and hum the hook if there is one. The how-to linked above has the full routine.

When apps run out, people take over: tracklist sites such as 1001Tracklists and MixesDB, the comments under the set, and the DJ. Our guide to identifying a track in a DJ mix walks through that order.

MixTracklist fits between those two. You paste a SoundCloud set at /scan and get a tracklist for the whole set with a timestamp on every track and a confidence score from 0 to 100 on a colored badge: lime is high (75 and up), amber is in between, and gray is low and worth checking by ear. A "Double match" label means two independent matches agreed. Sets already identified are free to read in the library. It accepts SoundCloud links only, not YouTube, Mixcloud or uploaded files, and it cannot name music that has never been released or cataloged; those rows stay unidentified.

Our view: recognition is a lookup and not an ear, so the most useful skill is knowing when the lookup cannot work and asking a person instead.