Chord and key recognition

Play this. Your right hand would hold the same three notes the whole time: C, E and G. Only the bass moves, from C down to A.

Watch the name under the piano roll. The first chord is C. The second is A, C, E and G, which most charts would write as Am7, and it sounds it: softer, a little wistful, less settled. Yet you never moved your right hand. If you'd been playing C over a bass player's A, a chord-recognition app listening to the band would write "Am7", and so would a good jazz musician.

Before reading on: when a recognizer hears that second bar, is "Am7" the right answer and "C" the wrong one?

What a recording looks like to a chord recognizer

To name a chord from audio, a tool first has to throw most of the audio away. The usual first step is a chroma view, also called a pitch-class profile: for each short slice of time, how much energy falls on each of the twelve note names, regardless of octave. Every C on the piano counts as C. Every E counts as E. A chord becomes twelve numbers.

That's a great simplification for harmony. It keeps which notes are sounding and loses how they're spaced, which instrument plays them and which one is on top. It's the same keep-or-drop trade you meet in Music as data, applied to sound instead of notes.

It also catches more than you played. A real piano note is a stack of harmonics, and some of them land on other note names.

Look at B and D. Nobody played them, yet they're there: the third harmonic of E is a B, and the third harmonic of G is a D. To a template that matches only C, E and G, they're noise. To one that knows Cmaj7 or Em, a bright, loud piano might start to look like those chords instead. Modern recognizers are trained models rather than simple templates, and they learn to discount a lot of that leakage, but they still start from the same kind of picture.

A chord label is a vocabulary choice

In machine-learning terms, chord recognition is sequence labelling: every slice of time, or every beat, gets one label from a fixed list. Key estimation is a cousin: one label, or a few, for a whole stretch of music. Both depend on that list, the vocabulary, more than people expect.

Research benchmarks, such as the yearly MIREX chord-estimation task, score chord recognizers at several vocabulary sizes. The smallest has just 24 chords, a major and a minor triad on each note, plus "no chord". Larger ones add sevenths, inversions, suspended chords and more. Maru's Chord Seq, which writes progressions rather than recognizing them, uses a vocabulary of 1,033 chord shapes. None of these is the true list. Each one decides in advance which answers are possible.

You played	Major/minor vocabulary	Sevenths vocabulary
Cmaj7	C	Cmaj7
C over an A bass	Am or C	Am7
C7 in a blues	C	C7
Csus4	C or F, whichever fits better	C or F, whichever fits better

So when a tool names your Cmaj7 as "C", it hasn't misheard. It answered in the only words it had. When it says "Em" for the same chord, it noticed the E, G and B and had no seventh chord to offer instead.

Why there are often two answers

Some chords are genuinely two chords. That's not a failure of the tool or of your ear. It's how harmony is built.

Same notes, different root. C6 and Am7 have exactly the same four notes. Which name fits depends on the bass, the chord before and after, and what the melody leans on. The readout in the first lab chose by looking at the lowest note, which is what a bassist would do too.

Inversions. C with E in the bass is still C major, but it sounds lighter and more in motion, and in the middle of a line it can work like a passing chord. A recognizer that only reports roots will call it C and lose that feel; one that reports bass notes will write C/E.

Missing thirds. A power chord, A and E with no third, is neither major nor minor until the music around it decides. A recognizer with no power-chord label has to pick A or Am anyway, and it often picks from the vocal or a stray harmonic. Ask it twice on two takes and it may answer differently. Both answers are guesses at something that isn't there.

Relative keys. C major and A minor use the same seven notes, and which one a song is "in" depends on which note feels like home. Key estimators weight the notes using profiles measured from listeners (the best known are Carol Krumhansl and Edward Kessler's, from 1982). Those profiles help distinguish the two keys when the music leans clearly one way. The result is a ranking with a fit score rather than a certainty. When the music doesn't lean clearly toward either key, you'll see the estimates switch.

Same idea, different place: a tempo tracker that says 140 when you feel 70 has made the same kind of choice, between two levels of the same pulse. See Beat and tempo tracking.

Listen to the bass first

When you're working out chords by ear, start at the bottom. The bass note usually tells you the root, or tells you the chord is an inversion. Then listen for the third above it, major or minor. Then for anything extra on top: a seventh, a sixth, a suspended note.

Recognizers that work well tend to do the same thing in their own way, estimating the bass and the upper harmony separately and then combining them. When a tool and your ear disagree, check the bass first. More often than not, one of you was hearing the bass player and the other was hearing the keyboard.

That split is also why the same notes can feel so different when you play them. If you play harmony from pads or a small keyboard, choosing which note goes in the bass changes the name and the feeling of every chord you play, with your right hand doing exactly the same thing.

Key is a summary over time

A chord lasts a beat or a bar. A key is a summary of many bars. To estimate one, a tool collects the pitch classes over a stretch of music and compares the total with a profile for every major and minor key. The answer depends on how long that stretch is.

That key change up a step is a classic way to lift a last chorus, and it's one of the ideas in Progressions and harmonic movement. For a key estimator, it's a problem of window size. A long window gives a steady answer and smears the change. A short window catches the change and also catches every borrowed chord, flipping to a new key for one bar each time.

So when a tool gives a whole song one key, read it as "the key that best fits all of these notes together under this model". DJs use exactly that kind of summary for harmonic mixing, often on a key wheel that gives each key a number and puts compatible keys, like a key and its relative minor, side by side, and it works well when songs stay put. For a song that moves, ask for the key by section.

Music outside the label set

A vocabulary of major and minor chords and 24 keys was built around a particular repertoire: Western tonal music, and the pop, rock and film music that grew from it. Lots of music is organized another way, and a recognizer trained on that vocabulary can only answer in its terms.

• Modal vamps. Miles Davis's "So What" sits on a D Dorian sound for sixteen bars, then moves up a half step. A key estimator has to call that D minor or C major, and neither name says what makes Dorian sound the way it does.
• Blues. The home chord in a blues is often a dominant seventh, and the melody bends between the minor and major third over it. "Major or minor?" doesn't fit the question the music is asking.
• Raga. Raga Bilawal uses the same seven notes as a major scale, so a key estimator would most likely say "C major" over a drone on C, and miss its characteristic phrases and movements, which Scales, modes and tonal centers shows are what make a raga.
• Maqam. Maqam Rast's third sits between minor and major, and a twelve-bin picture has to round it one way or the other, the same blind spot any twelve-note count has.

In each case the tool's answer can still be a useful clue, a rough map to start from. Just don't read it as a description of the music.

Make it yours

Pick a song you know well that has at least one chord you've never been sure of. Work out its chords by ear, bass first: bass note, then third, then anything on top. Use Chords if you want a refresher on how the triads are built.

Then run the same song through a chord-recognition app or your DAW's audio-to-chords feature. Wherever you and the tool disagree, write both readings on your chart, like "C or Am7", and note what supports each: "bass on A", "melody holds C", "the chorus resolves here". Note the key the tool reports, and whether it changes in the song.

You now have a chart with the ambiguity marked rather than hidden. Save it with the song. When you play it with someone else, those marked bars are exactly where you'll want to agree on a reading before the first rehearsal.

Quick check

1. You play C, E and G over a bass player's A. The app says Am7. Did it mishear? No. The full sound has A, C, E and G, which is a fine Am7. Your hand played C; the band played Am7. Which name you write depends on whether the chart is for your part or for the whole band.
2. An app with only major and minor chords labels your Cmaj7 as Em. Why that, of all things? Cmaj7's top three notes, E, G and B, are an E minor triad. With no seventh chords in its vocabulary, the tool picked the triad that explained the most of what it heard.
3. A song gets "A minor" from one tool and "C major" from another. Which is wrong? Possibly neither. They share every note. Look for which note acts as home: where phrases end, what the bass lands on, where the chorus resolves. If both are supported, write both; that's a real musical ambiguity, not an error.

Next steps

• Sideways: Tagging, search and similarity. A key estimate sums up a song in one label. Genre, mood and instrument tags do the same, with the same window and vocabulary choices hiding behind them.
• Go deeper: Ear training. Hearing the bass and the third quickly is a habit you can build, and it's the fastest way to check any tool's answer.