Repair and restoration

Here's a spoken phrase recorded close to the mic. Somewhere in it, a "p" or a "b" hits the capsule with a puff of air and turns into a low thump. The repair below is already catching those thumps. Before you touch anything, switch Audition to Removed. You'll hear only what the repair is taking away.

Make a guess first: is the removed part pure pop? If you heard a muffled, low version of the voice under the thumps, you heard the most important idea in repair. The pop and the voice share the same low frequencies. Anything that takes one away takes a little of the other.

That's true of every clean-up tool, from a one-knob gate to a model trained on thousands of hours of speech. They pay the cost in different ways, and the most modern ones pay it in a way you can't hear by pressing Removed.

What did the repair take?

Push Amount up and listen on Removed again. The thumps get louder in the removed pile, and so does the voice's low body. Raise Cutoff and the repair reaches higher, into the warmth that makes a close voice sound intimate. Switch back to Processed: the voice is cleaner, and a little thinner, and on a big setting a little further away.

This tool is a careful version of an old trick. It watches the low band for a sudden rise in energy, the signature of a puff of air, and turns that band down for a moment. Leave Transient protection on and it ignores low notes that settle and hold, so a sustained low voice doesn't keep triggering it. But it's still a guess about what a plosive sounds like. A sudden vowel or a deep "oh" can fool it, and the start of a burst can slip through before it reacts.

With this kind of repair, Processed plus Removed adds back to exactly the original take. Nothing is invented. Every sound ends up in one pile or the other, and you can listen to both. That makes it honest: if a breath or a consonant went missing, it's sitting in the Removed pile where you can find it.

Plosive Repair Lab is one small example, built for one flaw. Repair in general covers a whole family:

• Denoise: hiss, hum, fan noise, traffic.
• De-reverb: too much room, a voice that sounds like it was recorded in a bathroom.
• De-click and de-crackle: vinyl ticks, digital clicks, mouth noise.
• De-clip: peaks that hit the ceiling and got flattened.
• Plosive and breath repair: pops, thumps, loud inhales.
• Bandwidth extension: putting back the high end on a phone call, an old tape or a heavily compressed file.

The names in that list are what the tools call themselves. What you're really asking each one is the same musical question: keep the performance, lose the accident.

Two ways to clean a sound

There are two very different machines behind those buttons.

Classic repair decides what to remove. A gate decides when: below a level, turn the sound off. A filter decides where: below 80 Hz, turn it down. Spectral subtraction, the workhorse of older noise reduction, decides both. You play it a stretch of pure noise, the gap before the singer comes in, and it learns that noise's fingerprint across frequencies (often called a noise print). Then, moment by moment and band by band, it turns down whatever matches the fingerprint. Push it too hard and you hear its signature flaw: a watery, twinkling warble in the background, nicknamed musical noise, where little scraps of noise survive in random bands.

Learned repair decides what to keep, and fills in the rest. A denoising model is trained on pairs: a clean recording, and the same recording with noise or reverb added on purpose. Over many examples it learns to answer one question: "given the messy version, what did the clean one sound like?" Some models answer with a mask, the same moment-by-moment, band-by-band keep-or-drop grid a stem separator uses. Others write the clean sound out directly, and the most ambitious ones generate detail that wasn't audible in the input at all.

In machine-learning papers you'll see this called speech enhancement, source separation or audio restoration. To a musician, it's one job: give me the take without the accident. But the output is an estimate. It's the model's best guess at a recording that never existed.

Same idea, different place: a denoiser that keeps "the voice" and drops "everything else" is a two-stem separator. Turn "everything else" into drums, bass and guitars and you have stem separation, which shows how that mask works, with the same habits and the same kinds of mistakes.

Why not just use a gate? A gate decides only when, so it silences the gaps and leaves the noise under every note, often making it pulse on and off with the phrases; Gates, sidechain and ducking shows that happening. To take noise out from under a note, a tool has to decide at every moment and every frequency, which is where the real trade-offs start.

What goes out with the noise?

Noise rarely sits alone. It shares frequencies and moments with the parts of a recording that make it feel human. Before any repair, listen for these, because they're usually the first to go:

• Breaths. To a denoiser, an inhale looks a lot like noise: broadband, unpitched, quiet. Strip them all and a vocal can sound eerily continuous, like a voice that never needs air.
• Consonants. "S", "f", "t" and "k" are short bursts of noise by nature. Aggressive repair softens them and the words get harder to follow.
• The room. The sound of the space is part of the take. De-reverb can make a voice drier and closer, but too much and it sounds cut out with scissors, as if it were pasted on top of the track.
• Tails. The last ring of a guitar chord and the decay of a piano note fade down into the noise floor. A repair can't tell where the note ends and the hiss begins, so it often ends the note early.
• Character. The hum of an old amp, the hiss of a cassette, the crackle of a record. Sometimes the "flaw" is the reason the recording feels like a memory. Plenty of lo-fi producers add vinyl crackle on purpose.

Our take: set a repair while listening to the quietest, most delicate moment in the take, not the noisiest one. The noisy part tells you how much repair you'd like. The delicate part tells you how much you can afford. And if you're recording samples rather than fixing a take, the cheaper fix is upstream: Preparing samples explains why consistent room noise beats noise that comes and goes.

When the model fills in the gaps

Learned repair can do things subtraction never could, and it's worth seeing why. Here's what an old recording or a phone call often looks like: plenty of sound up to about 4 kHz, then almost nothing above it.

There's nothing above that line to recover. No EQ can boost what isn't there. A bandwidth extension model draws the top end back in from what it learned about voices: when the low part looks like this, the high part usually looks like that. Learned de-clipping does the same for flattened peaks, sketching in the tops of the waveform the recorder cut off.

That's genuinely useful. It can make a voice memo demo sit next to a studio recording. But listen to what it means. The "s" you hear after the repair is the model's idea of an "s", not the one the singer made. Usually that's fine. Occasionally it's wrong in a way that sounds right, and you won't find it by listening for noise, because there isn't any. Listen instead for a voice that's slightly too smooth, consonants that all sound alike, or a breath that turned into a whisper of something else.

And remember the two piles from the first lab. You can still subtract the repaired version from the original to hear a difference, but that difference now contains two things mixed together: noise the model removed, and detail it added, flipped upside down. It's still worth a listen, as long as you don't read it as "only the noise".

Where repair breaks

Some flaws are much harder than others, and knowing which saves you hours:

• Noise that sounds like the music. A fan's whoosh is easy; a crowd's chatter under a live vocal is hard, because both are voices. The model has to guess which voice is "the" voice.
• Reverb on reverb. Room sound is made of delayed copies of the voice itself. De-reverb has to tell the singer apart from their own echo, and on long, dense reverbs it tends to leave a hollow, phasey sound.
• Instruments it never heard. Most learned denoisers were trained mainly on speech. On a bowed cello or a distorted guitar, they may decide some of the instrument is noise. That's the same lesson as what an audio model learns: a model's mistakes follow what it was trained on.
• Stacking repairs. Denoise, then de-reverb, then de-click, each at a medium setting, can add up to more damage than one strong repair. Try each one alone first.

One more practical point: a learned repair tool might run on your own computer, or it might upload your recording to a server. Check which before you send it an unreleased song.

Make it yours

Find a take of your own with one flaw you've been living with: a pop, a hum, a room you don't love. Name the job first, in one line: "I want the second verse vocal to stop thumping on 'baby'", not "make it cleaner".

If it's a pop or a low thump, put the take through Plosive Repair Lab. Start with Amount low, raise it until the thump stops drawing your attention in the mix, then switch Audition to Removed and listen once through. If you can hear the singer's chest voice clearly in the removed pile, back Amount or Cutoff down a step. Check the quietest word in the take before you decide.

If it's noise or room, try whatever cleaner you use and listen to the difference between before and after, keeping in mind that some of that difference may be invented.

Save the setting with the singer's name and the mic in the title, because a plosive repair that suits one voice and one mic rarely suits another. Add one line about what it cost: "loses a bit of chest on the low notes". Next session, you'll start from a setting that already knows this voice.

Quick check

1. You denoise a vocal and the hiss is gone, but the singer now sounds like they never breathe. What happened, and what would you try? The breaths looked like noise to the repair: quiet, broadband, unpitched. Lower the strength, or repair only the gaps where there's no singing, and listen to a phrase with a big inhale before you commit.
2. With a classic repair, you can listen to the removed part. Why is that less straightforward with a learned denoiser? A classic repair splits the take into kept and removed, which add back to the original. A learned repair writes a new estimate, so the difference between before and after mixes removed noise with detail the model added. It's still worth hearing, but it isn't only the noise.
3. A phone-call recording has nothing above 4 kHz. Can an EQ boost bring back the singer's real high end? No. There's nothing there to boost; you'd only raise the faint noise. A bandwidth-extension model can draw in a plausible top end, but it's the model's estimate of what voices usually have, not what this singer sounded like.

Next steps

• Go deeper: How models see audio. A denoiser decides tile by tile in a picture of the sound. Why do clicks and consonants suffer most in that picture?
• Use it: Preparing samples. Clean starts, tidy ends and the small fixes that make a recording ready to play.
• Sideways: Spectral effects. The same picture of sound that repair works on, used to freeze, smear and warp it on purpose.