AudioFit is a toolkit for deriving synthesizers from real audio. It doesn't generate samples. It measures an editable instrument against recordings, then lets an agent change the DSP and tune the parameters. What you get is still a synth you can play and edit.
Our first target was a hard one: an acoustic piano.
Getting a piano sound is easy. You could play back samples or reach for the classic FM electric piano. We wanted a physical model, with strings, a felt hammer, a bridge and a soundboard that are all DSP you can reshape. That's Tricorda. We fitted it to 24 notes from the Salamander Grand Piano (CC BY 3.0).
It doesn't sound like the sample yet. It has far too many parameters to make a good interface. But it taught us a lot.
The loop
The agent ran close to 30 studies in two days, each one the same cycle: listen to where the model differs from the recording, hypothesize a fix, build it into the real instrument, tune it with an optimizer, and check it on notes it never trained on.
What it found
- Waveguide vs. modal strings: nearly a tie. The algorithm wasn't the bottleneck.
- Eight registers, eight instruments: giving each part of the keyboard its own decay and touch response was the biggest win.
- Bridge coupling: better in training, worse on new notes. Rejected.
- No soundboard: worse everywhere. It stays.
- A different hammer per register: better score, but soft treble notes nearly vanished and dense chords broke our headroom limit. Rejected.
That last one is the lesson. A better score isn't a better instrument.
The point is the harness
An agent loop, analysis tools that show where a sound is wrong, an engine where new DSP ideas run in the real instrument, and a toolkit to tie it all together. With all four in place, every new modeling idea gets built, tuned and tested the same way, and nothing starts from scratch.
What the agent can't do yet is choose the knobs a musician actually wants. That's the next problem.
Coming soon
We're still experimenting, and we hope to release the AudioFit toolkit soon, so you can build synthesizers from your own sounds.
