MT3
Transcribe multiple instruments from audio into note-event sequences.
Transcription & rhythmOpen-source code
- Start here
- Try material with several known instruments. Judge instrument assignment and note timing separately; a dense MIDI file is not proof of accuracy.
- Training data
- MAESTRO, GuitarSet, MusicNet, URMP, Slakh and its Cerberus4 derivative in the paper's multi-dataset setup.
- Training rights
- Training sources have different licenses and access conditions; a research mixture is not one uniformly open dataset.
- License & output terms
- Code: Apache 2.0; verify checkpoint and training-source terms for reuse.
- Limitations
- Instrument leakage, missed notes and alignment errors; research infrastructure and instrument vocabulary constrain deployment.
- Technique
- Sequence-to-sequence transformer
- Runtime
- Local research inference, with a heavier JAX/T5X setup than small transcription models.
- Parameters
- Not itemized for this release/family. Parameter sizes shown elsewhere may exclude encoders, decoders or other runtime components.
Sources reviewed . Training and rights notes summarize provider disclosures. A model license does not clear every source recording or generated output. Parameter counts refer to the named variant or component, not total runtime memory.