Generate audio
Generate sounds, loops, kits and variations from a prompt, or turn recordings into stems and MIDI. Maru runs the models, and people using your instrument generate with their own account.
Generative audio gives an instrument new material on demand. A musician describes "a short wooden knock" and gets candidates for a drum pad. A texture instrument asks for an eight-second atmosphere. A producer fills a sampler bank with variations of one sound. You decide how the result becomes part of the instrument; Maru does the generating.
What you can do
- Generate sounds. One-shots, loops and full drum kits from a text prompt, with several variations to choose from.
- Bring your own recordings. Upload a recording to split it into stems, extract a monophonic melody as MIDI, or analyze it.
- Use the result anywhere. Generated takes are ordinary audio: load them into a sampler bank, a pad, a texture player or an export.
How it works
You never host a model. Generation runs on Maru's servers, outside the realtime audio path, so an instrument keeps playing while a new sound is being made. There are no model weights to ship, no GPUs to rent and no inference code to maintain.
A request goes through four steps:
- Ask. A person describes a sound in your interface, in Studio or in a conversation with their assistant.
- Generate. Maru runs the job and reports progress. The person can cancel, and a lost connection does not lose the result.
- Choose. The takes are saved to that person's own library, where they can listen and pick.
- Use. Your instrument loads the chosen take through its normal sample workflow.
Who pays, and whose sounds they are
When someone uses generation inside your instrument, it runs on their own Maru account. They approve spending once, on Maru's own consent page, which names your app and module. After that, each job uses their credits, and their Usage page shows it under your app's name. You do not pay for the people using your instrument, and your interface never holds their password, token or account details.
Generated takes and uploaded recordings stay in the person's own library. They can keep them, reuse them in other projects and bring their own recordings to work with. When you test generation in your own module, it uses your own credits, like anyone else's.
Where you can use it
| Where | What happens | Needs |
|---|---|---|
| Studio | Generate a kit or take, preview its clips and add them to a sampler bank's sources | A beta account admitted to hosted generation |
| Your AI assistant | Ask the hosted connector for one-shots, loops or 16-pad kits, saved to your Generated Sounds | The connector, with generation permission |
| Your instrument's interface | A Generate button that asks the host to generate for the person playing | Enabling sign-in and inference for the module; supported website hosts today |
| Your own app or site | Your server submits jobs and receives audio results | A personal token or a registered creator app |
| Your computer | Save generated WAV files straight into a folder, ready to import | The CLI, signed in with generation permission |
Hosted generation is in preview. It needs an available service, an account admitted to hosted generation and permission to spend credits. Native plug-ins do not offer the in-interface Generate button yet; their ordinary controls keep working without it.
Ideas to start from
- A drum instrument with a New hit button per pad that offers three takes.
- A texture player that generates an atmosphere from a mood word.
- A sampler bank built from generated variations of one recorded sound.
- A practice tool that turns a sung melody into MIDI to play back on a synth.
Technical details
The generation contract has five task types: generate_audio,
separate_audio_stems, transcribe_audio_to_midi, analyze_audio and
transcribe_speech. A shared contract does not mean every model is available.
Generation, stem separation and monophonic MIDI extraction run today; speech
transcription has no selected model and is refused before execution.
Jobs are durable. Each submission carries a request key, so repeating it recovers the same job instead of paying twice. Jobs report progress events and can be cancelled or retried.
| You are building | Start with |
|---|---|
| A Generate button in a module face | maru module app enable, then declare the optional app.generation/v1 host capability and use face.appGeneration |
| An app with its own server | createMaruClient from maruaudio/api with a token that has inference:run |
| A standalone app with sign-in | A registered creator app and the generation SDK's hosted connector |
| Sample preparation | maru audio generate "…" --out ./take.wav, --variations or kit |
A face, web page or plug-in is a public client: it holds only a public app id and never a secret. A face's Generate action saves a library take; loading it into the playing instrument stays in the host's resource loader, outside the realtime callback.
Add generative audio to your app has the full code, consent and billing flow, recovery and keys. See also Build an audio generator and Accompanist for a MIDI-steered audio model integration.