Jam with a model

Here are two bars of melody and two bars that answer them. Press play. Pretend a model wrote the answer.

Look at the piano roll. The answer keeps the call's rhythm exactly, four quick notes and two long ones, and flips its shape: your phrase climbed, the answer comes back down to A. Nothing in it is new, and yet it sounds like a reply. That's what a model does well: it carries on from what you gave it. Musicians have always traded phrases like this, and this project builds that conversation with a model as the other player.

You'll set the ground rules, play a call, let a model answer in notes, take turns and steer it, then arrange the best exchanges into a short piece you can keep. Before you start: when a model answers your phrase, do you think it's listening to what you played the way a bandmate would?

Stage 1: Set the ground rules

Before two musicians jam, they agree on a few things, often without saying a word: the key, the tempo, how long each person plays. A model needs those rules spelled out, and they matter even more, because it has no other way to know what you want.

Choose four things and write them down:

• Key and scale. A minor or pentatonic scale is forgiving; every note sounds at home. Start there.
• Tempo. Something relaxed, 80 to 100 BPM, so you have time to listen.
• Turn length. Two or four bars each. Short turns keep the model close to your idea.
• Register. Who plays high and who plays low. If you play in the middle, ask the answer to sit an octave above or below, so the two voices don't tangle.

If you'd like chords to jam over, let a model propose them: Chord Seq writes a progression one chord at a time and hands it over as MIDI. Generating notes shows how its Temperature and Seed controls move it between familiar and surprising, and those same two controls steer every answer in this project. Pick a progression you'd enjoy playing over for ten minutes, and loop it.

In machine-learning terms, your ground rules are constraints: limits on what the model is allowed to choose. A scale constraint removes every note outside the key from the model's options.

Checkpoint: a key, a tempo, a turn length, a register plan and, if you want, a looping progression.

Stage 2: Play a call worth answering

A good call is short, clear and has a shape you can recognise: a rhythm that repeats, a leap, a note held at the end. Think of the phrases that get answered in a blues: a line that rises and leaves a gap. The gap is the invitation.

Play or write two bars and record them as MIDI. Leave the end open; a phrase that lands firmly on the home note sounds finished, and the model will have less to say.

To the model, your call is the prompt: the material it continues from. It doesn't hear your intention, only the notes, their timing and their loudness. So make the musical idea audible in the notes themselves. If the rhythm is the point, play the rhythm clearly. Melody and motifs has more on what makes a short phrase memorable enough to answer.

Checkpoint: a two-bar call, saved as MIDI, with a gap at the end.

Stage 3: Let the model answer, notes first

Give your call to a model that generates notes: a tool that takes a MIDI phrase and continues it. Ask for an answer the same length as your call. Ask for several, with different seeds, so you have something to choose from.

A note model predicts one event at a time from the events before it, so your call is all it has to go on. The model doesn't listen, it continues. It answers your call the way a sentence continues from its first words, not the way a friend replies after thinking about what you meant. Generating notes shows that step-by-step prediction in detail.

Ask for notes first, not audio: they're an answer you can edit. We adapted one open note model, MIDI-GPT, for browser experiments, and our hackathon story shows how its phrases became a playable loop instrument. An instrument that answers you live is something we're still building.

You can also ask an audio model to answer, and the result can be wonderful: a whole band or a texture playing around your phrase. Our Accompanist module takes a phrase you play and renders a short audio take in a style you choose, using Google's open Magenta RealTime 2 model; it isn't generating on the public site yet. An audio answer is a recording, so you'll arrange it by cutting and layering rather than editing notes. Audio generation explains the difference.

Checkpoint: three to five answers to your call, as MIDI, each with the seed and settings that made it.

Stage 4: Take turns and steer

Listen to the answers right after your call, one at a time. Most will be fine and forgettable. One or two will make you want to play again, and that's the one to answer.

Now play your next call in response to what the model gave you. Echo its last idea, contradict it, take its rhythm and change the notes. Then let the model answer that. After a few turns you're having a real exchange, and it goes somewhere neither of you would have gone alone.

When the answers drift, steer:

• Too safe, or copying you back? Raise temperature a little, or give a call with more variety in it.
• Wandering out of key or too busy? Lower temperature, tighten the scale constraint, or shorten the turn.
• Losing the thread after many turns? Start the next turn from your original call, not the whole conversation. Long contexts drift.
• Almost right? Fix the one wrong note yourself. That's not cheating. That's the musician's half of the job.

Keep a log of what you kept and why. It's the only way to remember which settings worked by the end of the session.

Checkpoint: at least six turns, with the answers worth keeping marked.

Stage 5: Live or offline?

Playing with a person, you hear their answer the instant they play it. With a model, there's always a wait: it has to receive your call, generate, and send the answer back. Playing with a model live explains where that wait comes from and how to make it land on the beat. For planning a session, what matters is the kind of model.

Note models are small and their answers are short lists of events, so a small one can often finish two bars before your phrase has ended. That's why a call-and-response jam with notes can feel close to live: you play, and the answer follows on the next bar. Audio models generate thousands of sound samples for every second, and the large ones take much longer than real time. With those, jam offline: play your call, let it render, listen, choose.

Either way, the model works in turns. It answers after your phrase ends, never while you're playing it.

Checkpoint: you know which of your tools answers on the next bar and which one you wait for, and you've planned your session around it.

Stage 6: Arrange the conversation

A jam is raw material. Now shape it. Lay out your kept turns in order and listen from the start. Then cut, as you would a recorded jam session with a friend:

• Pick the four or six best exchanges and drop the rest.
• Give the answers a different instrument from your calls, so the listener hears two voices. A soft electric piano for you, a plucked synth or a vibraphone for the model.
• Make one section where the turns get shorter, two bars then one, so the exchange speeds up. Then a final section where both voices play together, your call over its best answer.
• Fix anything that bothers you, by hand. Every note is yours to change.

Play the whole thing through. If it sounds like a conversation that goes somewhere, it's done. If it's only call, answer, call, answer, change one section so the pattern breaks. Form and arrangement has more on making sections feel different.

Checkpoint: an arranged piece, 16 to 32 bars, with two clear voices.

Stage 7: Save it with its settings

Bounce the piece and keep the MIDI of every kept turn beside it, plus your turn log. Note which model and version you used, and its settings, so you can return to a good moment and continue the conversation from there. Check the tool's terms before you release something built on its answers.

Checkpoint: a bounce, the MIDI, and the log, in one folder. Next session, you can pick up the conversation where you left it.

Same idea, different place: a looper is the simplest jam partner there is. It answers your call with exactly what you played, every time, and you improvise over it. Live performance and looping shows how to build a whole performance from that.

Quick check

1. Ten turns in, the answers have drifted far from your first idea and you don't like where they've gone. What do you change? Start the next turn from your original call, or a call you kept, rather than from the whole conversation. A model continues from everything it's given, so a long, wandering history pulls its answers along with it.
2. Your friend says the model "didn't listen" because its answer ignored your big leap. Fair? In a sense, yes: it doesn't listen at all. It predicts what comes next from the notes it was given, and a single leap may be less likely to it than a step. Make the leap part of a repeated rhythm, or keep the answer that uses it and discard the others.
3. You want a model to answer on the very next bar during a live set. Which will work better, a note model or a large audio model? A note model. Its answers are short lists of events and can be ready within a bar. A large audio model usually takes much longer than real time to render, so it suits taking turns offline.

Next steps

• Sideways: An instrument trained on your sound. The model's answers play on a synth. What if the instrument playing them were a model of your own voice or instrument?
• Use it: Form and arrangement. Your exchange is 32 bars. Turn calls and answers into verses, a build and a return.