How to Create Dialogue Scenes With AI Video

To create a dialogue scene with AI video, don't try to generate the whole conversation in one clip. Time the dialogue first, break it into short shots of a few seconds each (speaker close-ups, listener reactions, and a wide to establish geography), then generate each shot from an approved start frame and add lip sync only where a face is clearly speaking on screen. That is how live-action editors have cut conversations for a century, and it happens to be exactly what current AI video models are good at.

Dialogue is one of the hardest things to do well in AI filmmaking. The good news is that the solution is mostly craft, not technology.

Key takeaways

• Most AI video models generate clips measured in seconds. A dialogue scene has to be built from many shots, not one long take.
• Time the dialogue before you generate anything. Shot durations should come from the lines, not a fixed default.
• Reaction shots and listeners are your best friends. They hide cuts, save lip sync work, and carry the emotion.
• Choose per shot between lip sync, model-generated speech, and voiceover. Not every line needs a moving mouth.
• Continuity across consecutive generations comes from shared reference images, a locked eyeline plan, and consistent lighting in your start frames.
• In Cinemachina, Video Sequence Timing lays out shots against the dialogue so durations are set before you spend anything on video.

Why AI dialogue scenes are hard

With a camera, a two-minute conversation is simple: you roll, they talk, you get coverage. With AI video you hit three walls at once.

Duration limits. Most video models generate clips from a few seconds up to roughly ten. Some go longer, up to around 30 seconds with multiple reference images, but limits change constantly (see our current AI video model table). A line fits. A scene doesn't.

Performance drift. The longer a clip runs, the more the model wanders: a face softens, a hand does something odd, the expression stops matching the line.

Mouths. Viewers read lips without knowing it. A mouth that moves over silence, or stays still over speech, breaks the illusion instantly.

The fix isn't a longer prompt or the model with the longest duration. It's to stop asking the model to do the editor's job.

Video-model duration limits vs. dialogue length

Here is the core math. A typical spoken line runs somewhere between two and eight seconds. A back-and-forth exchange of ten lines might run 45 seconds to a minute once you include pauses and reactions. If your model tops out at a handful of seconds per clip, that exchange needs somewhere around 10 to 20 shots.

That sounds like a lot. It isn't. Count the cuts in any conversation in a film or TV drama. Short shots are not an AI compromise; they're the grammar of screen dialogue. The question becomes "how should these 60 seconds be divided?" That's a directing question.

How to break a dialogue scene into shots

1. Record or generate the dialogue audio first

You can't time shots to dialogue you don't have. Lock the voice performances (recorded, or generated with your characters' approved voices) before you design shots. If you're working from an existing screenplay, the process of getting to this point is covered in from script to screen with AI.

2. Mark the beats, not just the lines

Mark where the emotional turns happen. "I got the job" is a line. The two seconds of silence after it, before the other character reacts, is a beat. Beats are where cuts want to live.

3. Assign each beat a shot type

For each beat, ask: whose face do we need to see right now? Often it's the listener. Then pick a shot:

• Establishing wide to show where everyone is.
• Medium two-shot for the opening exchange or a shift in dynamics.
• Over-the-shoulder to keep both characters in the frame while favoring one.
• Close-up single for important lines and reactions.
• Insert for a hand, an object, a phone screen. Great for covering a line in voiceover.

4. Size each shot to the audio

This is the step people skip. Don't give every shot a default four seconds. A 2.5-second line gets a 2.5-second shot plus a little handle. Keep most clips in the three-to-five-second range, with roughly ten seconds as the ceiling on many models.

5. Storyboard the start frame for every shot

Each shot gets an approved still frame before it becomes video, locking framing, eyelines, wardrobe, and lighting. More on this in AI storyboarding before generating video.

6. Generate, lip sync where needed, then cut

Generate each clip from its start frame, lip sync only the shots where a speaking mouth is clearly visible, and assemble. Because the audio was locked first, the edit mostly falls into place.

Reaction shots and coverage

Coverage is the set of angles a scene is shot from so the editor has choices. In AI filmmaking every angle costs a generation, so you design the coverage you actually need. Reaction shots earn their place three ways:

• They carry the emotion. "I'm leaving" matters less than the face of the person hearing it.
• They hide the seams. Cutting away mid-line and back lets you split a long speech across several generations invisibly.
• They don't need lip sync. A silent reacting face is one of the most reliable things a video model produces.

A rule from Crystal's own productions: the still frame captures the moment right before the emotion, and the video direction creates the turn. For a reaction, the start frame is the listener's neutral face; the direction is "her jaw tightens and she looks away." Start from the peak and the model has nowhere to go.

Character speaking vs. listening

Treat speaking shots and listening shots as two different kinds of generation, because they are.

Write speaking directions gently; models exaggerate "talking" into head bobs and theatrical gestures. For listening shots, name one specific reaction. "Reacts emotionally" gives you soup. "Her eyes drop to the table, then back up to him" gives you a performance.

Lip sync vs. generated dialogue vs. voiceover

There are three ways to put words in a character's mouth, and a good scene often uses all three.

Lip sync. A lip sync model reshapes the mouth of a generated clip to match your audio, or some engines generate a talking-photo-style clip from a still plus audio. You keep full control of the approved voice and delivery.

Model-generated dialogue. Some video models generate speech and mouth movement together. It can look natural, but you get less control over voice and timing, and a consistent voice across a film is harder. See choosing AI video models for each shot for when it's worth it.

Voiceover and off-screen dialogue. The line plays over a shot where the speaker's mouth isn't readable: a wide, the back of their head, the listener's face, an insert. No sync needed.

In Cinemachina, each scene has a lip sync button next to video generate, with a dropdown to choose the lip sync model (options such as Free Sync, Sync.so, HeyGen Lipsync) or Skip Lip Sync for shots that don't need it. Depending on the engine, lip sync is either a second step applied after the video is generated or a single talking-photo style generation. Skipping it on reaction shots and inserts is usually the right call.

Maintaining continuity between consecutive generations

Every cut back and forth is a chance for something to change that shouldn't: a jacket gets a new collar, the window moves, afternoon turns to dusk. Here's how to hold it together.

Use the same approved character references for every shot. Character consistency should be handled once, at the production level, not re-described in every prompt. In Cinemachina, characters are created once and their approved reference images, outfits, and voices are reused across the film. We cover this fully in how to keep characters consistent in AI video.

Pair names with a visual descriptor. Generators can take character names literally. On one of Crystal's music videos, an AI-avatar character kept getting rendered as a human man at a desk until the prompt added a short parenthetical description. In dialogue scenes with two or more named characters, always write "Maya (woman, 30s, short black hair, green jacket)" rather than just "Maya."

Lock the eyeline and the 180-degree line. Decide who is on the left of frame and who is on the right, and keep it that way. If Maya looks screen-right toward Theo in her close-up, Theo should look screen-left in his. Draw this into your storyboard frames so the model never has to guess.

Keep location and lighting consistent in your start frames. Reuse one approved location and match the direction and color of light. The video model mostly inherits what the start frame gives it.

Use start/end frames or multi-reference where supported. When a shot must land exactly where the next begins, a start/end frame pair gives a much tighter match.

Review in sequence. A clip that looks great alone can jump badly next to its neighbor. Watch the rough cut before regenerating.

Why dialogue should influence shot design

In many AI workflows, shots get designed first and dialogue gets bolted on later. That's backwards. The words tell you what the camera should do:

• A line that lands hard wants a close-up, either on the speaker or on the person it hits.
• A long speech wants to be broken up with reactions, or it will feel like a hostage video.
• A power shift wants a change in framing: from a balanced two-shot to a dominant single, or from eye level to a slight low angle.
• A pause wants its own shot. Silence on a face is often the best moment in the scene.
• Overlapping or fast banter wants wider shots where both characters are visible, so you're not cutting every second.

Dialogue also dictates duration, and duration dictates which model can handle the shot. Designing from the dialogue settles all of this before you spend a single generation.

Using Video Sequence Timing to sync dialogue before you generate

Most waste in AI dialogue scenes comes from generating clips first, then discovering they're the wrong length for the lines. Cinemachina's Video Sequence Timing flips that order. It lays out your blocks and shots against the dialogue, so every shot's duration is set before any video is generated.

A practical Video Sequence Timing pass for a dialogue scene looks like this:

1. Bring in the scene's dialogue audio.
2. Divide the scene into blocks (a block is a logical chunk of the scene, often one exchange or beat).
3. Within each block, place shots against the lines: speaker, listener, insert, wide.
4. Adjust shot lengths to the real audio, with a little handle on each side for cutting room.
5. Note which shots will need lip sync and which play as voiceover or reaction, so you know what to choose in the lip sync dropdown later.

From there, storyboard frames become the approved start frames, and Director Studio is where you review each shot's video direction, batch generate the clips, apply lip sync per shot, and watch a rough cut, reordering shots by drag-and-drop. For the full pipeline, see how to make an AI movie: the complete AI filmmaking workflow.

A worked example: a 40-second kitchen argument

Here's how a short scene might break down. Two characters, Maya and Theo, argue about a job offer. The locked dialogue runs about 40 seconds.

Ten shots, four lip synced, all well inside typical model limits. The emotional weight lands on shots 4, 7 and 9, and none needed a moving mouth.

FAQ

Can AI video generate a full conversation in one clip?

Some models can generate short two-character clips. For anything beyond a brief exchange, you'll get far better control by breaking the conversation into shots and cutting between speakers and listeners, as live-action dialogue is edited.

How do I make AI characters talk with lip sync?

Lock the dialogue audio, generate a clip from an approved start frame with the mouth neutral, then apply a lip sync model to match the audio. Some engines do it in one talking-photo style step. Only lip sync shots where the mouth is clearly visible.

How long should each shot be in an AI dialogue scene?

Size each shot to the line or beat it covers. Most land in the three-to-five-second range, longer for speeches, shorter for reactions and inserts. Check current limits on our AI video models page.

How do I keep two characters consistent across a dialogue scene?

Reuse the same approved reference images in every shot, pair each name with a short visual description, keep eyelines and screen direction fixed, and generate from storyboarded start frames with matching lighting.

Do I need lip sync for every line of dialogue?

No. Lines can play over reactions, over-the-shoulders, wides, or inserts where the mouth isn't readable. It saves time and cost, and often feels more cinematic.

Should I use a video model's built-in speech or separate lip sync?

Built-in speech is quick but gives less control over voice and delivery across a film. Separate lip sync keeps one approved voice per character. A good split: built-in speech for background lines, lip sync for key close-ups.

Start your dialogue scene

Great AI dialogue scenes are planned, not prompted. Lock the lines, time the shots, design the coverage, and only then generate. Cinemachina puts that whole sequence in one place, from character voices and storyboard frames to Video Sequence Timing, per-shot lip sync, and a rough cut in Director Studio. Start your film in Cinemachina.