# Seedance Lip Sync: Dialogue Prompts and 4 Real Clips > We wrote spoken lines into Seedance 2.0 prompts and sent our own voice clip as a reference: mouth, audio track, request fields and four clips from one day. - Canonical HTML: https://omniakey.com/blog/seedance-lip-sync - Markdown: https://omniakey.com/blog/seedance-lip-sync.md - Published: 2026-11-03 (updated 2026-11-03) - Author: Kiki, COO at OmniaKey - Language: en - Category: Guide - Reading time: 11 min - All posts: https://omniakey.com/blog/llms.txt --- Seedance lip sync starts in the prompt: our two Seedance 2.0 clips that differed only in the audio switch both ran closed, slightly open, wide open. Our own 1.8-second voice clip went through `reference_to_video`, and the output carried the line. Audio alone is rejected. BytePlus charges $0.15 per second at 720P, so 5 seconds costs $0.75. ## Short answer - The prompt drives Seedance lip sync. Two 5-second 720P clips from one prompt, one with generated audio and one silent, both ran closed → slightly open → wide open. - Audio alone fails. The 2.0 series needs at least one reference image or reference video in the same request (BytePlus's create-task reference, checked November 3, 2026). - Reference audio takes up to 3 clips, 2–15 seconds each, 15 seconds in total, wav or mp3, 15 MB each. - The track is used. Our 1.8-second mp3 came back inside the output audio, with the words intact. - Price at BytePlus's list rate: $0.15 per second at 720P, so 5 seconds is $0.75. - Four clips from one day: 720P with audio, 720P silent, a square 720P image-to-video, and a 480P clip driven by our own recording. ## Does Seedance 2.0 do lip sync? The docs never use the phrase. What they describe is a model that generates picture and audio together, plus one line of advice about dialogue in double quotes. ByteDance's Seed page calls Seedance 2.0 "a unified multimodal audio-video joint generation architecture that supports text, image, audio, and video inputs" (checked November 3, 2026). The BytePlus ModelArk guide says the same and lists "Generate videos with audio" among its capabilities (page updated September 29, 2026). Search those pages and the words are missing: "lip sync" and "talking" never appear, "mouth" appears once inside a sample prompt, and "dialogue" twice, both in sentences about audio generation. The nearest thing to an instruction is "It is recommended to put dialogue content in double quotes to optimize the audio generation effect" (create-task reference, updated October 28, 2026). So the capability sits in the gap: the model makes a picture and a track, and no official page promises the lips land on each syllable. Our four runs show the mouth moving with audio on and with audio off, which fits a picture model told in the prompt that someone is speaking. Readers have been guessing for months. A Hacker News comment on February 18, 2026 sent a song along with a clip and found "the model did reference the singing music I provided, but it reinterpreted it on its own" ([comment](https://news.ycombinator.com/item?id=47056286)). A GitHub issue from June 3, 2026 sets "Mouth movements match phoneme timing" as a pass condition, a measurement the docs never mention ([issue](https://github.com/etblues449/Faceless-Finance/issues/92)). ## How to write a Seedance lip sync prompt Name the character, put the words in double quotes, then describe what the mouth and face do. Our prompt did the first and third and skipped the quotes, which is why the model wrote its own line. The recipe from our runs: 1. Cast the character in one sentence: stop-motion clay, big round glasses, knitted scarf, sitting at a kitchen table. 2. Say what they do with their voice: "speaks one short line to the viewer". 3. Put the exact words in double quotes, the way BytePlus recommends. 4. Describe the mouth and face: "opens and closes in time with the words", eyebrows moving with the sentence, a head tilt on the stressed word. 5. Lock the camera and the light, so the only motion left is the performance. This is the prompt we sent, unchanged, for all four clips: > A stop-motion clay character with big round glasses and a knitted scarf sits at a kitchen table, looks straight into the camera, and speaks one short line to the viewer. The mouth opens and closes in time with the words, the eyebrows move with the sentence, and the head tilts slightly on the stressed word. Fixed camera, warm lamp light, a mug of tea in front of them, gentle 24 fps stop-motion feel. Note what is absent: no quoted sentence. The prompt asked for "one short line", so the model picked the words. With audio on, the track says "You look like you could use a warm hug." — a sentence that appears nowhere in the request. That is the reason to add step 3 before you send anything. We read the mouth on a grid of frames sampled every half second. In the audio version it starts closed, sits slightly open from frame 2 to 7, opens wide at 8, then slightly open and closed. The silent version runs closed, slightly open, wide, slightly open, wide, then slightly open for two frames. ![Ten close-ups of a clay character with round glasses and a knitted scarf, one every half second: mouth shut in the first and last frames, open in between and widest in frame 8.](https://omniakey.com/_img/blog/seedance-lip-sync/t1-t2v-audio-strip.3d59731f.1600.webp) _Audio on, one frame every half second: the mouth is shut in frame 1, slightly open from 2 to 7, wide at 8 and shut again by 10. Seedance 2.0 via OmniaKey, text to video, 5 seconds, 720P, made November 3, 2026._ ![Ten close-ups of an orange clay character in a dark green scarf, one every half second, mouth opening and closing, from a clip with no audio track.](https://omniakey.com/_img/blog/seedance-lip-sync/t1-t2v-silent-strip.58b1caee.1600.webp) _Audio off, same prompt: the mouth still opens and closes, widest in frames 5, 6 and 8, with no audio track in the file at all. 5 seconds, 720P, made November 3, 2026._ The two clips also rendered different characters from the same words, so treat the casting sentence as a suggestion; the silent one came back orange, with a grey-brown parting and a dark green scarf. The framing held still in both, which is what "fixed camera" bought us. ## Can Seedance lip sync to your own audio? Yes, with one reference image or one reference video in the same request. Our 1.8-second mp3 went in as `reference_audio`, and the words came back in the output track. We used `reference_to_video` with three pieces: the clay character still as `reference_image`, our recording as `reference_audio`, and the same spoken-line prompt. The request was 480P, 5 seconds, `generate_audio: true`. OmniaKey answered 202, the task `task_3Ay0Ktf2mbYXh9GNZpZ5j9kGyOIzz8Pv` succeeded, and the MP4 arrived after 120.2 seconds at 864×496 with a 5.06-second aac track. The transcription of that track reads: "Why tow it? I told you the keys were in the drawer." The second half is our recording, word for word. The first three words are not in the file we sent, and a transcript cannot tell whether they are a breath of ours or new speech, so we report the sentence as heard and stop there. ![Ten frames of a brown clay character with a hat and orange scarf at a wooden table, mouth closed, open, closed and open again, with steam moving above a mug.](https://omniakey.com/_img/blog/seedance-lip-sync/t3-ref-audio-strip.edfa40c7.1600.webp) _Our own recording as the reference: the mouth moves and the steam above the mug drifts, while the output audio carries the words we sent. 5 seconds, 480P, made November 3, 2026._ Ten frames at 0.5-second steps show movement, and that is all they show. Phoneme-level alignment needs a frame rate and a measurement this test does not have, so nothing here says the lips hit each sound. The limits come from BytePlus's create-task reference (checked November 3, 2026): up to 3 audio clips, 2–15 seconds each, 15 seconds in total, wav or mp3, 15 MB each. A clip can arrive as a public URL, a Base64 string or an asset ID. Audio on its own is not accepted: "Audio-only input is not supported; include at least one reference image or video." > Keep real people out of the request. The 2.0 series does not accept reference images or videos that contain real human faces, so a clay, illustrated or CG character is the safe input for a talking shot. ## Image-to-video moves the camera by itself Our image-to-video clip cut from a standing full-body shot to a seated kitchen close-up at frame 4, and the prompt asked for neither a cut nor a camera move. The first frame was the still we made for this test: a clay character standing in front of a warm cream wall, hands at the sides, whole body in frame, 1254×1254 pixels. The prompt is the same one above, the one that puts the character at a kitchen table. Frames 1 to 3 hold the standing shot; frame 4 jumps to a seated close-up behind a table and stays there for the rest of the clip. The mouth is closed until frame 7, slightly open at 8, wide at 9 and 10. The track says "Well, what do you think?" over a cloth rustle. ![Ten frames from a square clip: a standing clay character in the first frames, then a seated character at a kitchen table with the mouth open.](https://omniakey.com/_img/blog/seedance-lip-sync/t2-i2v-audio-strip.c23818f5.1600.webp) _Image to video: the standing first frame holds for a moment, then the clip cuts to a seated kitchen shot the prompt asked for and the picture never showed. 5 seconds, 720P, made November 3, 2026._ The lesson is about staging, not about mouths. Here the prompt decides where the character is and what the camera does, and the first frame mostly decides what the character looks like. If a shot matters, write it into the prompt. The clip arrived at 960×960, not 1280×720, because we sent no `ratio` with a square still and the API default is `adaptive`. Ask for the frame you want instead of assuming the resolution label decides it. ## The Python request Create the task, poll the task id, download the MP4. Three calls, standard library only, and the same helper serves all three operations. ```python import base64 import json import os import time import urllib.error import urllib.request import uuid API = "https://api.omniakey.com/v1/media/tasks" KEY = os.environ["OMNIAKEY_API_KEY"] MODEL = "dreamina-seedance-2-0-260128" PROMPT = ( "A stop-motion clay character with big round glasses and a knitted scarf sits at a kitchen table, " "looks straight into the camera, and speaks one short line to the viewer. The mouth opens and closes " "in time with the words, the eyebrows move with the sentence, and the head tilts slightly on the " "stressed word. Fixed camera, warm lamp light, a mug of tea in front of them, gentle 24 fps " "stop-motion feel." ) MIME = {".png": "image/png", ".jpg": "image/jpeg", ".jpeg": "image/jpeg", ".webp": "image/webp", ".mp3": "audio/mpeg", ".wav": "audio/wav"} def call(method, url, body=None, idempotency_key=None): headers = {"Authorization": f"Bearer {KEY}", "User-Agent": "omniakey-example/1"} data = None if body is not None: headers["Content-Type"] = "application/json" data = json.dumps(body).encode() if idempotency_key: headers["Idempotency-Key"] = idempotency_key req = urllib.request.Request(url, data=data, headers=headers, method=method) with urllib.request.urlopen(req, timeout=120) as resp: return json.load(resp) def data_url(path, limit_mb): raw = open(path, "rb").read() if len(raw) > limit_mb * 1024 * 1024: # image 30 MB, audio 15 MB raise ValueError(f"{path} is over {limit_mb} MB, the limit for this input type") mime = MIME[os.path.splitext(path)[1].lower()] return f"data:{mime};base64," + base64.b64encode(raw).decode() def wait(task_id, timeout=900, step=5): deadline = time.time() + timeout while time.time() < deadline: state = call("GET", f"{API}/{task_id}") if state["status"] not in ("queued", "processing"): return state time.sleep(step) raise TimeoutError(f"{task_id} is still running: poll it again, do not resubmit") def run(body, out_path): task_id = call("POST", API, body, str(uuid.uuid4()))["id"] print("task", task_id, flush=True) state = wait(task_id) if state["status"] != "succeeded": raise RuntimeError(state.get("error") or state["status"]) signed = state["assets"][0]["url"] # expires, so download straight away req = urllib.request.Request(signed, headers={"User-Agent": "omniakey-example/1"}) with urllib.request.urlopen(req, timeout=300) as resp, open(out_path, "wb") as f: f.write(resp.read()) return task_id def reference_to_video(prompt, image_path, audio_path, out_path): body = { "model": MODEL, "type": "video", "operation": "reference_to_video", "input": { "prompt": prompt, "media": [ {"type": "reference_image", "url": data_url(image_path, 30)}, {"type": "reference_audio", "url": data_url(audio_path, 15)}, ], }, "parameters": {"duration": 5, "resolution": "480P", "ratio": "16:9", "generate_audio": True, "watermark": False}, } return run(body, out_path) if __name__ == "__main__": print("done", reference_to_video(PROMPT, "character.png", "line.mp3", "line-clip.mp4")) ``` Three things in that file earn their keep. The `Idempotency-Key` is a fresh UUID per body, so a retried request returns the original task instead of paying twice. The wait loop stops after 15 minutes and raises, because a timeout means the clip is still rendering and resubmitting buys a second one. The download happens on the first `succeeded`, because `assets[0].url` is a signed link that expires. The other two operations swap the body and keep the helpers: ```python def text_to_video(prompt, out_path): body = { "model": MODEL, "type": "video", "operation": "text_to_video", "input": {"prompt": prompt}, "parameters": {"duration": 5, "resolution": "720P", "ratio": "16:9", "generate_audio": True, "watermark": False}, } return run(body, out_path) def image_to_video(prompt, image_path, out_path): body = { "model": MODEL, "type": "video", "operation": "image_to_video", "input": {"prompt": prompt, "media": [{"type": "first_frame", "url": data_url(image_path, 30)}]}, "parameters": {"duration": 5, "resolution": "720P", "ratio": "adaptive", "generate_audio": True, "watermark": False}, } return run(body, out_path) ``` The field list is short: `model`, `type`, `operation`, `input.prompt`, `input.media[]` with `type` set to `first_frame`, `reference_image`, `reference_video` or `reference_audio`, and `parameters.duration`, `resolution`, `ratio`, `generate_audio`, `watermark`. OmniaKey's [video API docs](https://docs.omniakey.com/en/media/video) carry the same shape, and [the media API overview](https://docs.omniakey.com/en/media/overview) covers the create-poll-download cycle. The model id maps to the [Seedance 2.0 model page](https://omniakey.com/dreamina-seedance-2-0-260128). ## What it costs BytePlus bills this model by the second. The list price for a 5-second, 16:9 output with no video input looks like this (ModelArk pricing, checked November 3, 2026): |Output|Per second|5 s clip| |-|-|-| |480p|$0.07|$0.35| |720p|$0.15|$0.76| |1080p|$0.37|$1.87| |4K|$0.78|$3.89| Work it out from the per-second column and you get the same numbers, with a cent of rounding on the page: 5 seconds × $0.15 = $0.75, against the $0.76 in the 720p row. Our two text-to-video clips were 720P at 5 seconds, so each sits at that 75-cent mark. A request that carries a video input is billed differently, and higher, so a reference-video job does not land on these figures (ModelArk pricing, checked November 3, 2026). OmniaKey charges per generated second and returns the charge when a task ends `failed` or `canceled` (media API overview, checked November 3, 2026); current rates are on the [model page](https://omniakey.com/dreamina-seedance-2-0-260128). ## What goes wrong - Audio on its own is rejected. Send at least one reference image or reference video with the track. - Real faces are rejected. A reference image or video with a real person in it will not pass, so cast a clay, drawn or CG character. - `generate_audio` defaults to `true`. A silent file needs `false`; a spoken line needs the default and a prompt that says someone is talking. - Audio-on jobs can be refused at submit for an output-audio policy reason. All three of our audio-on jobs went through, but if a submit comes back rejected, try the same body with `generate_audio: false`. - The delivered size follows `ratio`. 720P gave 1280×720 from text, 960×960 from a square still with no ratio, and 480P at 16:9 gave 864×496. - Speed varies. Four 5-second jobs took 259.0, 202.0, 155.3 and 120.2 seconds, so plan for two to five minutes and poll. - Length is capped at 4–15 seconds of output, and reference clips at 2–15 seconds each with 15 seconds in total. All four files came back at 24 fps. ## Common questions ### Can I upload a WAV file and have the mouth follow it? You can send the WAV with a reference image, and the model will use it: our own recording came back in the output audio. Whether Seedance lip sync lands on each sound in your file is something our frames cannot show; one frame every half second shows the mouth moving, not a measured match. ### Why was my audio-only request rejected? The 2.0 series does not accept audio as the only input. Add a reference image or a reference video to the same request and the audio is accepted alongside it. ### Should generate\_audio be on or off for a spoken line? Leave it on, which is the default. With `false` the file has no audio track at all, and the character still opens its mouth if the prompt says it is speaking, as our silent clip did. ### How long can the reference audio be? Each clip runs 2–15 seconds, up to 3 clips, 15 seconds in total, in wav or mp3 at 15 MB each (BytePlus's create-task reference, checked November 3, 2026). Our own clip was 1.8 seconds, shorter than that minimum, and still went through as a Base64 data URL. ## Sources checked - [ByteDance Seed: Seedance 2.0](https://seed.bytedance.com/en/seedance2_0), checked November 3, 2026 - [BytePlus ModelArk: Seedance 2.0 guide](https://docs.byteplus.com/en/docs/ModelArk/seedance-2-0), checked November 3, 2026 - [BytePlus ModelArk: create video generation task API](https://docs.byteplus.com/en/docs/ModelArk/create-video-generation-task-api), checked November 3, 2026 - [BytePlus ModelArk: model pricing](https://docs.byteplus.com/en/docs/ModelArk/model-pricing), checked November 3, 2026 ## Update log - November 3, 2026: first version, with four clips made that day.