Lipsync

The Lipsync card takes a talking-head video and a speech track, then re-renders the video so the speaker’s lip movements match the audio. The speech can be an existing audio file or synthesized in the same job.

Note

All generation endpoints require the agent_id query parameter.

Models

data.lip_sync_model	Notes
"gen"	Default. Faster, good quality.
"sync.so"	Higher quality, slower processing.

---

Generate lipsync

POST /v1/vidsheet/{id}/cells/{cell_id}/generate?agent_id={agent_id}

Request body

Field	Type	Required	Description
generation_type	string	Yes	"lipsync".
data.talking_avatar_video_data.video.value	string (URI)	Yes	Public URL of the source video.
data.talking_avatar_video_data.audio.value	string (URI)	One of	Public URL of the speech audio.
data.voice_generation_data	object	One of	Synthesize the speech in this job — same shape as Speech from Text data (text, voice_id, …).
data.lip_sync_model	string	No	"gen" (default) or "sync.so".
data.remove_background	boolean	No	Remove the video background. Default false.
data.remove_silence	boolean	No	Trim silence from synthesized speech. Default false.

Exactly one of audio.value or voice_generation_data must be supplied. No other top-level fields are accepted.

Tip

Resource URLs come from GET /v1/content_resources. Chain the steps: generate speech first, then pass its output URL as audio.value.

Example

curl -X POST "https://api.gen.pro/v1/vidsheet/101/cells/3000/generate?agent_id=42" \
  -H "X-API-Key: your-api-key" \
  -H "Content-Type: application/json" \
  -d '{
    "generation_type": "lipsync",
    "data": {
      "talking_avatar_video_data": {
        "video": { "value": "https://cdn.gen.pro/uploads/talking-head.mp4" },
        "audio": { "value": "https://cdn.gen.pro/outputs/voiceover-abc123.mp3" }
      },
      "lip_sync_model": "gen"
    }
  }'

You can also generate on a layer within the cell:

POST /v1/vidsheet/{id}/cells/{cell_id}/layers/{layer_id}/generate?agent_id={agent_id}

The request body is identical.

---

Response

Poll Check generation status until complete. The finished generation returns the lip-synced video in output_resources.

{
  "id": 9002,
  "status": "completed",
  "output_resources": [
    {
      "id": 1502,
      "url": "https://cdn.gen.pro/outputs/lipsynced-def456.mp4",
      "object_type": "video",
      "type": "output"
    }
  ]
}

Errors

Status	Error code	Description
422	validation_error	Missing video.value, or neither audio.value nor voice_generation_data supplied.
404	not_found	Sheet, cell, or layer not found.

---

Cell configuration

Tip

Cell configuration is set via PATCH /v1/vidsheet/{id}/cells/{cell_id} with additional_attributes in the request body. These defaults are used when generation is triggered.

Field	Type	Description
lipSyncModel	string	Default model: "gen" or "sync.so".
autoGenerate	boolean	Auto-trigger generation when source video/audio changes.