All Generation Types

Every cell or layer generation declares a generation_type and a data object. This page lists every valid combination in one place. See the Creation Cards section for per-type deep dives with examples.

Quick index

`generation_type`	Produces	Key models
`text`	text	openai, gemini
`image_from_text`	image	gemini, gemini_pro, gemini_2, gemini_2_lite, imagen, seedream-4.0, seedream-4.5, gpt-image-2, gpt-image-2.5-flare, gpt-image-2.5-sunburst, midjourney, grok
`video_from_text`	video	veo-3, veo-3-fast, veo-3.1, veo-3.1-fast, veo-3.1-lite, gemini-omni-flash-preview, kling-v3, kling-v1-6, seedance-lite, seedance-pro, seedance-1.0-lite, seedance-1.0-pro, seedance-1.5-pro, seedance-2.0, seedance-2.5, grok
`video_from_image`	video	veo-3, veo-3-fast, veo-3.1, veo-3.1-fast, veo-3.1-lite, gemini-omni-flash-preview, kling-v3, kling-v2-6, kling-v2-1, kling-v1-6, seedance-lite, seedance-pro, seedance-1.0-lite, seedance-1.0-pro, seedance-1.5-pro, seedance-2.0, seedance-2.5, grok
`video_from_ingredients`	video	pika
`speech_from_text`	audio	(uses a saved `voice_id`)
`lipsync`	video	`lip_sync_model`: sync.so, gen
`captions`	caption layer (timed segments)	—
`transcription`	transcript text + sentence timings	—

Uploaded media is a cell value, not a generation — see Media. Compositing is the dedicated render endpoint, with no generation_type.

Text

{
  "generation_type": "text",
  "data": {
    "prompt": "Write a 12-second TikTok hook for {{topic}}",
    "model": "gemini",
    "variables": { "topic": "San Antonio tacos" }
  }
}

Variables substitute {{key}} in the prompt. Output lives in the cell’s value as a plain string.

Image from Text

{
  "generation_type": "image_from_text",
  "data": {
    "prompt": "a neon-lit street food stall at night, handheld feel",
    "model": "midjourney",
    "aspect_ratio": "9:16",
    "variables": { }
  }
}

Aspect ratios: 1:1, 9:16, 16:9, 4:3, 3:4; Grok also supports 3:2, 2:3. Output is a content resource (image).

Video from Text

{
  "generation_type": "video_from_text",
  "data": {
    "prompt": "San Antonio taco truck at golden hour, steam rising, handheld camera",
    "model": "veo-3.1",
    "aspect_ratio": "9:16",
    "resolution": "1080p",
    "duration": 8,
    "native_audio": false,
    "negative_prompt": "no text overlays, no logos"
  }
}

Aspect ratios: 1:1, 9:16, 16:9; Grok also supports 4:3, 3:4, 3:2, 2:3. Accepted duration (seconds) per model: veo-3, veo-3-fast, veo-3.1, veo-3.1-fast, veo-3.1-lite (4|6|8s); gemini-omni-flash-preview (1–10s); kling-v3, kling-v1-6; seedance-lite, seedance-pro, seedance-1.0-lite, seedance-1.0-pro, seedance-1.5-pro (3–12s); seedance-2.0 (4–15s); seedance-2.5 (4–15s); grok (1–15s). Resolution is model-dependent. Use resolution: "1080p" or "720p" where supported. native_audio is optional and defaults to false.

Video from Image

{
  "generation_type": "video_from_image",
  "data": {
    "image_resource_id": 4821,
    "image_tail_resource_id": 4822,
    "prompt": "zoom in slowly, handheld feel",
    "model": "kling-v2-6",
    "aspect_ratio": "9:16",
    "resolution": "1080p",
    "native_audio": false,
    "duration": 5
  }
}

image_tail_resource_id optional — provides a target end frame for the video. resolution and native_audio are optional and model-dependent.

Video from Ingredients

{
  "generation_type": "video_from_ingredients",
  "data": {
    "prompt": "combine these 3 products in a tabletop pan-around shot",
    "images": [
      { "value": "https://cdn.gen.pro/uploads/product-1.png" },
      { "value": "https://cdn.gen.pro/uploads/product-2.png" }
    ],
    "native_audio": false
  }
}

Pika composites the uploaded images (public URLs from your content resources) into one clip. prompt and images are required; native_audio is optional. No other fields are accepted.

Speech from Text

{
  "generation_type": "speech_from_text",
  "data": {
    "text": "Welcome to Santiago's taco tour...",
    "voice_id": "21m00Tcm4TlvDq8ikWAM",
    "enhance_voice": true,
    "remove_silence": false
  }
}

text is required (1–4000 characters); voice_id is a saved voice from GET /v1/agents/{agent_id}/voice/library. Designing or cloning a voice are separate jobs (design_voice, clone_voice), not modes of speech_from_text.

Lipsync

{
  "generation_type": "lipsync",
  "data": {
    "talking_avatar_video_data": {
      "video": { "value": "https://cdn.gen.pro/uploads/talking-head.mp4" },
      "audio": { "value": "https://cdn.gen.pro/outputs/voiceover-abc123.mp3" }
    },
    "lip_sync_model": "gen",
    "remove_background": false
  }
}

talking_avatar_video_data.video.value is required. Supply the speech either as talking_avatar_video_data.audio.value or as voice_generation_data (the same data shape as speech_from_text) to synthesize it in the same job. lip_sync_model: sync.so or gen (default gen).

Captions

{
  "generation_type": "captions",
  "data": {
    "video": { "value": "https://cdn.gen.pro/outputs/final-abc123.mp4" },
    "trim": { "start_seconds": 0, "duration_seconds": 30 }
  }
}

Pass exactly one source: audio or video ({ "value": "<uri>" }). trim is optional. Produces a caption layer with timed segments (result.captions, plus full_text and audio_duration).

Transcription

{
  "generation_type": "transcription",
  "data": {
    "audio": { "value": "https://cdn.example.com/episode-12.mp3" }
  }
}

Pass exactly one source: audio, video, or content_resource_id. Add trim with start_seconds and duration_seconds to transcribe only part of it. Returns full_text, sentences with startMs/endMs, and audio_duration. Also available without a sheet via POST /v1/transcriptions?agent_id=. See Transcription.

Render (composite)

POST /v1/vidsheet/:id/cells/:cell_id/render?agent_id=

No generation_type in the body — the render endpoint is dedicated. It composites all layers on the cell in order into one final video. Output lives in the cell’s output_resources.