WebMCPv4.0.518

Remotion Studio exposes WebMCP tools that allow an AI agent to inspect and steer it.

Capabilities

Agents can:

See which items you currently have selected
See selectable items and select them
See available compositions and navigate to them
Get the current HTML of your composition
See current outlines of elements and their geometry
Get your current playhead position and playback state
Play, pause and seek
Set timeline zoom, playback rate, and mute state
See, add and remove guides

Requirements

Start the Remotion Studio and open it in a browser with an AI agent that supports WebMCP.

Compatible agents

Out of the major harnesses, only ChatGPT Codex supports WebMCP. We hope more agents add support soon.

Available tools

note

The shape of inputs and outputs is not currently stable.

Tools that modify the timeline, playback or guides require a composition to be open.

`get_compositions`

Returns the mounted compositions in the same nested folder hierarchy shown in the Studio sidebar.

Input
{}

Output
type CompositionTreeItem =
  | {
      type: 'composition';
      compositionName: string;
    }
  | {
      type: 'folder';
      folderName: string;
      children: CompositionTreeItem[];
    };

{
  compositions: CompositionTreeItem[];
}

`select_composition`

Opens a registered composition by name.

Input
{
  compositionName: string;
}

Output
{
  currentComposition: string;
}

`get_sequences`

Returns the timeline sequences mounted for the current composition, including their hierarchy, timing, source stack and selection support.

Input
{}

Output
{
  currentComposition: string | null;
  sequences: Array<{
    sequenceId: string;
    name: string | null;
    type: 'sequence' | 'audio' | 'video' | 'image';
    parentSequenceId: string | null;
    depth: number;
    startFrame: number;
    endFrame: number;
    durationInFrames: number;
    stack: string | null;
    selectable: boolean;
  }>;
}

`select_sequence`

Selects and reveals a sequence using a sequenceId returned by get_sequences.

Input
{
  sequenceId: string;
}

Output
{
  currentComposition: string;
  selectedSequence: {
    sequenceId: string;
    name: string | null;
    type: 'sequence' | 'audio' | 'video' | 'image';
    parentSequenceId: string | null;
    depth: number;
    startFrame: number;
    endFrame: number;
    durationInFrames: number;
    stack: string | null;
    selectable: boolean;
  };
}

`get_composition`

Returns metadata and the current frame for the open composition. All fields are null when the canvas is showing an asset or other non-composition content.

Input
{}

Output
{
  compositionName: string | null;
  stack: string | null;
  durationInFrames: number | null;
  height: number | null;
  width: number | null;
  fps: number | null;
  currentFrame: number | null;
}

`get_canvas_html`

Returns the rendered composition HTML at the current frame without the surrounding Studio interface. The returned HTML is capped at 100,000 characters. Canvas and WebGL pixels are not represented.

Input
{}

Output
{
  currentComposition: string | null;
  currentFrame: number | null;
  html: string | null;
  htmlLength: number | null;
  truncated: boolean;
}

`get_outlines`

Returns the measurable selectable component outlines active at the current frame. Coordinates use composition pixels and do not change with the Studio canvas zoom.

Input
{}

Output
{
  currentComposition: string | null;
  currentFrame: number | null;
  outlines: Array<{
    sequenceId: string;
    parentSequenceId: string | null;
    name: string | null;
    location: {
      filename: string;
      line: number;
    } | null;
    geometry: {
      points: [
        {x: number; y: number},
        {x: number; y: number},
        {x: number; y: number},
        {x: number; y: number},
      ];
      boundingBox: {
        x: number;
        y: number;
        width: number;
        height: number;
      };
    };
  }>;
}

`get_playback_state`

Returns the current playhead, playback, audio, looping and timeline zoom state. Timeline zoom is normalized between 0 (fully zoomed out) and 1 (maximum zoom). Playback fields are null when the canvas is not showing a composition.

Input
{}

Output
{
  currentComposition: string | null;
  currentFrame: number | null;
  playing: boolean | null;
  muted: boolean | null;
  volume: number | null;
  playbackRate: number | null;
  looping: boolean | null;
  timelineZoom: number | null;
}

`get_selection`

Returns source-code context when exactly one supported timeline item is selected. currentSelection is null when there is no selection, multiple items are selected, a guide is selected or the source location cannot be resolved. selectedSequence identifies the owning sequence when a property, effect or keyframe is selected.

Input
{}

Output
{
  currentFrame: number;
  currentComposition: string | null;
  currentSelection: string | null;
  selectionType:
    | 'guide'
    | 'sequence'
    | 'sequence-prop'
    | 'sequence-all-effects'
    | 'sequence-effect'
    | 'sequence-effect-prop'
    | 'keyframe'
    | 'easing'
    | null;
  selectedSequence: {
    sequenceId: string;
    name: string | null;
    type: 'sequence' | 'audio' | 'video' | 'image';
    parentSequenceId: string | null;
    depth: number;
    startFrame: number;
    endFrame: number;
    durationInFrames: number;
    stack: string | null;
    selectable: boolean;
  } | null;
}

`get_guides`

Returns guides for the current composition. Vertical positions are x-coordinates from the left edge; horizontal positions are y-coordinates from the top edge.

Input
{}

Output
{
  currentComposition: string | null;
  guidesVisible: boolean;
  guides: Array<{
    id: string;
    orientation: 'horizontal' | 'vertical';
    position: number;
    visible: boolean;
  }>;
}

`set_guides_visible`

Shows or hides every guide in the current composition.

Input
{
  visible: boolean;
}

Output
{
  currentComposition: string;
  guidesVisible: boolean;
}

`add_guide`

Adds and shows a guide in the current composition. Positions use composition pixels.

Input
{
  orientation: 'horizontal' | 'vertical';
  position: number;
}

Output
{
  currentComposition: string;
  guide: {
    id: string;
    orientation: 'horizontal' | 'vertical';
    position: number;
    visible: true;
  };
}

`remove_guide`

Removes a guide using an ID returned by get_guides or add_guide.

Input
{
  guideId: string;
}

Output
{
  currentComposition: string;
  guideId: string;
  removed: true;
}

`play`

Starts playback from the current frame. Still compositions cannot be played.

Input
{}

Output
{
  currentComposition: string;
  playing: true;
}

`pause`

Pauses playback at the current frame.

Input
{}

Output
{
  currentComposition: string;
  playing: false;
}

`mute`

Mutes audio playback.

Input
{}

Output
{
  currentComposition: string;
  muted: true;
}

`unmute`

Unmutes audio playback.

Input
{}

Output
{
  currentComposition: string;
  muted: false;
}

`set_timeline_zoom`

Sets the timeline zoom using a normalized value between 0 (fully zoomed out) and 1 (maximum zoom). The output contains the effective normalized zoom after Studio applies its supported zoom step. Still compositions do not have timeline zoom.

Input
{
  zoom: number;
}

zoom must be a positive finite number.

Output
{
  currentComposition: string;
  timelineZoom: number;
}

`set_playback_rate`

Sets the playback multiplier. Negative values play backwards.

Input
{
  playbackRate:
    | -4
    | -2
    | -1
    | -0.5
    | -0.25
    | 0.25
    | 0.5
    | 1
    | 1.5
    | 2
    | 4;
}

Output
{
  currentComposition: string;
  playbackRate: number;
}

`seek_to_frame`

Seeks to a zero-based frame. Values beyond the duration are clamped to the final frame.

Input
{
  frame: number;
}

frame must be a non-negative integer.

Output
{
  currentComposition: string;
  currentFrame: number;
}

Example prompts

You can ask a browser agent:

"What is currently selected in the Studio?"
"List all mounted compositions and their folders."
"Open the Shapes composition."
"List the sequences and select the Title sequence."
"Inspect the HTML rendered on the canvas."
"List the visible components and their canvas geometry."
"What is the current playback state?"
"Set the timeline zoom to 2.5x."
"Seek to frame 90, set the playback rate to 0.5x and play."
"List the guides in the current composition."
"Hide the guides."
"Add a vertical guide at 640 pixels."
"Mute the preview."

The agent chooses and invokes the matching tools. Tool calls affect the open Studio tab immediately.

Browser support

WebMCP is a draft web platform API. Refer to the WebMCP specification and Chrome's WebMCP documentation for current browser and agent support.

Capabilities​

Requirements​

Compatible agents​

Available tools​

get_compositions​

select_composition​

get_sequences​

select_sequence​

get_composition​

get_canvas_html​

get_outlines​

get_playback_state​

get_selection​

get_guides​

set_guides_visible​

add_guide​

remove_guide​

play​

pause​

mute​

unmute​

set_timeline_zoom​

set_playback_rate​

seek_to_frame​

Example prompts​

Browser support​

See also​