MP4E — Interactive video that travels in the file

Overview

Plugins run in a sandboxed iframe and communicate with the player via the mp4e API. This API provides access to video state, variables, and the ability to execute actions.

Interactive API Explorer

You can also explore the plugin API interactively in the Studio's Plugin Editor — open the API tab to see all available methods with live examples.

Sandboxed Environment

Plugin JavaScript runs in a sandboxed iframe and talks to the player through the mp4e bridge API. By default the iframe has no same-origin access: your plugin cannot reach the host page's DOM, cookies or JavaScript, or other plugins. Don't use localStorage, sessionStorage, cookies or window.parent directly — use mp4e.storage and mp4e.fetch instead. A host can restrict what plugins may reach over the network with config.pluginSecurity (network: 'none' or 'manifest'), or opt in to same-origin access with trustContent (trusted); write the plugin for the isolated default.

Overlay clicks & actions

If your plugin is used as an overlay, clicks inside the iframe are forwarded to the overlay click trigger. That means you can attach onClick actions/rules to the overlay without writing custom click handlers in plugin JavaScript.

Global Object

When a plugin is triggered by an object click or hover, the object global is available with information about the triggering object:

object global structure

// The 'object' global is available when plugin is triggered by an object click/hover
{
  id: "obj_watch_1",           // Unique object identifier
  label: "watch",              // Original AI-detected label
  userLabel: "Apple Watch",    // User-customized label (optional)
  position: {
    x: 320,                    // Center X position
    y: 180,                    // Center Y position
    width: 150,                // Bounding box width
    height: 100,               // Bounding box height
    bbox: [245, 130, 395, 230] // [x1, y1, x2, y2]
  },
  data: {                      // Custom data from metadata
    price: 399,
    sku: "WATCH-001",
    description: "Series 9 GPS",
    productUrl: "https://example.com/watch"
  }
}

Object May Be Null

For overlay plugins that aren't attached to specific objects, object may be null. Always check: if (object) { ... }

Lifecycle

Lifecycle methods control plugin initialization and visibility. The bridge automatically calls mp4e.ready() after DOMContentLoaded, but you can call it earlier if your plugin finishes setup before that.

mp4e.ready()

Signal that the plugin has finished initializing. Call after your DOM setup is complete. This is automatically called after DOMContentLoaded if your script does not call it first. Calling it multiple times is safe — only the first call has effect.

mp4e.ready();

mp4e.hide()

Hide this overlay. The plugin script continues running in the background and can still listen to events and update variables. The hide request is routed through the engine so the host can gate or track visibility changes. Call show() to make it visible again.

mp4e.hide();

mp4e.show()

Show this overlay after it was previously hidden with hide(). The show request is routed through the engine so the host can gate or track visibility changes.

mp4e.show();

mp4e.isVisible()

Check if the overlay is currently visible. Returns false after hide() has been called, true after show() or initially.

Returns: boolean

if (!mp4e.isVisible()) mp4e.show();

mp4e.close()

Closes the current plugin (modal or display). For display plugins triggered by object clicks/hovers, this dismisses the display. For modal plugins, this closes the modal and resumes video if it was paused.

mp4e.close();

mp4e.getPluginId()

Returns this plugin's unique ID. For pool-managed (group-bound) overlays, this includes the bound object ID.

Returns: string

const id = mp4e.getPluginId(); // e.g., "overlay_tooltip_1" or "pool-hotspot:obj_chair_1"

Sync Methods

These methods return values immediately (synchronously):

mp4e.getConfig()

Returns the plugin configuration object. Contains values from the plugin editor settings panel. Config values are automatically interpolated by the engine when they contain template expressions like {{variableName}}.

Returns: object

Common Mistake: Don't Use mp4e.config

Always use mp4e.getConfig() to access configuration in JavaScript. Do NOT use mp4e.config - it doesn't exist!

CORRECT:const config = window.mp4e?.getConfig();

WRONG:const config = mp4e.config;(doesn't exist)

WRONG:const title = '{{title}}';(template syntax only works in HTML/CSS)

getConfig() example

// Access plugin configuration values
const config = mp4e.getConfig();

// config contains values from the plugin editor:
{
  title: "Product Details",
  showPrice: true,
  currency: "USD",
  buttonText: "Add to Cart",
  // ... any values defined in your plugin settings
}

// Use in your code:
document.getElementById('title').textContent = config.title;
if (config.showPrice) {
  document.getElementById('price').textContent =
    config.currency + ' ' + object.data.price;
}

mp4e.getData()

Returns the full data object passed to the plugin (most commonly { object, groups }). For controls plugins, this also includes menuConfig, tracks, markers, thumbnails, and state.

Returns: { object?: any, groups?: any[], display?: { objectId: string, event: string }, menuConfig?: any, tracks?: any, markers?: any[], state?: any }

getData() example

// Access the full data object
const data = mp4e.getData();

// data structure:
{
  object: { ... },        // Same as the 'object' global
  groups: [               // Object groups this object belongs to
    { id: "products", name: "Products", color: "#3b82f6" },
    { id: "featured", name: "Featured Items", color: "#10b981" }
  ]
}

When the plugin was opened as an object's display (a group's or object's display settings: clicked, hovered, on screen), getData().display is { objectId, event }. Use it to render a single card from your config instead of a full-frame, timed layout:

Display mode

const display = mp4e.getData().display;   // { objectId: "obj_2", event: "clicked" } or undefined
if (display) {
  // Opened for one object: the frame is the card. The config's fields were
  // filled from the object by the binding ("name": "{{object.data.title}}").
  renderCard(mp4e.getConfig());
  closeButton.onclick = () => mp4e.close();   // the player closes the display
} else {
  renderTimedLayout();
}

Declare the card's size in your plugin's manifest as "display": { "width": 600, "height": 400 } — design pixels on a 1920 base, measured on the video's long side. The player draws the display frame exactly that size unless the creator gives the binding a size of its own, and the Studio offers a plugin that declares one as an object display even when its type is overlay.

mp4e.getPageUrl()

Returns the URL of the host page where the player is embedded, or null if not available.

Returns: string | null

const pageUrl = mp4e.getPageUrl();

mp4e.getVideoState()async

Gets the full video state object from the engine including current time, duration, playing state, volume, and more. startTime is where the host started playback (its initialTime; 0 at the beginning) — a viewer who arrived at a moment, e.g. from search, has startTime > 0.

Returns: Promise<object>

const state = await mp4e.getVideoState();

mp4e.log(message)(message: string)

Logs a message to the browser console with a plugin prefix. Useful for debugging.

mp4e.log('Button clicked!');

Async Methods

These methods return Promises and must be awaited. They communicate with the parent player via postMessage and resolve when the response arrives. A call rejects if no response arrives within 5 seconds, except mp4e.getMediaUrl, which waits up to 60 seconds because a large media item can take longer to load.

mp4e.getVariable(name)(name: string)async

Gets the current value of a variable from the engine. A media:// value comes back as a URL this frame can load, so the promise waits for that media to load first.

Returns: Promise<any>

mp4e.setVariable(name, value)(name: string, value: any)async

Sets a variable value in the engine. This triggers the rules engine to re-evaluate any rules that depend on this variable.

Returns: Promise<void>

Variable examples

// GET a variable (async - returns Promise)
const cartCount = await mp4e.getVariable('cartCount');
const userName = await mp4e.getVariable('userName');

// SET a variable (async - returns Promise)
await mp4e.setVariable('selectedProduct', object.label);
await mp4e.setVariable('cartCount', cartCount + 1);

// Multiple operations
const [price, quantity] = await Promise.all([
  mp4e.getVariable('unitPrice'),
  mp4e.getVariable('quantity')
]);
const total = price * quantity;
await mp4e.setVariable('total', total);

Video Control

Methods for controlling and querying video playback. Time and state getters are synchronous (updated in real-time via bridge messages). Playback commands are fire-and-forget.

Playback Commands

mp4e.playVideo()

Starts or resumes video playback. Alias: mp4e.play().

mp4e.playVideo();

mp4e.pauseVideo()

Pauses video playback. Alias: mp4e.pause().

mp4e.pauseVideo();

mp4e.seekToTime(seconds)(seconds: number)

Seeks the video to a specific time in seconds.

mp4e.seekToTime(30);

mp4e.goToScene(sceneId)(sceneId: string)

Jumps to a named scene. The engine resolves the scene to a seek time internally.

mp4e.goToScene('chapter-2');

mp4e.setPlaybackPath(groupId)(groupId: string)async

Plays only the scenes of one scene group. Refused with playbackPath.locked when the host has set config.lockPlaybackPath.

Returns: Promise<{ ok: true, state } | { ok: false, error: { code, message, groupId } }>

const res = await mp4e.setPlaybackPath('scenegroup_short');

mp4e.clearPlaybackPath()async

Returns to the full timeline. The playhead is not moved.

Returns: Promise<{ ok: true, state } | { ok: false, error: { code, message, groupId } }>

await mp4e.clearPlaybackPath();

mp4e.getPlaybackPath()async

Current playback path: group id, revision, presentation duration and time, current scene, completed and locked flags, initiator, ranges and scenes. Resolves to null when no path is active.

Returns: Promise<object | null>

const path = await mp4e.getPlaybackPath();

mp4e.getSceneGroups()async

Snapshot of every scene group in metadata order, disabled ones included. Each group carries its scenes (start time, exclusive end time, duration, and, when the video has been analysed, the summary of the chapter that overlaps it most — with the creator's corrections, and none from a chapter the creator hid or kept private), the duration of the group's combined scene ranges, and the scene currently active in that group (null for disabled groups). Same shape as the host's getSceneGroups().

Returns: Promise<object[]>

const groups = await mp4e.getSceneGroups();

mp4e.setMenuItems(items)(items: MenuItem[])async

Contributes items to the player's settings menu. Each item is { id, label, icon?, type?: 'action' | 'toggle' | 'submenu', selected?, checked?, items? }. The list replaces this plugin's previous contribution (pass [] to remove it) and is validated: at most 20 top-level items, two levels deep, ids matching [a-z0-9_-]{1,40}, labels up to 60 characters; invalid entries are dropped. Contributions are per plugin definition: the block is keyed by the plugin's manifest id (namespace:identifier), several live instances of one plugin share it (the last call wins), and in the menu tree the items appear as plugin:<pluginId>:<itemId> (submenu children: plugin:<pluginId>:<itemId>:<childId>) — the ids a controls plugin sees. Items render under the order set in the file's playerMenuConfig.pluginItems[pluginId] (default 50) and are hidden when that entry has enabled: false. When the viewer picks one, the contributing iframe receives a 'menu:select' event with { pluginId, instanceId, itemId, childId? } — the plugin decides what the pick does. Dismissing the menu after a pick of an item you contributed is the engine's job: picking an 'action' in the main list closes the menu, while a 'toggle', a submenu being opened, and a submenu child leave it up, the way the speed and subtitle lists do. Never close the menu from plugin code.

Returns: Promise<{ accepted: number; dropped: number }>

await mp4e.setMenuItems([ { id: 'length', label: 'Length', type: 'submenu', items: [ { id: 'full', label: 'Full', selected: true }, { id: 'short', label: 'Short' }, ] }, ]); mp4e.on('menu:select', ({ itemId, childId }) => { if (itemId === 'length') applyLength(childId); });

mp4e.setVolume(volume)(volume: number)

Sets the video volume. Value should be between 0 (muted) and 1 (full volume).

mp4e.setVolume(0.5);

mp4e.setPlaybackRate(rate)(rate: number)

Sets the playback speed. 1 is normal speed, 0.5 is half speed, 2 is double speed.

mp4e.setPlaybackRate(1.5);

mp4e.setSubtitleTrack(trackId)(trackId: string)

Activates a subtitle track by its ID.

mp4e.setSubtitleTrack('track_es');

mp4e.setAudioTrack(trackId)(trackId: string)

Switches to a different audio track by its ID.

mp4e.setAudioTrack('track_en');

Playback State (Synchronous)

These methods return values synchronously. They are kept up-to-date by the parent player via mp4e-playback-state messages.

mp4e.getCurrentTime()

Gets the current video playback time in seconds. Clamped to the video duration.

Returns: number

const t = mp4e.getCurrentTime(); // e.g., 42.5

mp4e.getCurrentFrame()

Gets the current video frame number.

Returns: number

const frame = mp4e.getCurrentFrame(); // e.g., 1275

mp4e.getDuration()

Gets the total video duration in seconds. Returns 0 if not yet loaded.

Returns: number

const dur = mp4e.getDuration(); // e.g., 120.5

mp4e.isPlaying()

Check if the video is currently playing.

Returns: boolean

if (mp4e.isPlaying()) mp4e.pauseVideo();

mp4e.isBuffering()

Check if the video is currently buffering.

Returns: boolean

if (mp4e.isBuffering()) showSpinner();

mp4e.getVolume()

Gets the current volume level (0 to 1).

Returns: number

const vol = mp4e.getVolume(); // e.g., 0.8

mp4e.isMuted()

Check if the video is currently muted.

Returns: boolean

if (mp4e.isMuted()) muteIcon.classList.add('active');

mp4e.getPlaybackRate()

Gets the current playback speed.

Returns: number

const rate = mp4e.getPlaybackRate(); // e.g., 1.5

mp4e.isFullscreen()

Check if the player is currently in fullscreen mode.

Returns: boolean

if (mp4e.isFullscreen()) showExitButton();

Video control examples

// Get current playback position (synchronous - updated via bridge)
const currentTime = mp4e.getCurrentTime();  // seconds
const currentFrame = mp4e.getCurrentFrame(); // frame number

// Control playback (fire-and-forget)
mp4e.playVideo();
mp4e.pauseVideo();
mp4e.seekToTime(30);       // Seek to 30 seconds
mp4e.goToScene('chapter-2'); // Jump to named scene

// Volume and rate control
mp4e.setVolume(0.5);          // Set volume to 50%
mp4e.setPlaybackRate(1.5);    // Set playback speed to 1.5x

// Query playback state (synchronous)
const duration = mp4e.getDuration();         // Total duration in seconds
const playing = mp4e.isPlaying();            // true if currently playing
const buffering = mp4e.isBuffering();        // true if buffering
const volume = mp4e.getVolume();             // Current volume (0-1)
const muted = mp4e.isMuted();               // true if muted
const rate = mp4e.getPlaybackRate();         // Current playback rate
const fullscreen = mp4e.isFullscreen();      // true if in fullscreen

Track Information

mp4e.getTracks()

Returns all available tracks organized by type.

Returns: { subtitles: Track[], audio: Track[] }

const tracks = mp4e.getTracks(); // { subtitles: [{ id: 't1', label: 'English', language: 'en' }], audio: [...] }

mp4e.getSubtitleTracks()

Returns available subtitle tracks.

Returns: Track[]

const subs = mp4e.getSubtitleTracks(); // [{ id: 't1', label: 'English', language: 'en', source: 'file' }]

mp4e.getAudioTracks()

Returns available audio tracks.

Returns: Track[]

const audio = mp4e.getAudioTracks();

mp4e.getActiveSubtitleTrackIds()

Returns the IDs of currently active subtitle tracks.

Returns: string[]

const activeIds = mp4e.getActiveSubtitleTrackIds();

mp4e.getState()

Returns the current player state including viewer preferences and active tracks.

Returns: object

const state = mp4e.getState();

Events

Subscribe to events from the player to react to changes in real time. Events are forwarded from the parent player to the plugin iframe via postMessage.

mp4e.on(eventName, callback)(eventName: string, callback: (data: any) => void)

Subscribe to a named event. Commonly used for input events (e.g., 'input:keydown', 'input:keyup') forwarded from the parent window, and for engine events forwarded to every plugin iframe: 'scene:enter', 'scene:exit', 'scene:transition', 'path:changed', 'path:advanced', 'path:completed', 'playback:restricted', 'menu:open', 'menu:close', and 'intelligence:changed' (see Intelligence below). 'hitRegions:cleared' is delivered to an overlay plugin whose clickable regions the player cleared while it was hidden, when it is shown again (see setHitRegions). 'menu:open' and 'menu:close' fire whenever the player's settings menu opens or closes, whoever caused it, so a plugin that opened a panel from a menu item can use 'menu:open' to close that panel when the viewer returns to the menu. Handle them idempotently: one transition can deliver more than one event, so they are state signals rather than something to count. 'menu:select' is also delivered here, but only to the plugin that contributed the selected item via setMenuItems. Engine events are also dispatched as DOM CustomEvents on document named 'mp4e:engine:<name>' (e.g. 'mp4e:engine:scene:enter'), with the payload in event.detail.

mp4e.on('input:keydown', (data) => { if (data.key === 'Escape') mp4e.close(); });

mp4e.off(eventName, callback)(eventName: string, callback: (data: any) => void)

Unsubscribe from a named event. Pass the same callback reference used with on().

mp4e.off('input:keydown', myHandler);

mp4e.onVariableChange(callback)(callback: (variables: Record<string, any>, changedIds?: string[]) => void)

Register a callback that fires whenever variables change. The callback receives the full variables map and an optional array of changed variable IDs (undefined means full broadcast, e.g., initial load). Use changedIds to filter and avoid unnecessary work.

mp4e.onVariableChange((variables, changedIds) => { if (changedIds && !changedIds.includes('score')) return; scoreEl.textContent = variables.score; });

mp4e.onConfigUpdate(callback)(callback: (config: object) => void)

Register a callback that fires when the plugin configuration changes at runtime. This happens when template variables in config values are re-interpolated by the engine, or when media:// URLs are resolved.

mp4e.onConfigUpdate((config) => { document.getElementById('title').textContent = config.title; });

mp4e.onSubtitleWordClick(callback)(callback: (wordData: object) => void)

Register a callback that fires when a user clicks on a subtitle word. The callback receives word data including the clicked text and position.

mp4e.onSubtitleWordClick((data) => { console.log('Clicked word:', data.text); });

mp4e.emit(event, data)(event: string, data?: any)

Emits a custom event that the engine can listen for via event rules. Rules configured for 'plugin:<namespace>:<id>:<event>' will fire when this is called.

mp4e.emit('product-selected', { id: 'prod_123', price: 29.99 });

Event examples

// Subscribe to keyboard events forwarded from the parent
mp4e.on('input:keydown', (data) => {
  if (data.key === 'Escape') {
    mp4e.close();
  }
});

// Listen for config updates (when variables change config values)
mp4e.onConfigUpdate((newConfig) => {
  document.getElementById('title').textContent = newConfig.title;
});

// Listen for variable changes
mp4e.onVariableChange((variables, changedIds) => {
  // changedIds is an array of variable IDs that changed,
  // or undefined for full broadcast (e.g., initial load)
  if (changedIds && !changedIds.includes('score')) return;
  document.getElementById('score').textContent = variables.score;
});

// Unsubscribe from a named event
mp4e.off('input:keydown', myHandler);

DOM Custom Events

Most bridge events are also dispatched as DOM CustomEvents on document. For example, variable updates fire mp4e:variableUpdate, playback state fires mp4e:playbackState, time updates fire mp4e:video:timeupdate, and forwarded engine events fire mp4e:engine:<name>(e.g. mp4e:engine:scene:enter). You can use document.addEventListener() as an alternative to the callback API.

Objects

Access detected objects in the video and their interaction statistics.

Boxes are in the video's own pixels, as [x1, y1, x2, y2] (top-left and bottom-right corners), at the current frame — the same coordinates as the pointerX / pointerY variables. bbox is null while an object is present but has no box at this frame.

mp4e.getObject()async

Gets the object this plugin instance belongs to: the object a display was opened for (a click, a hover, or an object being on screen), the object of a group-bound overlay instance, or the object an overlay is bound to. The promise rejects with 'no object for this plugin' for a plugin that belongs to no object.

Returns: Promise<{ id: string, label: string, userLabel: string, data: object, groupIds: string[], bbox: [number, number, number, number] | null }>

try { const obj = await mp4e.getObject(); console.log(obj.id, obj.userLabel || obj.label, obj.data.title); } catch (e) { // not opened for an object }

mp4e.getVisibleObjects()async

Gets every object present at the current time, ordered by id. Each entry has id, label, userLabel, data, groupIds (the groups the object's displays are looked up in) and bbox.

Returns: Promise<{ id: string, label: string, userLabel: string, data: object, groupIds: string[], bbox: [number, number, number, number] | null }[]>

const objects = await mp4e.getVisibleObjects(); const products = objects.filter(o => o.groupIds.includes('products'));

mp4e.getObjectBox(objectId)(objectId: string)async

Gets one object's box at the current frame, and its outline when it was tracked with one. An object not on screen answers { visible: false, bbox: null, polygon: null }.

Returns: Promise<{ visible: boolean, bbox: [number, number, number, number] | null, polygon: number[][] | null }>

const { visible, bbox } = await mp4e.getObjectBox('obj_2'); if (visible) { const [x1, y1, x2, y2] = bbox; console.log('centre', (x1 + x2) / 2, (y1 + y2) / 2); }

mp4e.getVisibleOverlayTexts()async

Gets text content summaries of all currently visible overlays. Each entry includes id, type, and texts array. Useful for accessibility plugins or AI integrations.

Returns: Promise<{ id: string, type: string, texts: string[] }[]>

const texts = await mp4e.getVisibleOverlayTexts();

mp4e.getStats(objectId?)(objectId?: string)async

Gets interaction statistics. Pass an objectId for a single object's stats (hover count, click count, total view time), or omit to get the full stats map for all objects.

Returns: Promise<ObjectStats | Record<string, ObjectStats>>

const stats = await mp4e.getStats('obj_watch_1');

Action Methods

Fire-and-forget methods that trigger actions. These are convenience wrappers around common operations:

mp4e.openUrl(url, target)(url: string, target?: string)

Opens a URL in the host page. http:, https:, mailto: and tel: URLs open, and a relative URL (such as /product/123) opens resolved against the host page, when that gives an http: or https: URL. Any other URL (a javascript:, data: or vbscript: URL) is refused, and the call rejects with an error. Target can be '_blank' (new tab, the default), '_self' (same tab), etc. The host's action rules apply as to the openUrl action: a host can block it (config.sandbox.actions) or require its approval (config.gated).

mp4e.openUrl('https://example.com', '_blank');

Styling & Animation

Style and animate the overlay container that wraps your plugin. These methods automatically target your own overlay — no need to know your overlay ID.

mp4e.setStyle(style, animate?)(style: StyleObject, animate?: AnimateOptions)async

Set visual style on the overlay container. Supports opacity, scale, rotation, borderRadius, boxShadow, filter, backdropFilter, backgroundColor, translateX, and translateY. Optionally animate the transition.

Returns: Promise

mp4e.setStyle({ opacity: 1, scale: 1.05 }, { durationMs: 300, easing: 'ease-out' });

mp4e.resetStyle()async

Reset overlay style back to metadata defaults, removing any styles applied by setStyle().

Returns: Promise

mp4e.resetStyle();

setStyle() / resetStyle() examples

// Fade in with scale
mp4e.setStyle(
  { opacity: 1, scale: 1.05 },
  { durationMs: 300, easing: 'ease-out' }
);

// Reset after delay
setTimeout(() => mp4e.resetStyle(), 2000);

// Apply style without animation
mp4e.setStyle({ borderRadius: '50%', rotation: 45 });

// Combine with wait for timed effects
mp4e.executeActions([
  { type: 'setOverlayStyle', overlayId: 'my-overlay',
    scale: 1.5, durationMs: 300, easing: 'ease-out' },
  { type: 'wait', durationMs: 2000 },
  { type: 'resetOverlayStyle', overlayId: 'my-overlay' }
]);

Available style properties

// Available style properties:
mp4e.setStyle({
  opacity: 0.5,              // 0-1 opacity
  scale: 1.2,                // Scale factor
  rotation: 45,              // Degrees
  borderRadius: '12px',      // CSS border-radius
  boxShadow: '0 4px 12px rgba(0,0,0,0.3)',
  backgroundColor: '#ff000080',
  translateX: '10px',        // Horizontal offset
  translateY: '-5px',        // Vertical offset
  filter: 'blur(2px)',       // CSS filter
  backdropFilter: 'blur(8px)', // CSS backdrop-filter
}, {
  durationMs: 500,           // Animation duration (ms)
  easing: 'ease-out',        // CSS easing function
  delayMs: 100               // Animation delay (ms)
});

Use with wait for timed sequences

Combine setStyle with the wait action in executeActions to create timed animation sequences — for example, scale up on click, wait, then reset.

Positioning & Scale

Methods for responsive layout within the video player's coordinate system. The player applies CSS transforms for scaling, so these helpers ensure your plugin renders correctly at any size.

mp4e.getScaleFactor()

Returns the current viewport scale factor. The player scales overlays based on the video container size relative to the native video dimensions. Use this to adjust pixel-based measurements.

Returns: number

const scale = mp4e.getScaleFactor(); // e.g., 0.75

mp4e.scale(pixels)(pixels: number)

Scales a pixel value by the current scale factor. Shorthand for pixels * getScaleFactor().

Returns: number

const scaledSize = mp4e.scale(16); // e.g., 12 if scale is 0.75

mp4e.getCSSRect(element)(element: HTMLElement)

Returns element dimensions in unscaled CSS coordinates. Use this instead of getBoundingClientRect() when you need to set CSS properties like style.left or style.width. The player applies CSS transforms for scaling, so getBoundingClientRect() returns visual (post-transform) coordinates while CSS properties expect pre-transform values.

Returns: { left: number, top: number, width: number, height: number, right: number, bottom: number }

const rect = mp4e.getCSSRect(progressBar); const leftPx = (percent / 100) * (rect.width - thumbWidth); preview.style.left = leftPx + 'px';

When to use getCSSRect()

Use mp4e.getCSSRect(element) when positioning elements with CSS properties. The standard getBoundingClientRect() returns scaled/visual coordinates, but style.left, style.width, etc. expect unscaled CSS coordinates.

mp4e.requestLayout(opts)

(opts: { position?: 'bottom' | 'fullscreen' | 'center', width?: number | string, height?: number | string, zIndex?: number, pointerEvents?: 'none' | 'auto' })

Ask the player to change this plugin's container. For overlay plugins only pointerEvents ('none' makes the plugin click-through so clicks reach the video and overlays beneath it, 'auto' makes it interactive again) and a finite zIndex are applied. zIndex replaces the z-index the player composes for the overlay (layer order × 1000 plus the overlay's own zIndex), so a plugin that must sit above authored content needs a correspondingly large value; position, width and height are set on the overlay in the Studio and are ignored here. The value holds across the player's per-frame updates until the overlay leaves the render tree or metadata reloads; a hidden overlay is click-through anyway because its container is visibility: hidden. Controls, menu and subtitle plugins may also request position, width and height.

// Fullscreen, transparent overlay plugin: click-through while idle mp4e.requestLayout({ pointerEvents: 'none' }); // ...interactive again when it shows UI mp4e.requestLayout({ pointerEvents: 'auto', zIndex: 140 });

mp4e.setHitRegions(regions)(regions: Array<{ x: number, y: number, width: number, height: number }>)

Make an overlay plugin clickable only where its surfaces are. Pass the rectangles of this plugin's frame that should take the pointer, in the frame's own CSS pixels — the space getBoundingClientRect() measures in inside the plugin, not the unscaled space of mp4e.getCSSRect(). Mouse hover, clicks and touch taps inside a rectangle reach the plugin; everywhere else they pass through to whatever is beneath it — the video, object hotspots, other overlays. An empty array makes the plugin fully click-through. Each call replaces the previous set; the player drops malformed entries, clamps the rest to the frame and keeps at most 16, and the answer says how many it kept. The rectangles apply while the plugin has not taken the whole frame with requestLayout({ pointerEvents: 'auto' }) — then the whole frame is the plugin's, as before — and come back into force after requestLayout({ pointerEvents: 'none' }). They are cleared when the overlay is hidden, removed or reloaded; when a hidden overlay is shown again the plugin receives a 'hitRegions:cleared' event, the moment to declare them again. Anything painted above the plugin (a higher overlay, the player's controls, a dialog) keeps its own clicks, and keyboard focus is unaffected. A touch tap (and, rarely, a mouse click) on a rectangle can reach the plugin as synthetic pointer, mouse and click events rather than touch events, so respond to click (or pointerup), not to touchstart/touchend. Available to overlay plugins; check typeof mp4e.setHitRegions === 'function' and treat a rejected call as unsupported, falling back to requestLayout.

Returns: Promise<{ accepted: number }>

// A panel that stays tappable while the video plays, // without blocking anything around it const panel = document.getElementById('panel'); function declare() { if (typeof mp4e.setHitRegions !== 'function') return; const r = panel.hidden ? null : panel.getBoundingClientRect(); mp4e.setHitRegions(r ? [{ x: r.left, y: r.top, width: r.width, height: r.height }] : []) .catch(() => { /* an older player: use requestLayout instead */ }); } declare(); new ResizeObserver(declare).observe(panel); mp4e.on('hitRegions:cleared', declare);

mp4e.getViewport()

Get the size the video is displayed at, in screen pixels — the picture on screen, not this plugin's own frame — plus the scale factor, whether touch is available, and the orientation. Overlay, display and modal plugins all receive it, and it is kept current through resizes and fullscreen. Until the player first sends it, just after the plugin loads, it reports the plugin frame's own size; onViewportChange fires when the real size arrives.

Returns: { width: number, height: number, scaleFactor: number, hasTouch: boolean, orientation: 'landscape' | 'portrait' }

const vp = mp4e.getViewport(); if (vp.hasTouch) enableTouchControls();

mp4e.onViewportChange(callback)(callback: (viewport: ViewportInfo) => void)

Subscribe to viewport size changes. The callback fires on resize, orientation change, and scale factor updates. Receives the same object shape as getViewport().

mp4e.onViewportChange((vp) => { if (vp.width < 480) switchToCompactLayout(); });

mp4e.offViewportChange(callback)(callback: (viewport: ViewportInfo) => void)

Unsubscribe from viewport changes. Pass the same callback reference used with onViewportChange().

mp4e.offViewportChange(myCallback);

Viewport examples

// The size the video is displayed at, in screen pixels
const viewport = mp4e.getViewport();
// { width: 1280, height: 720, scaleFactor: 0.75, hasTouch: false, orientation: 'landscape' }

// Respond to viewport changes (resize, orientation, scale)
mp4e.onViewportChange((viewport) => {
  if (viewport.orientation === 'portrait') {
    document.body.classList.add('mobile-layout');
  } else {
    document.body.classList.remove('mobile-layout');
  }
});

// Later, unsubscribe
mp4e.offViewportChange(myCallback);

CSS Custom Properties

The bridge automatically sets --vw, --vh, --scale, --s, and --has-touch CSS custom properties on the document root. You can use these in your plugin CSS for responsive layouts without any JavaScript.

Media

Access media assets embedded directly in the MP4 file. Media items use portable media:// URLs that the bridge resolves to usable blob URLs at runtime.

mp4e.getMediaUrl(urlOrId)(urlOrId: string)async

Resolve a media:// URL (or raw media ID) to a usable blob URL. The bridge loads the media from the MP4's custom atoms and hands the data to your plugin, which creates the blob URL inside its own frame, so it works in the isolated iframe (use it in img, video, audio and CSS; it is not a URL the host page can open). Works with both 'media://my-image' and 'my-image' formats. Waits up to 60 seconds for the media before rejecting.

Returns: Promise<string>

const url = await mp4e.getMediaUrl('media://logo'); document.getElementById('logo').src = url;

mp4e.downloadMedia(urlOrId, filename?)(urlOrId: string, filename?: string)async

Download an embedded media asset. Optionally provide a filename; otherwise uses the manifest metadata if available.

Returns: Promise<void>

await mp4e.downloadMedia('media://brochure', 'product-brochure.pdf');

mp4e.listMedia()async

List all media items embedded in the video file. Returns an array of objects with id, type (MIME), purpose, and metadata.

Returns: Promise<MediaItem[]>

const items = await mp4e.listMedia(); items.forEach(item => console.log(item.id, item.type));

mp4e.statMedia(mediaId)(mediaId: string)async

Get metadata for a specific media item without loading the binary data. Returns the media item info or null if not found.

Returns: Promise<MediaItem | null>

const info = await mp4e.statMedia('product-image'); if (info) console.log('Type:', info.type, 'Size:', info.metadata?.size);

Media examples

// Resolve a media:// URL to a usable blob URL
const blobUrl = await mp4e.getMediaUrl('media://product-image-001');
document.getElementById('img').src = blobUrl;

// Download an embedded asset
await mp4e.downloadMedia('media://brochure-pdf', 'brochure.pdf');

// List all embedded media
const items = await mp4e.listMedia();
// [{ id: 'product-image-001', type: 'image/png', purpose: 'asset', metadata: {...} }, ...]

// Get metadata for a specific item without loading the binary
const info = await mp4e.statMedia('product-image-001');
// { id: 'product-image-001', type: 'image/png', purpose: 'asset', metadata: { width: 800, height: 600 } }

Automatic media:// Resolution in HTML

You do not need to call getMediaUrl() manually for media:// URLs used in HTML template attributes (e.g., src="media://my-image"). The bridge automatically resolves these via the template interpolation system.

Storage

Persistent key-value storage for plugins. Storage is sandboxed per plugin ID — each plugin can only access its own data. All methods are asynchronous because they proxy through the parent window. Because a plugin iframe is isolated, this is the plugin's storage: the browser's localStorage, sessionStorage and cookies are not available to it.

mp4e.storage.get(key)(key: string)async

Get a stored value by key. Returns the deserialized value, or undefined if the key does not exist.

Returns: Promise<any>

const theme = await mp4e.storage.get('theme');

mp4e.storage.set(key, value)(key: string, value: any)async

Store a value. The value is serialized and stored in the parent window's localStorage, namespaced to this plugin ID.

Returns: Promise<void>

await mp4e.storage.set('theme', 'dark');

mp4e.storage.remove(key)(key: string)async

Remove a stored value by key.

Returns: Promise<void>

await mp4e.storage.remove('theme');

mp4e.storage.clear()async

Clear all stored values for this plugin. Does not affect other plugins' storage.

Returns: Promise<void>

await mp4e.storage.clear();

mp4e.storage.keys()async

List all stored keys for this plugin.

Returns: Promise<string[]>

const keys = await mp4e.storage.keys(); // ['theme', 'fontSize']

Storage examples

// Store user preferences
await mp4e.storage.set('theme', 'dark');
await mp4e.storage.set('fontSize', 14);

// Retrieve stored values
const theme = await mp4e.storage.get('theme');   // 'dark'
const size = await mp4e.storage.get('fontSize');  // 14

// List all keys
const keys = await mp4e.storage.keys();  // ['theme', 'fontSize']

// Remove a single key
await mp4e.storage.remove('theme');

// Clear all plugin storage
await mp4e.storage.clear();

Viewer Privacy Preferences

When the viewer has enabled the limitStorage preference, storage automatically falls back to sessionStorage instead of localStorage. This means data will not persist across sessions. Your plugin should handle this gracefully.

Network

Plugin iframes are isolated, so a direct fetch() runs from an opaque origin that most servers reject, and a host that restricts plugin networking blocks direct fetch() and XMLHttpRequest calls with a Content Security Policy. Use mp4e.fetch() to proxy network requests through the parent window.

mp4e.fetch(url, options?)(url: string, options?: { method?: string, headers?: object, body?: any })async

CSP-proxied fetch. Performs a network request through the parent window, which enforces domain allowlists from the plugin security configuration. The host can gate requests via the onNetworkGate callback for domains not in the allowlist.

Returns: Promise<{ ok: boolean, status: number, headers: object, data: any }>

const res = await mp4e.fetch('https://api.example.com/data'); if (res.ok) console.log(res.data);

Fetch examples

// GET request
const response = await mp4e.fetch('https://api.example.com/products');
if (response.ok) {
  const products = response.data;
  console.log('Products:', products);
}

// POST request with body
const result = await mp4e.fetch('https://api.example.com/cart', {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({ productId: 'prod_123', quantity: 1 })
});

if (result.ok) {
  mp4e.executeActions([
    { type: 'showNotification', message: 'Added to cart!', type: 'success' }
  ]);
}

What the proxy does

The request is made by the host page, so the server must allow the host page's origin (CORS). It is sent without cookies or other credentials — authenticate with a token in the URL or a header. The response is JSON or text (no binary data). A call that takes longer than about 5 seconds rejects while the request itself may still complete, so make order-like endpoints idempotent and have them acknowledge quickly.

Domain Allowlist

When the host restricts plugin networking, it controls which domains plugins can access. Requests to domains not in the allowlist will be routed through the onNetworkGate callback for approval. If no callback is configured, ungated requests will be rejected.

Intelligence

A video analysed with Make interactive carries its Intelligence: chapters, key moments, the questions it answers, the things and people in it and what happens in each scene, all with times. A plugin can read the viewer's version of it — the creator's corrections applied, anything the creator hid or kept private left out — and the subtitles as a timed transcript. The player only serves it; what the plugin does with it (search, a chapter list, a quiz) is up to the plugin. It works offline, inside the player, with no server.

A plugin reads by scope. It receives a scope only when all three agree: its manifest requests the scope (permissions.intelligence, see Creating Plugins), the creator grants it (chosen when the plugin is installed in the Studio, and changeable in the plugin's settings), and the host allows it (config.permissions.intelligence). A plugin installed without that choice gets the scopes marked "yes" below that it requests.

Grants are per plugin

The host's allow-list applies to every plugin, and the creator's per-plugin grants are enforced: plugins run isolated from each other by default, so one plugin cannot read through another plugin's grant. Only a host that raises plugins to the trusted level (trustContent or a pluginSecurity rule) lets them reach each other's windows.

Manifest: the scopes a plugin asks for

{
  "permissions": {
    "intelligence": ["chapters", "moments", "transcript"]
  }
}

Scope	Keys in result	Granted unless the creator says otherwise
`summary`	`summary { oneLine, paragraph, detailed }, topics, language, contentType`	yes
`chapters`	`chapters[] { id, startTime, endTime, title, summary }`	yes
`moments`	`keyMoments[] { id, time, kind, title, description }`	yes
`questions`	`questionsAnswered[] { id, question, time }`	yes
`things`	`entities[] { id, name, category, lens, attributes, description, windows }`	yes
`scenes`	`scenes[] { id, startTime, endTime, title, description }`	yes
`people`	`people[] { id, name, role, description, speaks, windows }`	no — only when the creator grants it
`transcript`	`transcript[] { start, end, text, speaker? }`	no — only when the creator grants it

All times are seconds of the video. windows is a list of { start, end } ranges where a thing or person is seen or heard. A thing's category is a word or two (tool, ingredient, …); lens and attributes are present for things found through a lens the creator chose. transcript holds the lines of the subtitle track the viewer is watching, or, with subtitles off, of the creator's default track; speaker is the speaker's name when the track has one. Subtitles that come from the video file's own text track are not served. Nothing else of the analysis leaves the player.

mp4e.getIntelligence(scopes?)(scopes?: string[])async

Reads the viewer's Intelligence by scope. scopes omitted = every scope the manifest requests. result holds the keys of each scope that was served (an empty list stays an empty list); withheld lists every asked-for scope that was not served, as { scope, reason }, so a plugin can explain itself instead of failing silently. A scopes argument that is not a list of names rejects the call. Available to overlay plugins placed as one overlay on the video (rather than expanded over an object group), to the display plugins the player opens on object hover and click, and to modal plugins.

Returns: Promise<{ result: object, withheld: { scope: string, reason: string }[] }>

withheld reason	Meaning
`unknown-scope`	Not one of the eight scope names.
`not-requested`	The plugin's manifest does not list the scope in permissions.intelligence.
`not-granted`	The creator has not granted it. people and transcript are granted only when the creator ticks them.
`host`	The host's config.permissions.intelligence leaves it out.
`absent`	Allowed, but this video has none: it was not analysed, or it has no subtitles for transcript.

Reasons are checked in that order, so a scope the manifest does not request is reported as not-requested even when the video has none.

Reading the Intelligence

// Read what the plugin may read (scopes omitted = every scope the manifest requests)
async function load() {
  const { result, withheld } = await mp4e.getIntelligence();

  for (const chapter of result.chapters ?? []) {
    addRow(chapter.title, chapter.startTime);
  }
  for (const cue of result.transcript ?? []) {
    indexLine(cue.text, cue.start);
  }

  // Explain what is missing instead of failing silently
  for (const { scope, reason } of withheld) {
    if (scope === 'transcript' && reason === 'not-granted') {
      showNote('Ask the creator to allow the transcript');
    }
  }
}

load();

// The creator edited the analysis, or the viewer switched subtitle track
mp4e.on('intelligence:changed', ({ reason }) => load());

// Ask for some scopes only
const { result } = await mp4e.getIntelligence(['chapters', 'moments']);

What getIntelligence() resolves to

{
  "result": {
    "chapters": [
      { "id": "ch_1", "startTime": 0, "endTime": 42.5, "title": "Making the dough", "summary": "Flour, egg and water become a soft dough." }
    ],
    "keyMoments": [
      { "id": "km_1", "time": 61.2, "kind": "step", "title": "Adding the butter", "description": "Butter goes into the filling." }
    ],
    "transcript": [
      { "start": 0.4, "end": 3.1, "text": "Today we're making pierogi.", "speaker": "Grandma" }
    ]
  },
  "withheld": [
    { "scope": "people", "reason": "not-granted" }
  ]
}

intelligence:changed

Fires in the plugin (mp4e.on('intelligence:changed', …)) when what getIntelligence() returns may have changed: { reason: 'document' } when the video's document is reloaded with different Intelligence or subtitles (for example after the creator edits the analysis in the Studio), and { reason: 'transcript' } when the viewer's subtitle choice changes the track the transcript comes from. Read once on load, then again on this event.

mp4e.getViewerPreferences()async

The viewer's privacy and accessibility choices that concern a plugin: doNotTrack, reducedMotion and highContrast. No permission is needed. Use doNotTrack to leave personal input (what the viewer typed) out of the events you emit.

Returns: Promise<{ doNotTrack: boolean, reducedMotion: boolean, highContrast: boolean }>

Viewer preferences

const prefs = await mp4e.getViewerPreferences();
// { doNotTrack: false, reducedMotion: true, highContrast: false }

if (prefs.reducedMotion) document.body.classList.add('no-motion');

// Leave what the viewer typed out of your event when they asked not to be tracked
mp4e.emit('search:selected', prefs.doNotTrack ? { time } : { query, time });

Methods for custom controls plugins to access and interact with the player menu system, chapter markers, and thumbnail previews.

mp4e.getMenuConfig()

Get the player menu configuration from metadata. Returns the menu structure including system items, plugin items, custom items, and settings.

Returns: object | null

const menu = mp4e.getMenuConfig();

mp4e.executeMenuItemActions(item)(item: { actions: Action[] })

Execute the configured actions for a menu item. Useful when building custom menu UIs that need to trigger the same actions as the built-in player menu.

mp4e.executeMenuItemActions(menuItem);

mp4e.getVisibleMenuItems()async

Get currently visible custom menu items from the menu configuration. Filters out disabled items.

Returns: Promise<MenuItem[]>

const items = await mp4e.getVisibleMenuItems();

mp4e.getMarkers()

Get timeline chapter markers. Returns an array of marker objects with time, label, and optional color.

Returns: Marker[]

const markers = mp4e.getMarkers(); // [{ time: 0, label: 'Intro' }, { time: 30, label: 'Chapter 1' }]

mp4e.getCurrentMarker()

Get the chapter marker for the current playback position.

Returns: Marker | null

const marker = mp4e.getCurrentMarker(); if (marker) chapterLabel.textContent = marker.label;

mp4e.getMarkerAtTime(time)(time: number)

Get the chapter marker active at a specific time. Returns the last marker whose time is less than or equal to the given time.

Returns: Marker | null

const marker = mp4e.getMarkerAtTime(45.0);

mp4e.getMarkerSettings()

Get marker display settings from metadata (colors, visibility, positioning).

Returns: object | null

const settings = mp4e.getMarkerSettings();

mp4e.getThumbnails()

Get thumbnail strip configuration including enabled state, interval, dimensions, and sprite column count.

Returns: object | null

const thumbs = mp4e.getThumbnails(); if (thumbs?.enabled) console.log('Thumbnail interval:', thumbs.interval);

mp4e.getThumbnailSpriteUrl()

Get the URL of the thumbnail sprite sheet image.

Returns: string | null

const spriteUrl = mp4e.getThumbnailSpriteUrl();

mp4e.getThumbnailPosition(time)(time: number)

Calculate the sprite sheet position for a thumbnail at a specific time. Returns the backgroundPosition CSS value, dimensions, and grid coordinates.

Returns: { backgroundPosition: string, width: number, height: number, index: number, col: number, row: number } | null

const pos = mp4e.getThumbnailPosition(30); if (pos) { preview.style.backgroundPosition = pos.backgroundPosition; preview.style.width = pos.width + 'px'; preview.style.height = pos.height + 'px'; }

mp4e.enableThumbnailPreview(progressBarElement)(progressBarElement: HTMLElement)

Attach thumbnail hover detection to a progress bar element. On hover, the bridge sends thumbnail position data to the parent player for rendering the preview outside the iframe (to avoid clipping). Supports both mouse and touch events. Only needs to be called once per element. Controls plugins can also add the data-mp4e-timeline attribute to their progress bar for automatic detection.

const bar = document.getElementById('progressBar'); mp4e.enableThumbnailPreview(bar);

Plugin-to-Plugin Communication

Plugins can communicate with each other through a request-response action system. One plugin registers action handlers, and other plugins call those actions by the target plugin's ID. This is the primary mechanism for service plugins (e.g., shopping cart, analytics) to expose functionality to other plugins.

mp4e.onAction(actionId, handler)(actionId: string, handler: (config: object) => any | Promise<any>)

Register an action handler that other plugins can call. The handler receives the config object passed by the caller and can return a value (or a Promise). Multiple actions can be registered per plugin.

mp4e.onAction('addItem', (config) => { cart.push({ name: config.productName, price: config.price }); return { success: true, itemCount: cart.length }; });

mp4e.callAction(targetPluginId, actionId, config?)(targetPluginId: string, actionId: string, config?: object)async

Call another plugin's registered action. The request is routed through the parent to the target plugin's iframe. Returns the handler's return value.

Returns: Promise<any>

const result = await mp4e.callAction('acme:cart', 'addItem', { productName: 'Blue T-Shirt', price: 29.99 });

mp4e.callPluginAction(targetPluginId, actionId, config?)(targetPluginId: string, actionId: string, config?: object)async

Alias for callAction(). Matches the naming convention used by the service plugin system.

Returns: Promise<any>

const result = await mp4e.callPluginAction('analytics', 'track', { event: 'view' });

Plugin-to-plugin communication

// A cart plugin (key acme:cart, declaring the variable itemCount) registers an action handler
mp4e.onAction('addItem', (config) => {
  cart.push({ name: config.productName, price: config.price });
  mp4e.setVariable('acme:cart:itemCount', cart.length);
  return { success: true };
});

// Another plugin calls the cart's action by the cart's key
const result = await mp4e.callAction('acme:cart', 'addItem', {
  productName: 'Blue T-Shirt',
  price: 29.99
});

Calling a plugin by its type

The target can be a plugin's type (namespace:identifier, e.g. mp4e:search) as well as an id. Called by type, the action reaches every overlay of that plugin on the video once, each in its own frame, and the call resolves with the first one's result. The pluginAction action and plugin-declared shortcuts address plugins the same way.

Service Plugins

Service plugins (type: 'service') run in the background and are the ideal place to register action handlers, and can emit events that other plugins' rules listen for. A plugin of any type can declare variables in its manifest: installing it creates them named after the plugin's key (e.g. mp4e:cart:items for the Cart), and uninstalling it removes them.

executeActions()

The most powerful method - execute any action or sequence of actions. Actions are processed in order by the engine. This routes through the WASM engine, which means all actions trigger proper rule evaluation and state updates.

mp4e.executeActions(actions)(actions: Action[])

Executes an array of actions. Actions are processed in order by the engine. Each action is a plain object with a 'type' field and type-specific parameters.

mp4e.executeActions([{ type: 'pause' }, { type: 'showNotification', message: 'Hello!' }]);

executeActions() overview

// Execute one or more actions
mp4e.executeActions([
  // Playback actions
  { type: 'pause' },
  { type: 'seek', time: 45 },
  { type: 'setPlaybackRate', rate: 1.5 },
  { type: 'setVolume', volume: 0.8 },

  // Navigation
  { type: 'openUrl', url: 'https://example.com', target: '_blank' },
  { type: 'goToScene', sceneId: 'ending' },

  // UI actions
  { type: 'showOverlay', overlayId: 'promo-banner' },
  { type: 'hideOverlay', overlayId: 'intro-tooltip' },
  { type: 'showNotification', message: 'Added to cart!', type: 'success' },
  { type: 'enterFullscreen' },

  // Variable actions
  { type: 'setVariable', name: 'cartCount', value: 5 },
  { type: 'setVariable', name: 'total', operation: 'add', value: 99.99 },

  // Custom plugin actions
  {
    type: 'showCustomModal',
    pluginId: 'plugin_checkout_form',
    config: {
      title: 'Complete Purchase',
      productName: object.label,
      price: object.data.price
    },
    pauseVideo: true
  },

  // Analytics
  { type: 'trackEvent', event: 'add_to_cart', data: { product: object.id } },

  // API calls
  {
    type: 'api',
    url: 'https://api.example.com/cart',
    method: 'POST',
    body: { productId: object.data.sku },
    storeResultIn: 'apiResponse'
  }
]);

Playback Actions

Playback actions

// Control video playback
mp4e.executeActions([
  { type: 'play' },                    // Start playback
  { type: 'pause' },                   // Pause playback
  { type: 'seek', time: 30 },          // Seek to 30 seconds
  { type: 'seekRelative', offset: -5 }, // Seek back 5 seconds
  { type: 'setPlaybackRate', rate: 2 }, // 2x speed
  { type: 'setVolume', volume: 0.5 },  // 50% volume
  { type: 'mute' },                     // Mute audio
  { type: 'unmute' },                   // Unmute audio
  { type: 'setPlaybackStart', time: 10 }, // Set start point
  { type: 'setPlaybackEnd', time: 60 },   // Set end point
]);

Navigation Actions

Navigation actions

// Navigation and URL handling
mp4e.executeActions([
  { type: 'goToScene', sceneId: 'chapter-3' },
  { type: 'openUrl', url: 'https://example.com', target: '_blank' },
  { type: 'openUrl', url: '/product/123', target: '_self' },
]);

UI Actions

UI actions

// UI control
mp4e.executeActions([
  { type: 'showOverlay', overlayId: 'overlay_banner' },
  { type: 'hideOverlay', overlayId: 'overlay_intro' },
  { type: 'toggleOverlay', overlayId: 'overlay_menu' },
  { type: 'activateLayer', layerId: 'layer_products' },
  { type: 'deactivateLayer', layerId: 'layer_tutorial' },
  { type: 'highlightObject', objectId: 'obj_chair_1', duration: 3000 },
  { type: 'showNotification', message: 'Welcome!', type: 'info', duration: 5000 },
  { type: 'enterFullscreen' },
  { type: 'exitFullscreen' },
]);

Variable Actions

Variable actions

// Variable manipulation
mp4e.executeActions([
  // Set value directly
  { type: 'setVariable', name: 'score', value: 100 },

  // Mathematical operations
  { type: 'setVariable', name: 'score', operation: 'add', value: 10 },
  { type: 'setVariable', name: 'score', operation: 'subtract', value: 5 },
  { type: 'setVariable', name: 'score', operation: 'multiply', value: 2 },
  { type: 'setVariable', name: 'score', operation: 'divide', value: 4 },

  // String operations
  { type: 'setVariable', name: 'message', operation: 'append', value: ' World' },

  // Timer control
  { type: 'timerControl', name: 'watchTimer', action: 'start' },
  { type: 'timerControl', name: 'watchTimer', action: 'pause' },
  { type: 'timerControl', name: 'watchTimer', action: 'reset' },

  // Array operations
  { type: 'arrayControl', name: 'cart', action: 'push', value: 'item-1' },
  { type: 'arrayControl', name: 'cart', action: 'pop' },
  { type: 'arrayControl', name: 'cart', action: 'remove', value: 'item-1' },
  { type: 'arrayControl', name: 'cart', action: 'clear' },

  // Custom events
  { type: 'emitEvent', event: 'product-selected', data: { id: 'prod_123' } },
]);

Dialog Actions

Dialog actions

// Built-in dialogs
mp4e.executeActions([
  // Simple alert
  { type: 'alert', message: 'Video complete!' },

  // Confirmation dialog
  {
    type: 'confirm',
    message: 'Add to cart?',
    onConfirm: [
      { type: 'setVariable', name: 'cartCount', operation: 'add', value: 1 },
      { type: 'showNotification', message: 'Added!', type: 'success' }
    ],
    onCancel: [
      { type: 'showNotification', message: 'Cancelled', type: 'info' }
    ]
  },

  // Prompt for input
  {
    type: 'prompt',
    message: 'Enter quantity:',
    inputType: 'number',
    defaultValue: '1',
    storeResultIn: 'userQuantity'
  }
]);

Plugin Actions

Custom plugin actions

// Custom plugin modal
mp4e.executeActions([
  {
    type: 'showCustomModal',
    pluginId: 'plugin_checkout',  // Your plugin ID from the editor
    config: {
      title: 'Complete Purchase',
      productName: object.label,
      productImage: object.data.imageUrl,
      price: object.data.price,
      currency: 'USD'
    },
    showBackdrop: true,
    pauseVideo: true   // Pause video while modal is open
  }
]);

// Custom overlay
mp4e.executeActions([
  {
    type: 'showCustomOverlay',
    pluginId: 'plugin_banner',
    position: { x: 50, y: 10 },  // Percentage position
    config: { text: 'Limited time offer!' }
  },
  { type: 'hideCustomOverlay', pluginId: 'plugin_banner' }
]);

Analytics & API Actions

Analytics and API actions

// Analytics and API
mp4e.executeActions([
  // Track custom event
  {
    type: 'trackEvent',
    event: 'product_view',
    data: {
      productId: object.data.sku,
      productName: object.label,
      category: 'electronics'
    }
  },

  // Make API call
  {
    type: 'api',
    url: 'https://api.example.com/track',
    method: 'POST',
    headers: { 'Authorization': 'Bearer token123' },
    body: {
      event: 'video_interaction',
      objectId: object.id
    },
    storeResultIn: 'apiResponse'
  },

  // Debug logging
  { type: 'log', message: 'User clicked: ' + object.label },

  // Wait before next action
  { type: 'wait', durationMs: 1000 }  // 1 second
]);

Complete Example

A full example showing a product card plugin with add-to-cart and buy-now functionality:

Complete product card plugin

1// Complete example: Interactive product card plugin
2
3// ===== HTML (in plugin editor) =====
4<div class="product-card">
5  <img id="product-image" src="{{object.data.imageUrl}}" />
6  <h3 id="product-name">{{object.userLabel}}</h3>
7  <p id="product-price"></p>
8  <button id="add-to-cart">Add to Cart</button>
9  <button id="buy-now">Buy Now</button>
10</div>
11
12// ===== CSS =====
13.product-card {
14  background: white;
15  border-radius: 12px;
16  padding: 16px;
17  box-shadow: 0 4px 12px rgba(0,0,0,0.15);
18}
19#product-image { width: 100%; border-radius: 8px; }
20button {
21  padding: 8px 16px;
22  border-radius: 6px;
23  cursor: pointer;
24  margin-top: 8px;
25}
26#add-to-cart { background: #3b82f6; color: white; }
27#buy-now { background: #10b981; color: white; }
28
29// ===== JavaScript =====
30(async function() {
31  // Get config and format price
32  const config = mp4e.getConfig();
33  const price = object.data?.price || 0;
34  const formatted = new Intl.NumberFormat('en-US', {
35    style: 'currency',
36    currency: config.currency || 'USD'
37  }).format(price);
38  document.getElementById('product-price').textContent = formatted;
39
40  // Add to Cart button
41  document.getElementById('add-to-cart').addEventListener('click', async () => {
42    // Get current cart count
43    const cartCount = await mp4e.getVariable('cartCount') || 0;
44
45    // Update cart
46    await mp4e.setVariable('selectedProduct', object.userLabel || object.label);
47    await mp4e.setVariable('selectedPrice', price);
48    await mp4e.setVariable('cartCount', cartCount + 1);
49
50    // Show notification
51    mp4e.executeActions([
52      { type: 'showNotification', message: 'Added to cart!', type: 'success' },
53      { type: 'trackEvent', event: 'add_to_cart', data: { product: object.id, price } }
54    ]);
55
56    // Close this plugin
57    mp4e.close();
58  });
59
60  // Buy Now button
61  document.getElementById('buy-now').addEventListener('click', () => {
62    mp4e.executeActions([
63      { type: 'pause' },
64      {
65        type: 'showCustomModal',
66        pluginId: 'plugin_checkout_form',
67        config: {
68          productName: object.userLabel || object.label,
69          productImage: object.data?.imageUrl,
70          price: price,
71          sku: object.data?.sku
72        },
73        pauseVideo: true,
74        showBackdrop: true
75      },
76      { type: 'trackEvent', event: 'buy_now_click', data: { product: object.id } }
77    ]);
78  });
79})();

Creating Plugins →Template Syntax →All Actions →

Plugin API Reference

Overview

Global Object

Lifecycle

Sync Methods

Async Methods

Video Control

Playback Commands

Playback State (Synchronous)

Track Information

Events

Objects

Action Methods

Styling & Animation

Positioning & Scale

Media

Storage

Network

Intelligence

Menu & UI

Plugin-to-Plugin Communication

executeActions()

Playback Actions

Navigation Actions

UI Actions

Variable Actions

Dialog Actions

Plugin Actions

Analytics & API Actions

Complete Example

Related