MP4E — Interactive video that travels in the file

Getting Started

Custom plugins let you create any interactive content using HTML, CSS, and JavaScript. They run in a sandboxed iframe and communicate with the player through the mp4e bridge API.

Complete plugin example — Countdown Timer

{
  id: 'acme:countdown',
  name: 'Countdown Timer',
  type: 'overlay',
  description: 'Displays a countdown to a specific date',

  configSchema: {
    type: 'object',
    properties: {
      targetDate: { type: 'string', format: 'date-time', title: 'Target Date' },
      title: { type: 'string', title: 'Title', default: 'Coming Soon' },
      showSeconds: { type: 'boolean', title: 'Show Seconds', default: true }
    },
    required: ['targetDate']
  },

  emits: {
    complete: { label: 'Countdown Complete', description: 'Fired when countdown reaches zero' }
  },

  content: {
    html: `
      <div class="countdown">
        <h3 class="title">{{title}}</h3>
        <div class="time-display">
          <div class="unit"><span class="val" id="days">00</span><span class="lbl">Days</span></div>
          <div class="unit"><span class="val" id="hours">00</span><span class="lbl">Hours</span></div>
          <div class="unit"><span class="val" id="minutes">00</span><span class="lbl">Minutes</span></div>
          <div class="unit seconds" id="sec-wrap"><span class="val" id="seconds">00</span><span class="lbl">Seconds</span></div>
        </div>
      </div>`,
    css: `
      .countdown { text-align:center; padding:24px; background:linear-gradient(135deg,#667eea,#764ba2); border-radius:16px; color:#fff; font-family:system-ui,sans-serif; }
      .title { margin:0 0 16px; font-size:18px; font-weight:600; opacity:.9; }
      .time-display { display:flex; gap:16px; justify-content:center; }
      .unit { display:flex; flex-direction:column; align-items:center; }
      .val { font-size:36px; font-weight:700; line-height:1; font-variant-numeric:tabular-nums; }
      .lbl { font-size:12px; text-transform:uppercase; opacity:.7; margin-top:4px; }
      .seconds.hidden { display:none; }`,
    script: `
      var config = mp4e.getConfig();
      var targetDate = new Date(config.targetDate);
      if (!config.showSeconds) {
        document.getElementById('sec-wrap').classList.add('hidden');
      }
      function tick() {
        var diff = targetDate - new Date();
        if (diff <= 0) {
          mp4e.emit('complete');
          document.querySelector('.time-display').innerHTML = '<span class="val">Done!</span>';
          clearInterval(iv);
          return;
        }
        document.getElementById('days').textContent = String(Math.floor(diff/864e5)).padStart(2,'0');
        document.getElementById('hours').textContent = String(Math.floor(diff%864e5/36e5)).padStart(2,'0');
        document.getElementById('minutes').textContent = String(Math.floor(diff%36e5/6e4)).padStart(2,'0');
        document.getElementById('seconds').textContent = String(Math.floor(diff%6e4/1e3)).padStart(2,'0');
      }
      tick();
      var iv = setInterval(tick, 1000);`
  }
}

Using the Plugin Editor

The easiest way to create plugins is through the Studio's built-in Plugin Editor:

Create an overlay in the Overlays panel and select "Custom Plugin" as the type
Open the Plugin Editor — the full-screen editor appears with a two-panel layout
Write your HTML in the HTML tab, using {{variable}} syntax for dynamic values
Style with CSS in the CSS tab — use var(--s) for responsive scaling
Add interactivity in the Script tab using the mp4e bridge API
Define inputs in the Inputs tab — creates form fields in the overlay settings
Define events in the Emits tab — makes events available as rule triggers
Preview live in the Preview tab — changes reflect instantly

Two-Panel Workflow

Hold Cmd/Ctrl and click any tab to open it in the right panel. A common workflow: HTML on the left, Preview on the right. Or Script on the left, API Docs on the right.

Editor Tabs

Tab	Purpose	Maps To
HTML	Plugin markup with template interpolation	content.html
CSS	Scoped styles, responsive via CSS custom properties	content.css
Script	JavaScript — use mp4e API here, NOT template syntax	content.script
Inputs	JSON Schema for configuration fields (generates form UI)	configSchema
Outputs	Variables the plugin writes to	outputs
Emits	Custom events with data schemas (become rule triggers)	emits
Actions	Actions other plugins/rules can call on this plugin	actions
Variables	Auto-created project variables on install	variables
Display	Position, animation, auto-close settings	displaySettings
Settings	Name, version, type, category, author, license, tags	name, version, etc.
Preview	Live iframe preview — updates as you type	UI only
API Docs	Quick reference for the mp4e bridge API	UI only

Plugin Structure

A plugin has three content sections plus metadata:

HTML

Structure and content. Supports {{variable}} template interpolation for dynamic values from config and project variables.

CSS

Full CSS including animations, media queries, and {{variable}} interpolation via custom properties.

JavaScript

Logic and interactivity via the mp4e bridge API. Template syntax does NOT work here.

HTML Section

HTML templates support double-curly-brace syntax for interpolation:

HTML template syntax

<div class="product-card">
  <!-- Simple interpolation from config -->
  <h2>{{title}}</h2>

  <!-- Object data (for object-display plugins) -->
  <p>{{object.userLabel}}</p>
  <span class="price">${{object.data.price}}</span>

  <!-- Project variables -->
  <span class="cart-badge">{{cartCount}} items</span>

  <!-- HTML escaping (default) vs unescaped -->
  <p>{{description}}</p>           <!-- Escaped -->
  <div>{{{richContent}}}</div>     <!-- Unescaped HTML -->
</div>

CSS Section

Full CSS support. Use var(--s) for scale-responsive sizing:

CSS styling

/* Scale-responsive sizing via CSS custom property */
.card {
  padding: calc(16px * var(--s, 1));
  font-size: calc(14px * var(--s, 1));
  border-radius: calc(8px * var(--s, 1));
}

/* Config value interpolation via custom properties */
:root {
  --accent: {{accentColor}};
}
.button { background: var(--accent); }

/* Animations */
@keyframes fadeIn {
  from { opacity: 0; transform: translateY(10px); }
  to { opacity: 1; transform: translateY(0); }
}
.animated { animation: fadeIn 0.3s ease-out; }

/* Responsive */
@media (max-width: 480px) {
  .hide-mobile { display: none; }
}

JavaScript Section

Template Syntax Does NOT Work in JavaScript

{{variable}} interpolation only works in HTML and CSS. In JavaScript, use the mp4e API:

CORRECT:var config = mp4e.getConfig(); var title = config.title;

WRONG:var title = "{{title}}";

JavaScript — using the mp4e API

// Access configuration (sync)
var config = mp4e.getConfig();
var title = config.title;
var price = config.price;

// Access bound object data (for object-display/group-bound plugins)
var obj = window.object;  // { id, label, userLabel, data, position }

// DOM manipulation
document.getElementById('title').textContent = title;

// Event listeners
document.getElementById('buy-btn').addEventListener('click', function() {
  // Emit custom event (triggers rules)
  mp4e.emit('purchased', { productId: config.productId, price: price });

  // Execute player actions
  mp4e.executeActions([
    { type: 'setVariable', variableId: 'cartCount', operation: 'increment', value: 1 },
    { type: 'showNotification', message: 'Added to cart!', variant: 'success' }
  ]);
});

// Listen for variable changes
mp4e.onVariableChange(function(variables, changedIds) {
  // changedIds is an array of changed variable IDs, or undefined for full broadcast
  if (!changedIds || changedIds.includes('cartCount')) {
    document.getElementById('cart-badge').textContent = variables.cartCount + ' items';
  }
});

// Listen for config updates (when overlay config changes in studio)
mp4e.onConfigUpdate(function(newConfig) {
  document.getElementById('title').textContent = newConfig.title;
});

// Video events
mp4e.on('mp4e:video:timeupdate', function(state) {
  console.log('Time:', state.currentTime);
});

// Async operations
mp4e.getVariable('score').then(function(score) {
  mp4e.setVariable('score', score + 10);
});

// Cleanup intervals when plugin is destroyed
var iv = setInterval(tick, 1000);
// Note: intervals are automatically cleared when iframe is destroyed

Configuration Schema (Inputs)

Define configurable properties using JSON Schema in the Inputs tab. The Studio automatically generates form UI based on your schema:

Configuration schema

{
  configSchema: {
    type: 'object',
    properties: {
      // String — text input
      title: {
        type: 'string',
        title: 'Title',
        description: 'Product title',
        default: 'Product Name'
      },

      // Number — numeric input with constraints
      price: {
        type: 'number',
        title: 'Price',
        minimum: 0,
        maximum: 999999,
        default: 0
      },

      // Enum — dropdown select
      currency: {
        type: 'string',
        title: 'Currency',
        enum: ['USD', 'EUR', 'GBP', 'JPY'],
        default: 'USD'
      },

      // Boolean — checkbox/toggle
      showRating: {
        type: 'boolean',
        title: 'Show Rating',
        default: true
      },

      // Color — color picker
      accentColor: {
        type: 'string',
        format: 'color',
        title: 'Accent Color',
        default: '#3b82f6'
      },

      // Variable reference — dropdown of project variables
      scoreVariable: {
        type: 'string',
        title: 'Score Variable',
        'x-ref': 'variable'  // Shows variable picker in Studio
      },

      // Array
      tags: {
        type: 'array',
        title: 'Tags',
        items: { type: 'string' },
        default: []
      },

      // Nested object
      styling: {
        type: 'object',
        title: 'Styling',
        properties: {
          borderRadius: { type: 'number', title: 'Border Radius', default: 8 },
          shadow: { type: 'boolean', title: 'Shadow', default: true }
        }
      }
    },
    required: ['title', 'price']
  }
}

Config values are accessible in HTML via {{fieldName}} and in JavaScript via mp4e.getConfig().fieldName.

Sizing

The optional sizing block tells the Studio what shape a new overlay for this plugin should start out as. The author can always move and resize it afterwards — these are starting values, not constraints.

Sizing hints

{
  sizing: {
    // Start the overlay at position 0,0 filling the whole video (100% x 100%),
    // whatever the video's resolution. For plugins that place their own content
    // inside the frame — full-screen cards, chapter panels. Takes precedence
    // over preferredWidth/preferredHeight.
    fullFrame: false,

    // Starting size in video-native pixels. 'auto' is not content measurement:
    // it resolves to 400 x 300.
    preferredWidth: 400,
    preferredHeight: 300,

    // 'auto' lets the plugin's content determine the overlay's size;
    // 'fixed' uses the size above
    defaultSizeMode: 'fixed',

    // The frame this plugin was designed against. Documentation for whoever
    // reads the manifest — nothing in the runtime reads these.
    baseWidth: 1920,
    baseHeight: 1080
  }
}

The --s scale factor

Your plugin sits inside a container that is already scaled to the video, so plain pixels follow the video by themselves — most CSS needs no factor at all. Divide by --s only for chrome that should stay a constant size on screen however large the video is rendered: calc(16px / var(--s, 1)). Multiplying scales twice and shrinks. The value is always the video's rendered width divided by 1920, whatever a plugin declares for baseWidth.

Outputs

Outputs declare which project variables the plugin writes to. This is informational — it tells other developers and the Studio which variables this plugin modifies. The actual writing is done via mp4e.setVariable() in your JavaScript.

Output variable declarations

{
  outputs: [
    {
      variableId: 'score',
      name: 'Quiz Score',
      description: 'Updated when user answers a question'
    },
    {
      variableId: 'quizComplete',
      name: 'Quiz Complete',
      description: 'Set to true when all questions answered'
    }
  ]
}

Emits (Events)

Define events the plugin can emit. These become available as rule triggers in the Studio, so users can attach actions to your plugin's events.

Event definitions

{
  emits: {
    // Simple event (no data)
    clicked: {
      label: 'Clicked',
      description: 'Fired when the plugin is clicked'
    },

    // Event with typed data payload
    purchased: {
      label: 'Purchase',
      description: 'Fired when user completes a purchase',
      dataSchema: {
        type: 'object',
        properties: {
          productId: { type: 'string', description: 'Product ID' },
          price: { type: 'number', description: 'Purchase price' },
          quantity: { type: 'number', description: 'Quantity' }
        }
      }
    }
  }
}

// Emitting from JavaScript:
mp4e.emit('purchased', {
  productId: mp4e.getConfig().productId,
  price: mp4e.getConfig().price,
  quantity: 1
});

// The event is routed to rules as: plugin:<namespace>:<identifier>:purchased
// Rules can filter on event data using eventData conditions:
// { type: 'eventData', path: 'price', operator: 'gte', value: 100 }

Reference fields

When a field carries the ID of something in the video rather than free text, say so with x-ref. The Studio then offers that entity's picker as the Value control on an Event Data condition, so the author chooses Short from a list instead of typing a scene group ID they have no way of knowing. The value that gets stored is the same plain ID string either way, and the author can still switch to a template such as {{chosenCut}}.

Declaring a reference field

{
  emits: {
    'cut:selected': {
      label: 'Cut Selected',
      dataSchema: {
        type: 'object',
        properties: {
          groupId: {
            type: 'string',
            description: 'Scene group id of the cut',
            'x-ref': 'sceneGroup'      // -> scene group picker
          },
          sceneId: { type: 'string', 'x-ref': 'scene' },
          overlayId: { type: 'string', 'x-ref': 'overlay' },
          layerId: { type: 'string', 'x-ref': 'layer' },
          objectId: { type: 'string', 'x-ref': 'object' }
        }
      }
    }
  }
}

Values are scene, sceneGroup, overlay, layer, object, group, variable, plugin, subtitleTrack and audioTrack. Leave x-ref off any field that holds ordinary text or a number.

Where to write it

x-ref is part of the plugin manifest, so it is authored in the manifest you seed or import rather than in the Emits tab, whose property rows cover name, type and description. An annotation written in the manifest survives editing the plugin in the Studio.

Actions

Actions are methods that other plugins or rules can call on your plugin via the pluginAction action type.

Plugin actions

{
  actions: [
    {
      id: 'nextQuestion',
      label: 'Next Question',
      description: 'Advance to next quiz question',
      configSchema: {
        type: 'object',
        properties: {
          skipAnimation: { type: 'boolean', default: false }
        }
      }
    },
    {
      id: 'reset',
      label: 'Reset',
      description: 'Reset plugin to initial state'
    }
  ]
}

// Handle incoming actions in JavaScript:
mp4e.onAction('nextQuestion', function(config) {
  var skip = config && config.skipAnimation;
  advanceToNextQuestion(skip);
});

mp4e.onAction('reset', function() {
  resetState();
});

// Other plugins/rules call these via:
// { type: 'pluginAction', pluginId: 'quiz', actionId: 'nextQuestion', config: { skipAnimation: true } }

Variables

Plugins can declare variables that are auto-created as project variables when the plugin is installed. They are prefixed with the plugin's namespace:

Plugin variables

{
  variables: [
    {
      id: 'score',
      name: 'Quiz Score',
      type: 'number',
      default: 0,
      editable: false  // Users can't modify in Studio
    },
    {
      id: 'currentQuestion',
      name: 'Current Question',
      type: 'number',
      default: 0,
      editable: true
    }
  ]
}

// If plugin namespace is "mp4e:quiz", the variables become:
//   mp4e:quiz:score
//   mp4e:quiz:currentQuestion

// Access in JavaScript with full prefixed name:
mp4e.getVariable('mp4e:quiz:score').then(function(score) {
  console.log('Score:', score);
});
mp4e.setVariable('mp4e:quiz:score', 100);

Permissions

A plugin that reads the video's Intelligence (chapters, key moments, things, people, the transcript, …) names the scopes it needs in permissions.intelligence. The Studio shows the list to the creator when the plugin is installed — the way a phone asks before an app reads your contacts — and the creator decides what it may read. people and transcript start unticked; the other scopes start ticked. A plugin can never read a scope its manifest does not request.

Requesting Intelligence scopes

{
  permissions: {
    // summary, chapters, moments, questions, things, scenes, people, transcript
    intelligence: ['chapters', 'moments', 'things', 'transcript']
  }
}

Request only what the plugin uses: every scope you list is a line the creator has to accept. Read it with mp4e.getIntelligence() — see Intelligence in the API reference for the scopes, what each returns and how a refusal is reported.

Shortcuts

A plugin can offer keyboard shortcuts for its actions. Each entry names a key and the action it runs:

Plugin shortcuts

{
  actions: [
    { id: 'open', label: 'Open search' }
  ],
  shortcuts: [
    // key: a KeyboardEvent.key value — '/', 's', 'ArrowRight', …
    { id: 'open', key: '/', action: 'open', label: 'Open search' }
  ]
}

Installing the plugin writes each shortcut into the project as an ordinary input shortcut (Studio: Project → Events → Input Shortcuts): pressing the key runs pluginAction with the plugin's type and the shortcut's action. The Studio's install prompt lists them ("Open search with /") and the creator can leave any of them out. Nothing is overwritten: a shortcut is skipped when the project already uses that key for another shortcut, and the prompt says so. From then on the shortcut belongs to the creator, who can change or delete it like any other; installing the plugin again adds back one that was deleted.

Best Practices

Performance

Minimize DOM operations — batch updates when possible
Use CSS animations instead of JavaScript animations
Filter onVariableChange by changedIds to avoid unnecessary work
Use var(--s) CSS scaling instead of JavaScript-based resizing

Accessibility

Use semantic HTML elements
Add ARIA labels for interactive elements
Ensure sufficient color contrast
Support keyboard navigation (focusable elements)

Error Handling

Wrap async operations in try/catch
Provide fallback content for failed loads
Use mp4e.emit() to surface errors to rules
Validate config values before using (defaults may be undefined)

Config Gotchas

mp4e.getConfig() returns saved config, NOT merged with defaults
If a field was never saved, it will be undefined even if schema has a default
Always provide JS fallbacks: var mode = config.mode || 'sequential'
Template {{variable}} works in HTML/CSS only, use mp4e.getConfig() in JS

Next Steps

API Reference Template Syntax Events & Communication Subtitle Plugins Controls Plugins

Creating Plugins