Complete reference for all engine events via the onEvent prop.
Overview
MP4E Player uses a single onEvent prop to deliver all engine events. This provides a clean, unified API for integrating with analytics, syncing external UI, or building custom behaviors.
Player is ready for playback. Video metadata is loaded.
playback:loaded
{ duration: number }
-
Video loaded and duration is available.
metadata:loaded
MetadataView
-
The interactive document is loaded and the engine is initialized. Carries the document view, not the document itself: schemaVersion, videoDetails (fps, width, height, durationSeconds), layers (id, name, order and their overlays — each overlay with id, layerId, type, name, overlayCategory, position, size, config, opacity, zIndex), audioTracks, media, thumbnails, objectCount and timeline.mode ("chunked" when the file streams its object tracking — see the timeline:* events — else "full"). Your overrides are already merged in. Objects, variables, rules, scenes and plugins are not part of the view.
Per-stage load timing, emitted once per load after the engine has finished its load work. totalMs is when the player finished its own startup; settledMs is when the engine had finished all of it (null if unknown). Stages: extract, engineLoad, decrypt, jsonParse, setMetadata, diagnostics, bootstrapTick, uiInit — null when a stage did not run. For a document embedded in the file, decrypt covers the whole load (verify, decrypt, parse and apply) and setMetadata is null; for a document you pass as metadata (or one generated for a plain video), setMetadata covers it, including serializing the document, and decrypt is null. jsonParse is always null. setMetadata, decrypt and diagnostics time completed work in both modes.
timeline:entryReady
{ msSinceStart: number }
-
Streamed timeline only (a file exported with streamTimeline): object tracking for the frames at the playhead is loaded, so objects and object-bound overlays can show. Fires once per load — again after a metadata reload or overrides change, since the player then fetches the tracking for the new document. msSinceStart is measured from the start of the chunk fetching; it can arrive before or after metadata:loaded.
Streamed timeline only: after each chunk of object tracking is loaded. covered lists the frame ranges now loaded ([start, end), frames), inFlight the chunks still being fetched, playheadCovered whether the current frame is loaded. The player fetches the chunk at the playhead first (again after every seek), then the chunks ahead of playback.
timeline:chunkFailed
{ ordinal: number; error: string }
-
Streamed timeline only: a chunk of object tracking could not be fetched or used after four attempts, or the server ignores HTTP range requests (then no further chunks are fetched). Playback continues; objects in the frames that chunk covers stay hidden. With enableBlobCache, once the whole file is downloaded the player reads every chunk from it instead, including the failed ones.
timeline:inlineChunkFailed
{ error: string }
-
Streamed timeline only: the chunk of object tracking carried inside the file for the first frames could not be used. Playback continues; the player fetches that chunk from the file like the others, and object geometry for those frames appears once it arrives.
Fires once per init(), when the overlay reveal gate opens and overlays become visible. If none of the four reasons arrive (a source that never decodes and never errors), no event fires until the viewer starts playback. decoded is true for "ready"/"loadeddata" (an actual decoded pixel) and false for "error"/"play" (msSinceInit is just a reveal time) — drop decoded: false samples from time-to-first-pixel metrics.
playback:timeUpdate fires frequently during playback (~60fps). For performance-sensitive operations, consider debouncing or filtering by event name early.
15 // The viewer jumped to an in-video search result: { query?, kind, time }
16 analytics.track('video_search', data.data);
17 }
18 break;
19 }
20 }}
21/>
Plugin Events
Plugins send custom events with mp4e.emit(eventName, data). Each reaches onEvent twice: as plugin:emit with { event, data, pluginId } (pluginId is the sending instance), and under its own name plugin:<namespace>:<identifier>:<eventName> with the data alone — e.g. plugin:mp4e:search:search:selected.
While a playback path is active, controls and the progress bar operate in presentation time (position within the path) rather than media time. Authored rules and other plugins keep seeing ordinary media time via currentTime.
Menu Events
Fired when the viewer picks a menu item a plugin contributed at runtime via mp4e.setMenuItems(). What the pick does is the plugin's business — the engine resolves it and hands it back; hosts observe it via onEvent, plugins via mp4e.on('menu:select', …) (delivered only to the contributing plugin). Dismissing the menu afterwards is handled internally between the engine and the controls plugin; no event for it reaches onEvent.
Use the variable:changed event to sync video state with your application. For example, update a cart UI when a "cartTotal" variable changes. Each payload includes an initiator describing what caused the change — see Event Provenance below. A variable that holds embedded media carries its portable media://<id> reference as its value, not a blob URL; resolve it with resolveMediaUrl(url, videoUrl, loadedMedia) from @mp4e/player (also exported by @mp4e/react).
Action events use the action: prefix followed by the action type (e.g. action:seek, action:setVariable). Listen for specific action types or use event.startsWith('action:') to catch all actions. Every executed action also emits after:{actionType} with the fully resolved action payload and its initiator — see Event Provenance below.
Error Events
Prop
Type
Default
Description
state:error
string | null
-
General engine error occurred.
metadata:error
{ message: string }
-
Metadata-specific error.
plugin:error
{ pluginId: string; error: string }
-
Plugin error (non-fatal).
Error events
1<MP4EPlayer
2 src="/video.mp4"
3 onEvent={(event, data) => {
4 switch (event) {
5 case 'state:error':
6 if (data) {
7 console.error('Engine error:', data);
8 errorReporter.capture({ code: 'ENGINE_ERROR', message: data });
Object/overlay display is shown. box is the card's size and placement its top-left and size, in video pixels, when the player sized or placed the card itself. Showing a display that is already open shows it again.
A display closed: its object left the screen, an automatic card closed, hideObjectDisplay was called, or the viewer closed it (its close button, a tap outside it, the pointer leaving it, its timeout). One event per display.
Displays are dynamic UI elements (tooltips, product cards) that appear in response to object interactions. They differ from overlays, which are positioned on the video. A single overlay can trigger different displays based on hover vs click events.
Dialog Events
Dialog events fire when the engine presents or dismisses a dialog (alert, confirm, prompt, or custom modal).
The blob cache transitions through states: idle → downloading (with progress 0-1) → swapping → cached. If the file exceeds the size limit, the state will be skipped. Errors are reported as error.
Tracking Events
Tracking events fire when tracking visualizations (polygons, bounding boxes, mesh, corners) are shown or hidden at runtime.
Available tracking modes include polygon, bbox, corners, mesh, meshUV, amodal, and all. Tracking can target a specific object, a group, or all objects.
Metadata Diagnostics
The engine reports every problem it hits or silently skips in the document it just loaded. The metadata:diagnostics event carries those findings as { findings, source }. It fires after each load that produced at least one finding — source: 'set' for the initial load or setMetadata, source: 'update' on the soft-reload path taken by reloadMetadata or an overrides change (an uninitialised or destroyed engine falls through to a full reinit and reports 'set' instead). A clean document emits nothing. The player also logs the same findings in one collapsed console group on every load that produces findings.
Diagnostics depend on the engine that is actually running. Against an older engine — a video carrying an embedded engine that predates them, say — the player skips the per-load read entirely and stays silent: no event, no log. playerRef.diagnose(metadata) is an explicit call, so it rejects when the engine cannot answer rather than returning an empty array, which would read as “this document is clean”.
Finding shape
Prop
Type
Default
Description
severity
'error' | 'warning' | 'info'
-
error = this part of the document cannot work as written; the engine drops or skips it (a dangling reference, an unparseable condition or variable, missing required plugin config, an inverted frame range). warning = it runs, but probably not as intended, or the engine cannot verify it (an action type no plugin declares, a config value whose type contradicts the schema, a timer target its direction never reaches). info = an observation, no defect (a rule with no actions, a keyframe outside the visible window that still acts as a hold value).
code
string
-
Namespaced, stable identifier for the kind of finding — parse.*, ref.*, type.*, range.*, structure.*, plugin.config.*, template.*. Match on this, not on the message. New codes are added over time; existing codes are never renamed or re-meant.
path
string
-
Dotted path to the offending value in the metadata document, with numeric array indexes (e.g. layers.2.overlays.0.onClick.1.sceneId). Stable, and part of the contract alongside code.
message
string
-
Human-readable English explanation of what the engine does with this part of the document. Written for people and may change — never parse it.
entity
{ kind: string; id: string }
-
The nearest owning entity, when there is one — scene, sceneGroup, layer, overlay, variable, rule, objectGroup, or menuItem (the owner of a playerMenuConfig.customItems entry). Omitted for document-level findings.
related
{ kind: string; id: string }[]
-
Other entities the finding refers to, such as the missing target of a dangling reference.
suggestions
string[]
-
Concrete replacements the author probably meant: the closest existing ids for a dangling reference, the permitted values for a bad enum, the known types for an unknown type string. Omitted when nothing sensible exists.
Reacting to diagnostics
1<MP4EPlayer
2 src="/video.mp4"
3 onEvent={(event, data) => {
4 if (event !== 'metadata:diagnostics') return;
5
6 // data.source: 'set' on the initial load, 'update' after a reload or overrides change
12 f.suggestions?.length ? 'did you mean ' + f.suggestions[0] + '?' : '');
13 }
14 }}
15/>
Findings never block playback
Diagnostics are read-only: the engine reports what it would ignore, and plays the video either way. Use them in your own developer tooling — decide for yourself whether an error is worth surfacing to viewers. To check a document before applying it, call playerRef.diagnose(metadata), which returns the same findings without touching player state.
Event Provenance
Action and state events carry an initiator object describing what caused them — the viewer, a metadata rule (and which one), a plugin, your host application, or engine automation. It is present on after:* and action:* payloads, on variable:changed, and on scene transition events. This turns a bare “paused” into “paused by rule quiz_gate when the viewer clicked the quiz overlay” — the difference between playback logging and behavioral analytics.
What caused the action or change. user = direct viewer input (controls, key bindings, menu). rule = a metadata rule fired. plugin = a plugin API call. host = your application called a player method. init = metadata load / variable defaults. system = engine automation (timers, sequences, scene transitions).
id
string
-
The specific source when one exists: the rule ID, plugin ID, or input-binding ID.
event
string
-
For rule initiators, the event that fired the rule (e.g. overlay:click:btn-buy, control:pause).
via
string
-
Qualifier for how it happened: controls, inputBinding, menu, timer, sequence, sceneTransition, gateTimeout, emitEvent, setMetadata.
Attributing playback and state changes
1<MP4EPlayer
2 src="/video.mp4"
3 onEvent={(event, data) => {
4 // Playback attribution: was this pause the viewer, a rule, or a plugin?
5 if (event === 'after:pause') {
6 switch (data.initiator?.type) {
7 case 'user': console.log('Viewer paused'); break;
8 case 'rule': console.log('Paused by rule', data.initiator.id,
9 'on', data.initiator.event); break;
10 case 'plugin': console.log('Paused by plugin', data.initiator.id); break;
11 case 'host': console.log('Paused by your app'); break;
12 }
13 }
14 // Variable provenance: ignore initialization, keep real viewer choices
15 if (event === 'variable:changed' && data.initiator?.type !== 'init') {
16 analytics.track('choice', {
17 variable: data.variableName,
18 value: data.newValue,
19 causedBy: data.initiator?.type,
20 });
21 }
22 }}
23/>
Closed vocabulary
The initiator type is a closed enum — safe to group by in analytics dashboards. Fields that do not apply are omitted rather than null, so check with optional chaining.
Advanced
Wildcard Subscription
The onEvent prop acts as a wildcard subscriber — it receives every event emitted by the engine. The first parameter (event) contains the event name as a string, and the second parameter (data) contains the event payload. This makes it easy to forward all events to an analytics or logging system without listing individual event types.
Wildcard event subscription
1import { MP4EPlayer } from '@mp4e/react';
2
3// Subscribe to ALL events using onEvent
4<MP4EPlayer
5 src="/video.mp4"
6 onEvent={(event, data) => {
7 // 'event' contains the event name (e.g. 'playback:ready')
8 // 'data' contains the event payload
9 console.log(`[${event}]`, data);
10
11 // Forward all events to your analytics or logging system
12 analytics.track('mp4e_event', {
13 eventName: event,
14 payload: data,
15 timestamp: Date.now()
16 });
17 }}
18/>
Event Filtering
Since onEvent fires for every event, use event.startsWith() or a switch statement to filter for the categories you care about. For high-frequency events like playback:timeUpdate, consider early returns to avoid unnecessary processing.