Player Events

Complete reference for all engine events via the onEvent prop.

Overview

MP4E Player uses a single onEvent prop to deliver all engine events. This provides a clean, unified API for integrating with analytics, syncing external UI, or building custom behaviors.

CategoryEvent NamesDescription
Lifecycleplayback:ready, playback:loaded, metadata:loaded, metadata:loadStats, timeline:entryReady, timeline:coverage, timeline:chunkFailed, timeline:inlineChunkFailed, metadata:diagnostics, state:ready, playback:firstPlayLifecycle Events
Playbackstate:playing, playback:ended, playback:timeUpdate, playback:seeking, playback:seeked, playback:buffering, state:volume, state:muted, state:playbackRate, fullscreen:changed, playback:firstPixelPlayback Events
Objectobject:enter, object:exit, object:firstSeen, object:visible, object:hidden, object:click, object:hover, object:hoverEndObject Events
Overlayoverlay:click, overlay:hover, overlay:hoverEnd, overlay:clickAway, plugin:emit, plugin:<namespace>:<identifier>:<event>, plugin:init, overlay:presenceChangedOverlay Events
Scenescene:enter, scene:exit, scene:start, scene:endScene Events
Layerlayer:activate, layer:deactivateLayer Events
Playback Pathpath:changed, path:advanced, path:completedPlayback Path Events
Menumenu:selectMenu Events
Variablevariable:changed, timer:start, timer:complete, timer:pauseVariable Events
Actionaction:*, action:error, action:trackEventAction Events
Errorstate:error, metadata:error, plugin:errorError Events
Displaydisplay:show, display:hideDisplay Events
Dialogdialog:show, dialog:hideDialog Events
PerformanceblobCache:statusPerformance Events
Trackingtracking:show, tracking:hideTracking Events

Usage

All events are delivered through the onEvent prop. The callback receives the event name as a string and the event data as the second argument.

onEvent usage
1import { MP4EPlayer } from '@mp4e/react';
2
3<MP4EPlayer
4 src="/video.mp4"
5
6 // Single event handler for ALL engine events
7 onEvent={(event, data) => {
8 switch (event) {
9 case 'playback:ready':
10 console.log('Player ready, duration:', data.duration);
11 break;
12 case 'state:playing':
13 console.log(data ? 'Playing' : 'Paused');
14 break;
15 case 'object:click':
16 console.log('Clicked:', data.objectId, data.objectLabel);
17 break;
18 case 'variable:changed':
19 console.log(data.variableName, ':', data.previousValue, '->', data.newValue);
20 break;
21 }
22 }}
23/>

Lifecycle Events

PropTypeDefaultDescription
playback:ready{ duration: number }-Player is ready for playback. Video metadata is loaded.
playback:loaded{ duration: number }-Video loaded and duration is available.
metadata:loadedMetadataView-The interactive document is loaded and the engine is initialized. Carries the document view, not the document itself: schemaVersion, videoDetails (fps, width, height, durationSeconds), layers (id, name, order and their overlays — each overlay with id, layerId, type, name, overlayCategory, position, size, config, opacity, zIndex), audioTracks, media, thumbnails, objectCount and timeline.mode ("chunked" when the file streams its object tracking — see the timeline:* events — else "full"). Your overrides are already merged in. Objects, variables, rules, scenes and plugins are not part of the view.
metadata:loadStats{ totalMs: number; settledMs: number | null; source: "embedded" | "generated" | "host" | null; mode: "direct" | "worker" | null; payloadBytes: number; stages: Record<string, number | null> }-Per-stage load timing, emitted once per load after the engine has finished its load work. totalMs is when the player finished its own startup; settledMs is when the engine had finished all of it (null if unknown). Stages: extract, engineLoad, decrypt, jsonParse, setMetadata, diagnostics, bootstrapTick, uiInit — null when a stage did not run. For a document embedded in the file, decrypt covers the whole load (verify, decrypt, parse and apply) and setMetadata is null; for a document you pass as metadata (or one generated for a plain video), setMetadata covers it, including serializing the document, and decrypt is null. jsonParse is always null. setMetadata, decrypt and diagnostics time completed work in both modes.
timeline:entryReady{ msSinceStart: number }-Streamed timeline only (a file exported with streamTimeline): object tracking for the frames at the playhead is loaded, so objects and object-bound overlays can show. Fires once per load — again after a metadata reload or overrides change, since the player then fetches the tracking for the new document. msSinceStart is measured from the start of the chunk fetching; it can arrive before or after metadata:loaded.
timeline:coverage{ covered: [number, number][]; inFlight: number[]; playheadCovered: boolean }-Streamed timeline only: after each chunk of object tracking is loaded. covered lists the frame ranges now loaded ([start, end), frames), inFlight the chunks still being fetched, playheadCovered whether the current frame is loaded. The player fetches the chunk at the playhead first (again after every seek), then the chunks ahead of playback.
timeline:chunkFailed{ ordinal: number; error: string }-Streamed timeline only: a chunk of object tracking could not be fetched or used after four attempts, or the server ignores HTTP range requests (then no further chunks are fetched). Playback continues; objects in the frames that chunk covers stay hidden. With enableBlobCache, once the whole file is downloaded the player reads every chunk from it instead, including the failed ones.
timeline:inlineChunkFailed{ error: string }-Streamed timeline only: the chunk of object tracking carried inside the file for the first frames could not be used. Playback continues; the player fetches that chunk from the file like the others, and object geometry for those frames appears once it arrives.
metadata:diagnostics{ findings: DiagnosticFinding[]; source: 'set' | 'update' }-Engine diagnostics for the document just loaded (errors, warnings, info); never blocks playback.
state:readyboolean-Engine is ready (true) or not.
playback:firstPlayundefined-First playback started. Fires once per session.
Lifecycle events
1<MP4EPlayer
2 src="/video.mp4"
3 onEvent={(event, data) => {
4 switch (event) {
5 case 'playback:ready':
6 console.log('Player ready, duration:', data.duration, 'seconds');
7 break;
8 case 'metadata:loaded':
9 // The document view: render fields only, overrides already merged in
10 console.log('Objects:', data.objectCount);
11 console.log('Layers:', data.layers.length);
12 console.log('Frame rate:', data.videoDetails.fps);
13 break;
14 case 'metadata:diagnostics':
15 console.log(data.findings.length, 'findings from a', data.source, 'load');
16 console.log('First:', data.findings[0].code, 'at', data.findings[0].path);
17 break;
18 case 'playback:firstPlay':
19 console.log('User started watching');
20 analytics.track('video_start');
21 break;
22 }
23 }}
24/>

Playback Events

PropTypeDefaultDescription
state:playingboolean-true when playback starts, false when paused.
playback:endedundefined-Video reached the end.
playback:timeUpdate{ time: number; frame: number }-Time update (throttled to ~60fps).
playback:seeking{ fromTime: number; toTime: number }-Seek started.
playback:seeked{ currentTime: number }-Seek completed.
playback:bufferingboolean-true when buffering starts, false when it ends.
state:volumenumber-Volume changed (0-1).
state:mutedboolean-Mute state changed.
state:playbackRatenumber-Playback rate changed.
fullscreen:changedboolean-Fullscreen state changed.
playback:firstPixel{ msSinceInit: number; reason: "ready" | "loadeddata" | "error" | "play"; decoded: boolean }-Fires once per init(), when the overlay reveal gate opens and overlays become visible. If none of the four reasons arrive (a source that never decodes and never errors), no event fires until the viewer starts playback. decoded is true for "ready"/"loadeddata" (an actual decoded pixel) and false for "error"/"play" (msSinceInit is just a reveal time) — drop decoded: false samples from time-to-first-pixel metrics.
Playback events
1<MP4EPlayer
2 src="/video.mp4"
3 onEvent={(event, data) => {
4 switch (event) {
5 case 'state:playing':
6 console.log(data ? 'Playing' : 'Paused');
7 break;
8 case 'playback:ended':
9 console.log('Video ended');
10 break;
11 case 'playback:timeUpdate':
12 updateProgressBar(data.time);
13 break;
14 case 'playback:seeking':
15 console.log('Seeking:', data.fromTime, '->', data.toTime);
16 break;
17 case 'playback:seeked':
18 console.log('Seeked to:', data.currentTime);
19 break;
20 case 'playback:buffering':
21 if (data) showLoadingSpinner();
22 else hideLoadingSpinner();
23 break;
24 case 'state:volume':
25 console.log('Volume:', data);
26 break;
27 case 'state:muted':
28 console.log(data ? 'Muted' : 'Unmuted');
29 break;
30 case 'state:playbackRate':
31 console.log('Speed:', data + 'x');
32 break;
33 case 'fullscreen:changed':
34 console.log(data ? 'Fullscreen' : 'Windowed');
35 break;
36 }
37 }}
38/>
Performance
playback:timeUpdate fires frequently during playback (~60fps). For performance-sensitive operations, consider debouncing or filtering by event name early.

Object Events

PropTypeDefaultDescription
object:enter{ objectLabel: string; objectId: string; objectType: string; confidence: number; boundingBoxX: number; boundingBoxY: number; boundingBoxWidth: number; boundingBoxHeight: number; isVisible: boolean }-Object enters frame
object:exit{ objectLabel: string; objectId: string; objectType: string; confidence: number; boundingBoxX: number; boundingBoxY: number; boundingBoxWidth: number; boundingBoxHeight: number; isVisible: boolean }-Object exits frame
object:firstSeen{ objectLabel: string; objectId: string; objectType: string; confidence: number; boundingBoxX: number; boundingBoxY: number; boundingBoxWidth: number; boundingBoxHeight: number; isVisible: boolean }-First time object appears
object:visible{ objectLabel: string; objectId: string; objectType: string; confidence: number; boundingBoxX: number; boundingBoxY: number; boundingBoxWidth: number; boundingBoxHeight: number; isVisible: boolean }-Object becomes visible
object:hidden{ objectLabel: string; objectId: string; objectType: string; confidence: number; boundingBoxX: number; boundingBoxY: number; boundingBoxWidth: number; boundingBoxHeight: number; isVisible: boolean }-Object goes out of frame
object:click{ objectLabel: string; objectId: string; objectType: string; confidence: number; boundingBoxX: number; boundingBoxY: number; boundingBoxWidth: number; boundingBoxHeight: number; isVisible: boolean; clickX: number; clickY: number; clickCount: number }-User clicks object
object:hover{ objectLabel: string; objectId: string; objectType: string; confidence: number; boundingBoxX: number; boundingBoxY: number; boundingBoxWidth: number; boundingBoxHeight: number; isVisible: boolean }-User hovers object
object:hoverEnd{ objectLabel: string; objectId: string; objectType: string; confidence: number; boundingBoxX: number; boundingBoxY: number; boundingBoxWidth: number; boundingBoxHeight: number; isVisible: boolean }-User stops hovering
Object events
1<MP4EPlayer
2 src="/video.mp4"
3 onEvent={(event, data) => {
4 switch (event) {
5 case 'object:visible':
6 console.log('Object visible:', data.objectId, data.objectLabel);
7 analytics.track('product_impression', { id: data.objectId });
8 break;
9 case 'object:hidden':
10 console.log('Hidden:', data.objectId);
11 break;
12 case 'object:click':
13 console.log('Clicked:', data.objectId, data.objectLabel);
14 break;
15 case 'object:hover':
16 console.log('Hovering:', data.objectId);
17 break;
18 case 'object:firstSeen':
19 console.log('First time seeing:', data.objectLabel);
20 break;
21 }
22 }}
23/>
Object Visibility
object:visible and object:hidden fire when objects enter/leave the video frame based on AI tracking data. Use these for product impression tracking.

Overlay Events

PropTypeDefaultDescription
overlay:click{ overlayId: string; overlayType?: string; overlayName?: string }-User clicks overlay
overlay:hover{ overlayName: string; overlayId: string; overlayType: string; zIndex: number; opacity: number; positionX: number; positionY: number; isVisible: boolean }-User hovers overlay
overlay:hoverEnd{ overlayName: string; overlayId: string; overlayType: string; zIndex: number; opacity: number; positionX: number; positionY: number; isVisible: boolean }-User stops hovering
overlay:clickAway{ overlayName: string; overlayId: string }-User clicks outside this overlay (on video background)
plugin:emit{ event: string; data: any; pluginId?: string }-A plugin sent a custom event with mp4e.emit(event, data). pluginId is the sending instance.
plugin:<namespace>:<identifier>:<event>any-The same event under its own name, with the plugin's data as the payload — e.g. plugin:mp4e:search:search:selected.
plugin:init{ pluginId: string }-Plugin iframe initialized.
overlay:presenceChanged{ overlayIds: string[] }-The set of overlays with a live plugin iframe changed — one appeared, was released, or was shown/hidden by the engine.
Overlay events
1<MP4EPlayer
2 src="/video.mp4"
3 onEvent={(event, data) => {
4 switch (event) {
5 case 'overlay:click':
6 console.log('Clicked:', data.overlayId, data.overlayType, data.overlayName);
7 break;
8 case 'plugin:emit':
9 // Every event a plugin sends with mp4e.emit(name, data)
10 console.log('Plugin event:', data.event, 'from:', data.pluginId);
11 if (data.event === 'quiz:answer') {
12 handleQuizAnswer(data.data);
13 }
14 if (data.event === 'search:selected') {
15 // The viewer jumped to an in-video search result: { query?, kind, time }
16 analytics.track('video_search', data.data);
17 }
18 break;
19 }
20 }}
21/>
Plugin Events
Plugins send custom events with mp4e.emit(eventName, data). Each reaches onEvent twice: as plugin:emit with { event, data, pluginId } (pluginId is the sending instance), and under its own name plugin:<namespace>:<identifier>:<eventName> with the data alone — e.g. plugin:mp4e:search:search:selected.

Scene Events

PropTypeDefaultDescription
scene:enter{ sceneName: string; sceneId: string; sceneStartFrame: number; sceneEndFrame: number; sceneDuration: number; loopCount: number; isLooping: boolean; reason: string; fromScene?: string }-Scene becomes active (semantic — fires on any activation: playback, seek, goToScene)
scene:exit{ sceneName: string; sceneId: string; sceneStartFrame: number; sceneEndFrame: number; sceneDuration: number; loopCount: number; isLooping: boolean; reason: string; toScene?: string }-Scene becomes inactive (semantic — fires on any deactivation: playback end, seek away, goToScene away)
scene:start{ sceneName: string; sceneId: string; sceneStartFrame: number; sceneEndFrame: number }-Playhead crosses scene start frame (positional — natural playback only, does not fire on seeks)
scene:end{ sceneName: string; sceneId: string; sceneStartFrame: number; sceneEndFrame: number }-Playhead crosses scene end frame (positional — natural playback only, does not fire on seeks)
Scene events
1<MP4EPlayer
2 src="/video.mp4"
3 onEvent={(event, data) => {
4 switch (event) {
5 case 'scene:enter':
6 console.log('Entered:', data.sceneId, data.sceneName);
7 updateBreadcrumbs(data.sceneId);
8 break;
9 case 'scene:exit':
10 console.log('Exiting:', data.sceneId);
11 break;
12 case 'scene:start':
13 console.log('Playhead reached start of:', data.sceneId);
14 break;
15 case 'scene:end':
16 console.log('Playhead reached end of:', data.sceneId);
17 break;
18 case 'scene:loop':
19 console.log('Loop #', data.loopCount, 'for scene:', data.sceneId);
20 break;
21 }
22 }}
23/>

Layer Events

PropTypeDefaultDescription
layer:activate{ layerName: string; layerId: string; layerOrder: number; overlayCount: number; blendMode: string }-Layer becomes active
layer:deactivate{ layerName: string; layerId: string; layerOrder: number; overlayCount: number; blendMode: string }-Layer becomes inactive
Layer events
1<MP4EPlayer
2 src="/video.mp4"
3 onEvent={(event, data) => {
4 if (event === 'layer:activate') {
5 console.log('Layer activated:', data.layerId, data.overlayCount, 'overlays');
6 }
7 if (event === 'layer:deactivate') {
8 console.log('Layer deactivated:', data.layerId);
9 }
10 }}
11/>

Playback Path Events

Fired when a playback path (setPlaybackPath) is set, cleared, advances over a gap between scenes, or reaches the end of its last scene.

PropTypeDefaultDescription
path:changed{ groupId: string | null; previousGroupId: string | null; revision: number; duration: number; scenes: string[]; initiator: object }-A playback path was set or cleared, or was rebuilt (e.g. after a metadata update). groupId is null when the path was cleared.
path:advanced{ fromSceneId: string; toSceneId: string; skippedMediaStart: number; skippedMediaEnd: number; revision: number; initiator: object }-Playback reached the end of a scene in the path and silently jumped over the gap to the next scene.
path:completed{ groupId: string; duration: number; revision: number; initiator: object }-Playback reached the end of the path's last scene. The engine clamps and pauses; fires once per completion.
Playback Path events
1<MP4EPlayer
2 src="/video.mp4"
3 onEvent={(event, data) => {
4 switch (event) {
5 case 'path:changed':
6 console.log('Path:', data.groupId, 'duration:', data.duration, 'scenes:', data.scenes);
7 break;
8 case 'path:advanced':
9 console.log('Skipped gap:', data.fromSceneId, '->', data.toSceneId);
10 break;
11 case 'path:completed':
12 console.log('Path finished:', data.groupId);
13 break;
14 }
15 }}
16/>
Presentation Time
While a playback path is active, controls and the progress bar operate in presentation time (position within the path) rather than media time. Authored rules and other plugins keep seeing ordinary media time via currentTime.

Fired when the viewer picks a menu item a plugin contributed at runtime via mp4e.setMenuItems(). What the pick does is the plugin's business — the engine resolves it and hands it back; hosts observe it via onEvent, plugins via mp4e.on('menu:select', …) (delivered only to the contributing plugin). Dismissing the menu afterwards is handled internally between the engine and the controls plugin; no event for it reaches onEvent.

PropTypeDefaultDescription
menu:select{ pluginId: string; itemId: string; childId?: string; initiator: object }-The viewer selected a plugin-contributed menu item (or a child of a submenu item).

Variable Events

PropTypeDefaultDescription
variable:changed{ variableName: string; variableType: string; previousValue: any; newValue: any }-Any variable value changed.
timer:start{ timerName: string; target: number; direction: "up" | "down" }-Timer started.
timer:complete{ timerName: string; duration: number }-Timer completed.
timer:pause{ timerName: string; currentValue: number }-Timer paused.
Variable events
1<MP4EPlayer
2 src="/video.mp4"
3 onEvent={(event, data) => {
4 switch (event) {
5 case 'variable:changed':
6 console.log(data.variableName, ':', data.previousValue, '->', data.newValue);
7 // Sync with external state
8 if (data.variableName === 'cartTotal') {
9 updateCartUI(data.newValue);
10 }
11 break;
12 case 'timer:start':
13 console.log('Timer started:', data.timerName, 'target:', data.target);
14 break;
15 case 'timer:complete':
16 console.log('Timer completed:', data.timerName);
17 showTimeoutMessage();
18 break;
19 case 'timer:pause':
20 console.log('Timer paused at:', data.currentValue);
21 break;
22 }
23 }}
24/>
State Sync
Use the variable:changed event to sync video state with your application. For example, update a cart UI when a "cartTotal" variable changes. Each payload includes an initiator describing what caused the change — see Event Provenance below. A variable that holds embedded media carries its portable media://<id> reference as its value, not a blob URL; resolve it with resolveMediaUrl(url, videoUrl, loadedMedia) from @mp4e/player (also exported by @mp4e/react).

Action Events

PropTypeDefaultDescription
action:*{ type: string; [key: string]: any }-Action executed (e.g. action:seek, action:setVariable).
action:error{ actionType: string; error: string }-Action failed.
action:trackEvent{ eventName: string; properties?: Record<string, any> }-Analytics event fired via trackEvent action.
Action Events
Action events use the action: prefix followed by the action type (e.g. action:seek, action:setVariable). Listen for specific action types or use event.startsWith('action:') to catch all actions. Every executed action also emits after:{actionType} with the fully resolved action payload and its initiator — see Event Provenance below.

Error Events

PropTypeDefaultDescription
state:errorstring | null-General engine error occurred.
metadata:error{ message: string }-Metadata-specific error.
plugin:error{ pluginId: string; error: string }-Plugin error (non-fatal).
Error events
1<MP4EPlayer
2 src="/video.mp4"
3 onEvent={(event, data) => {
4 switch (event) {
5 case 'state:error':
6 if (data) {
7 console.error('Engine error:', data);
8 errorReporter.capture({ code: 'ENGINE_ERROR', message: data });
9 }
10 break;
11 case 'metadata:error':
12 console.error('Metadata error:', data.message);
13 // Fall back to non-interactive playback
14 break;
15 case 'plugin:error':
16 console.warn('Plugin error:', data.pluginId, data.error);
17 break;
18 }
19 }}
20/>

Display Events

Display events fire when object or overlay displays are shown or hidden, triggered by the display resolution cascade in response to user interactions.

PropTypeDefaultDescription
display:show{ displayId: string; objectId?: string; overlayId?: string; event: string; config?: object; position?: object; pluginType?: string; html?: string; css?: string; script?: string; autoClose?: object; box?: { width: number; height: number }; placement?: { x: number; y: number; width: number; height: number } }-Object/overlay display is shown. box is the card's size and placement its top-left and size, in video pixels, when the player sized or placed the card itself. Showing a display that is already open shows it again.
display:hide{ displayId?: string; objectId?: string; overlayId?: string }-A display closed: its object left the screen, an automatic card closed, hideObjectDisplay was called, or the viewer closed it (its close button, a tap outside it, the pointer leaving it, its timeout). One event per display.
Display events
1<MP4EPlayer
2 src="/video.mp4"
3 onEvent={(event, data) => {
4 switch (event) {
5 case 'display:show':
6 console.log('Display shown:', data.displayId, 'for:', data.event);
7 if (data.objectId) {
8 console.log('For object:', data.objectId);
9 }
10 break;
11 case 'display:hide':
12 console.log('Display hidden:', data.displayId);
13 break;
14 }
15 }}
16/>
Display vs Overlay
Displays are dynamic UI elements (tooltips, product cards) that appear in response to object interactions. They differ from overlays, which are positioned on the video. A single overlay can trigger different displays based on hover vs click events.

Dialog Events

Dialog events fire when the engine presents or dismisses a dialog (alert, confirm, prompt, or custom modal).

PropTypeDefaultDescription
dialog:show{ type: 'alert' | 'confirm' | 'prompt' | 'customModal'; title?: string; message?: string; config?: object }-Dialog presented to the user.
dialog:hide{ type: string; result: any }-Dialog dismissed. result contains the user's response.
Dialog events
1<MP4EPlayer
2 src="/video.mp4"
3 onEvent={(event, data) => {
4 switch (event) {
5 case 'dialog:show':
6 console.log('Dialog opened:', data.type, data.title);
7 break;
8 case 'dialog:hide':
9 console.log('Dialog closed:', data.type, 'result:', data.result);
10 if (data.type === 'confirm' && data.result === true) {
11 console.log('User confirmed action');
12 }
13 break;
14 }
15 }}
16/>

Performance Events

Performance events provide visibility into the blob cache system, which downloads the full video in the background for instant seeking.

PropTypeDefaultDescription
blobCache:status{ state: 'idle' | 'downloading' | 'swapping' | 'cached' | 'error' | 'skipped'; progress?: number; fileSize?: number }-Blob cache state changes. progress is 0-1 during download.
Performance events
1<MP4EPlayer
2 src="/video.mp4"
3 enableBlobCache={true}
4 onEvent={(event, data) => {
5 if (event === 'blobCache:status') {
6 switch (data.state) {
7 case 'downloading':
8 console.log('Caching video:', Math.round((data.progress || 0) * 100) + '%');
9 break;
10 case 'cached':
11 console.log('Video cached, size:', data.fileSize, 'bytes');
12 break;
13 case 'error':
14 console.warn('Blob cache error');
15 break;
16 case 'skipped':
17 console.log('File too large for caching');
18 break;
19 }
20 }
21 }}
22/>
Blob Cache States
The blob cache transitions through states: idle → downloading (with progress 0-1) → swapping → cached. If the file exceeds the size limit, the state will be skipped. Errors are reported as error.

Tracking Events

Tracking events fire when tracking visualizations (polygons, bounding boxes, mesh, corners) are shown or hidden at runtime.

PropTypeDefaultDescription
tracking:show{ mode: string; target: string; targetId: string; style?: object; duration?: number; zIndex?: number }-Tracking visualization shown (via showTracking action).
tracking:hide{ target: string; targetId: string }-Tracking visualization hidden.
Tracking events
1<MP4EPlayer
2 src="/video.mp4"
3 onEvent={(event, data) => {
4 switch (event) {
5 case 'tracking:show':
6 console.log('Tracking shown:', data.mode, 'for', data.target, data.targetId);
7 if (data.duration) {
8 console.log('Will auto-hide after', data.duration, 'ms');
9 }
10 break;
11 case 'tracking:hide':
12 console.log('Tracking hidden:', data.target, data.targetId);
13 break;
14 }
15 }}
16/>
Tracking Modes
Available tracking modes include polygon, bbox, corners, mesh, meshUV, amodal, and all. Tracking can target a specific object, a group, or all objects.

Metadata Diagnostics

The engine reports every problem it hits or silently skips in the document it just loaded. The metadata:diagnostics event carries those findings as { findings, source }. It fires after each load that produced at least one finding — source: 'set' for the initial load or setMetadata, source: 'update' on the soft-reload path taken by reloadMetadata or an overrides change (an uninitialised or destroyed engine falls through to a full reinit and reports 'set' instead). A clean document emits nothing. The player also logs the same findings in one collapsed console group on every load that produces findings.

Diagnostics depend on the engine that is actually running. Against an older engine — a video carrying an embedded engine that predates them, say — the player skips the per-load read entirely and stays silent: no event, no log. playerRef.diagnose(metadata) is an explicit call, so it rejects when the engine cannot answer rather than returning an empty array, which would read as “this document is clean”.

Finding shape

PropTypeDefaultDescription
severity'error' | 'warning' | 'info'-error = this part of the document cannot work as written; the engine drops or skips it (a dangling reference, an unparseable condition or variable, missing required plugin config, an inverted frame range). warning = it runs, but probably not as intended, or the engine cannot verify it (an action type no plugin declares, a config value whose type contradicts the schema, a timer target its direction never reaches). info = an observation, no defect (a rule with no actions, a keyframe outside the visible window that still acts as a hold value).
codestring-Namespaced, stable identifier for the kind of finding — parse.*, ref.*, type.*, range.*, structure.*, plugin.config.*, template.*. Match on this, not on the message. New codes are added over time; existing codes are never renamed or re-meant.
pathstring-Dotted path to the offending value in the metadata document, with numeric array indexes (e.g. layers.2.overlays.0.onClick.1.sceneId). Stable, and part of the contract alongside code.
messagestring-Human-readable English explanation of what the engine does with this part of the document. Written for people and may change — never parse it.
entity{ kind: string; id: string }-The nearest owning entity, when there is one — scene, sceneGroup, layer, overlay, variable, rule, objectGroup, or menuItem (the owner of a playerMenuConfig.customItems entry). Omitted for document-level findings.
related{ kind: string; id: string }[]-Other entities the finding refers to, such as the missing target of a dangling reference.
suggestionsstring[]-Concrete replacements the author probably meant: the closest existing ids for a dangling reference, the permitted values for a bad enum, the known types for an unknown type string. Omitted when nothing sensible exists.
Reacting to diagnostics
1<MP4EPlayer
2 src="/video.mp4"
3 onEvent={(event, data) => {
4 if (event !== 'metadata:diagnostics') return;
5
6 // data.source: 'set' on the initial load, 'update' after a reload or overrides change
7 const errors = data.findings.filter(f => f.severity === 'error');
8 console.log(data.findings.length, 'findings (' + errors.length + ' errors) from a', data.source, 'load');
9
10 for (const f of errors) {
11 console.error(f.code, 'at', f.path, '—', f.message,
12 f.suggestions?.length ? 'did you mean ' + f.suggestions[0] + '?' : '');
13 }
14 }}
15/>
Findings never block playback
Diagnostics are read-only: the engine reports what it would ignore, and plays the video either way. Use them in your own developer tooling — decide for yourself whether an error is worth surfacing to viewers. To check a document before applying it, call playerRef.diagnose(metadata), which returns the same findings without touching player state.

Event Provenance

Action and state events carry an initiator object describing what caused them — the viewer, a metadata rule (and which one), a plugin, your host application, or engine automation. It is present on after:* and action:* payloads, on variable:changed, and on scene transition events. This turns a bare “paused” into “paused by rule quiz_gate when the viewer clicked the quiz overlay” — the difference between playback logging and behavioral analytics.

PropTypeDefaultDescription
type'user' | 'rule' | 'plugin' | 'host' | 'init' | 'system'-What caused the action or change. user = direct viewer input (controls, key bindings, menu). rule = a metadata rule fired. plugin = a plugin API call. host = your application called a player method. init = metadata load / variable defaults. system = engine automation (timers, sequences, scene transitions).
idstring-The specific source when one exists: the rule ID, plugin ID, or input-binding ID.
eventstring-For rule initiators, the event that fired the rule (e.g. overlay:click:btn-buy, control:pause).
viastring-Qualifier for how it happened: controls, inputBinding, menu, timer, sequence, sceneTransition, gateTimeout, emitEvent, setMetadata.
Attributing playback and state changes
1<MP4EPlayer
2 src="/video.mp4"
3 onEvent={(event, data) => {
4 // Playback attribution: was this pause the viewer, a rule, or a plugin?
5 if (event === 'after:pause') {
6 switch (data.initiator?.type) {
7 case 'user': console.log('Viewer paused'); break;
8 case 'rule': console.log('Paused by rule', data.initiator.id,
9 'on', data.initiator.event); break;
10 case 'plugin': console.log('Paused by plugin', data.initiator.id); break;
11 case 'host': console.log('Paused by your app'); break;
12 }
13 }
14 // Variable provenance: ignore initialization, keep real viewer choices
15 if (event === 'variable:changed' && data.initiator?.type !== 'init') {
16 analytics.track('choice', {
17 variable: data.variableName,
18 value: data.newValue,
19 causedBy: data.initiator?.type,
20 });
21 }
22 }}
23/>
Closed vocabulary
The initiator type is a closed enum — safe to group by in analytics dashboards. Fields that do not apply are omitted rather than null, so check with optional chaining.

Advanced

Wildcard Subscription

The onEvent prop acts as a wildcard subscriber — it receives every event emitted by the engine. The first parameter (event) contains the event name as a string, and the second parameter (data) contains the event payload. This makes it easy to forward all events to an analytics or logging system without listing individual event types.

Wildcard event subscription
1import { MP4EPlayer } from '@mp4e/react';
2
3// Subscribe to ALL events using onEvent
4<MP4EPlayer
5 src="/video.mp4"
6 onEvent={(event, data) => {
7 // 'event' contains the event name (e.g. 'playback:ready')
8 // 'data' contains the event payload
9 console.log(`[${event}]`, data);
10
11 // Forward all events to your analytics or logging system
12 analytics.track('mp4e_event', {
13 eventName: event,
14 payload: data,
15 timestamp: Date.now()
16 });
17 }}
18/>
Event Filtering
Since onEvent fires for every event, use event.startsWith() or a switch statement to filter for the categories you care about. For high-frequency events like playback:timeUpdate, consider early returns to avoid unnecessary processing.