Search Plugin

Overview

Search (mp4e:search, an official plugin) adds a Search entry to the player's settings menu. The viewer types a few words, results appear as they type, and a tap jumps to that moment and plays. It searches the video's Intelligence — chapters, key moments, the questions it answers, the things and people in it, what happens in each scene — and its subtitles.

Everything runs inside the player, on the viewer's device: no server, no account, and it works offline and on any site that plays the file. The plugin reads the video through mp4e.getIntelligence() like any other plugin can, so it sees only what the creator allowed and never anything the creator hid or kept private.

A video needs an analysis (Make interactive in the Studio) or subtitles to be searchable. With neither, the menu entry does not appear.

Adding search to a video

In the Studio, any of these:

• Make interactive — after the analysis, search is added to the video for you, allowed to read the transcript (not the people).
• Intelligence panel → Let viewers search this video — installs Search, asks what it may read, and places it on a new Search layer above your other layers, in one step you can undo.
• Marketplace — install Search, then add one Search overlay covering the whole video. One full-frame overlay is the whole plugin; never bind it to an object group.

On install the Studio lists what Search asks to read. Chapters, key moments, questions, things and scenes start ticked; people and transcript (everything said) start unticked — tick them to make those searchable. The choice can be changed later under Intelligence access in the plugin's settings. The install also adds the shortcut / (open search) to the project's Input Shortcuts, unless that key is already in use; it is yours to change.

What viewers find

Words are matched without regard to case, small words such as "the" and "how" are ignored, and plurals match their singular. The last word also matches the start of a longer word, so results follow the typing. Results that match more of the words come first; a chapter title counts most, then questions, key moments, things and people, then summaries, key moment descriptions, scenes and subtitle lines, and a result that contains the whole query as a phrase is lifted. Neighbouring subtitle lines that match become one result. At most 20 results are shown. A kind the plugin may not read is simply not searched.

The search panel

• A sheet across the frame on a phone-sized player (narrower than 640 px), a side sheet on larger screens. The choice follows the size the video is shown at, and the panel always stays clear of the player's control bar.
• Readable at any player size: Search grows with the video like the other family plugins, but never below a minimum readable size — on a phone-sized player the text stays about 15 px.
• Styled by the plugin family's five themes (Clean, Bold, Cinematic, Playful, Studio) and the family's accent, size, colour, font, corner and motion settings. A viewer who asked for high contrast gets solid panels.
• Each row shows its kind (Chapter · Moment · Question · Thing · Person · Scene · Said), a title, the time and the text with the matching words highlighted.
• Arrow keys move between the search box, the close button and the results, Enter jumps, Escape closes — with a visible focus ring, from a keyboard, a TV remote or a gamepad. Enter in the search box jumps to the top result.
• While the panel is open the video is paused (the pauseWhileSearching setting). Closing without a pick resumes playback if search paused it; a pick seeks, closes and plays.
• The analysis is re-read when the creator edits it or the viewer changes subtitles.

With cuts

When a shorter cut of the video is playing (a playback path, e.g. from the Cuts plugin), results inside the cut show times as the progress bar does. Results outside it are marked Not in this cut; picking one leaves the cut and jumps there. When the host has locked the playback path, those results are shown but cannot be picked.

Settings

The overlay's settings in the Studio:

Actions and events

Search accepts two actions through pluginAction — from rules, other plugins and input shortcuts: open (does nothing when the video has nothing to search) and close.

{ "type": "pluginAction", "pluginId": "mp4e:search", "actionId": "open" }

When the viewer jumps to a result it emits search:selected with { query?, kind, time } — kind is one of the kinds above, time the second of the video it jumped to. Rules hear it as plugin:mp4e:search:search:selected:

Trigger:  plugin:mp4e:search:search:selected
Action:   setVariable  searches, operation increment, value 1

Your page hears it through onEvent (see Player Events):

<MP4EPlayer
  src="/video.mp4"
  onEvent={(event, data) => {
    if (event === 'plugin:emit' && data.event === 'search:selected') {
      // { query?, kind, time } — query is absent when the viewer turned on Do not track
      analytics.track('video_search', data.data);
    }
  }}
/>

Privacy

The index is built and searched in the player; nothing the viewer types leaves the device except in search:selected, and only when the viewer has not turned on Do not track (see Viewer Preferences). With it on, the event carries the kind and time only.