Writing FrameX plugins
A plugin adds to FrameX without changing it. It can:
- add commands to the Plugins menu,
- edit the project, through the same checked edits FrameX's AI uses, each one undoable,
- react to events: a project opened, media imported, a transcript made, a video exported,
- draw explainer scenes, which FrameX's explainer can then plan into videos.
A plugin is an ordinary program, in any language: Python, JavaScript, C#, Rust, Go or anything else that can read and write lines of text. FrameX starts it when it's first needed and talks to it in JSON over its standard input and output. Because it runs as a program of its own, a plugin can't crash FrameX. If it crashes, stops answering or sends something FrameX can't read, FrameX stops it, turns it off and tells the user why. It stays off until the user turns it back on in Plugins > Manage plugins….
Packaging a plugin
You hand out a plugin as a zip. Users install it by dropping the zip onto FrameX's window, or with Install from a file… in the Plugins window. FrameX shows its icon, name, version, author, description and what it adds, and installs it only once the user presses Install.
hello.zip
plugin.json required: says what the plugin is
icon.png required: named by "icon" in plugin.json
main.py whatever your plugin needs, in any folders
plugin.jsongoes at the top of the zip. It can also be inside one folder that holds everything, which is what you get by zipping a folder.author,versionandiconare required, as well as the fields every plugin needs (below).- The icon is a square PNG, at least 64×64 and at most 1024×1024 pixels; 256×256 looks sharp everywhere. FrameX shows it when installing and in the Plugins window.
- The program in
run, when it's a file ("hello.exe","bin/hello.exe"), must be in the zip. A bare name such as"python"or"node"is a program the user installs, so say so in your description. - At most 5000 files and 1 GB once unpacked.
- No file may point outside the plugin's folder, and there may be no links. A zip that breaks a rule isn't installed, and FrameX tells the user which rule.
FrameX unpacks a plugin into a folder named after its id. Installing a newer version replaces the old one in one step and keeps the user's settings for it. If the old version had been turned off after a crash, the new one starts turned on.
Where plugins live
Installed plugins are folders inside FrameX's plugins folder, which Plugins > Open the plugins folder opens. On Windows it's %LOCALAPPDATA%\FrameX\plugins. While you develop a plugin, you can work right there, without zipping it each time: change your files, then press Reload in the Plugins window. The same rules apply, including the icon.
plugins\
hello\
plugin.json
icon.png
main.py
Plugins run like any program on the user's PC, with the same access to their files. Tell your users what yours does. FrameX reminds them to install plugins only from people they trust.
plugin.json
{
"id": "hello",
"name": "Hello Plugin",
"version": "1.0.0",
"author": "Your name",
"icon": "icon.png",
"description": "What it does, in a sentence.",
"api": 1,
"run": ["python", "main.py"],
"commands": [{ "id": "greet", "title": "Add a greeting at the playhead" }],
"events": ["exported"],
"visuals": [{
"kind": "countdown",
"label": "Countdown",
"description": "When to use it and what its data means, for the model planning explainers.",
"example": { "from": 10, "label": "Until launch" }
}],
"settings": [{ "key": "greeting", "label": "Greeting", "kind": "text", "default": "Hello" }]
}
| Field | |
|---|---|
id | Unique and stable. Lower-case letters, digits, ., - and _. FrameX remembers settings and whether the plugin is on by its id. |
name, version, author | Required. Shown when installing and in the Plugins window. |
icon | Required. A square PNG in the plugin, 64 to 1024 pixels across, by its path in the plugin's folder. |
description | What it does, in a sentence or two. Shown when installing. |
api | The plugin API it was written for. This FrameX speaks 1. A plugin written for another version isn't loaded, and the window says why. |
run | The program and its arguments. A program in the plugin's folder ("hello.exe", "bin/hello.exe") is run from there. Otherwise it's found like a command, so ["python", "main.py"] and ["node", "index.js"] work. The plugin runs in its own folder. |
commands | Menu items, each with an id sent back to you and a title shown in the menu. |
events | Any of project_opened, imported, transcribed, exported. |
visuals | Explainer scenes you draw. The description and example are given to the model that plans explainers, so it can use your scene where it fits. |
settings | Shown in the Plugins window and sent with every request. kind is text, number, toggle or choice (with choices). |
The conversation
Every message is one JSON object on one line. FrameX sends requests on your standard input:
{"id": 7, "method": "command", "params": {...}}
You answer each on your standard output, with the same id and either a result:
{"id": 7, "result": {"ops": [...], "message": "Done."}}
or an error, when the request didn't succeed but your plugin is fine. FrameX shows the message and leaves the plugin on:
{"id": 7, "error": {"message": "Select a clip first."}}
Requests come one at a time: FrameX waits for your answer before sending the next. Write only messages to standard output, and flush after each. Anything else, such as debugging output, goes to standard error: FrameX shows the last few hundred lines in the Plugins window and keeps them in plugin.log in your folder. A line on standard output that isn't JSON turns the plugin off.
Every request's params includes your settings, as an object of their current values.
initialize
Sent once, when FrameX starts your plugin, before anything else. Answer with any result, such as {}. An error here stops the plugin starting.
{"api": 1, "framex": "0.3.0", "settings": {...}}
command
A user picked one of your commands.
{
"command": "greet",
"project": { ... },
"playhead": 12.5,
"selection": ["clip_ab12"],
"range": {"start": 10.0, "end": 20.0},
"settings": { ... }
}
project is the whole project as FrameX saves it: its settings, its assets (media, with their transcripts), and its timeline (clips on the main track, overlays, texts (titles and captions), audio, markers). range is the In and Out range if one is marked, or null. Times are in seconds.
Answer with the edits to make and a line for the user, both optional:
{"ops": [{"op": "add_text", "text": "Hello", "start": 12.5, "duration": 3}], "message": "Added a greeting."}
FrameX checks every edit and applies them together as one step, named after your plugin and command, so one Ctrl+Z takes them all back. An edit it can't read or can't make is left out, and the user is told. The edits you can send are listed in Edits plugins can make.
event
Something happened that your plugin asked to hear about. The params are a command's, without command, plus:
event | details |
|---|---|
project_opened | {} |
imported | {"asset": {...}}: the media, once it's ready to use |
transcribed | {"asset": {...}}, with its transcript |
exported | {"path": "C:\\...\\video.mp4"} |
Answer like a command: edits and a message, both optional. {} is fine.
paint
One frame of one of your explainer scenes. FrameX draws the shapes you send with its own fonts and colours. It then adds the video's light, grain and camera moves, so your scene matches the rest of the video.
{"kind": "countdown", "data": {"from": 10, "label": "Until launch"}, "time": 1.25, "duration": 6.0, "width": 1920, "height": 1080, "words": [{"text": "Ten", "start": 0.4, "end": 0.7}], "settings": {...}}
timeis seconds into the scene;durationis how long it lasts.- The picture is always 1080 tall and
widthwide (1920 for 16:9, 608 for 9:16). 0,0 is the top left. wordsare the words spoken over the scene, timed from its start, so you can show things as they're said.- Draw it the same way every time for the same
time. FrameX may ask for frames in any order, and several times for motion blur.
Answer with up to 4000 shapes, drawn in order:
{"shapes": [
{"shape": "rect", "x": 100, "y": 200, "w": 400, "h": 80, "color": "accent", "opacity": 1},
{"shape": "circle", "x": 960, "y": 540, "r": 120, "color": "faint"},
{"shape": "line", "points": [[100, 900], [1800, 900]], "width": 3, "color": "dim"},
{"shape": "polygon", "points": [[0, 0], [200, 0], [100, 150]], "color": "#d4443a"},
{"shape": "text", "text": "Until launch", "x": 960, "y": 820, "size": 46, "font": "title", "align": "center", "color": "ink"}
]}
- Colours:
ink(text),dim,faint,accentandbackgroundfollow the video's look.#rrggbbis a fixed colour. - Fonts:
body,bold,serif,mono,light, ortitle, which sets narrow, evenly spaced capitals like FrameX's own titles. - Text:
yis the top of the line, andalignisleft,centerorrightofx. - A shape FrameX can't read is left out.
In an explainer's plan, your scene appears as:
{"type": "plugin", "plugin": "hello", "kind": "countdown", "data": {"from": 10, "label": "Until launch"}}
You can also write it into a scene by hand, in the Generated scene box of the Inspector. If your plugin is missing or off, the scene says it isn't available instead of being drawn.
Messages from you
While working, you can send FrameX messages of your own, without an id:
{"method": "notify", "params": {"text": "Found 12 titles", "problem": false}}
{"method": "progress", "params": {"fraction": 0.5}}
{"method": "log", "params": {"text": "for the Plugins window"}}
notify shows a line to the user. Set problem to show it as one.
When FrameX turns a plugin off
| What happened | |
|---|---|
| Its program couldn't be started | Wrong run, or a missing interpreter |
| It ended while FrameX waited for an answer | A crash, or it quit |
| It took too long to answer | 15 s to start, 60 s for a command, 30 s for an event, 5 s for a frame |
| It wrote a line that isn't JSON | Usually a stray print |
FrameX stops its program, turns it off, remembers why, and shows the user. The plugin stays off, even after FrameX restarts, until the user turns it back on. It then starts afresh. An error answer is not one of these: it's a plugin saying no, and it stays on.
When FrameX closes, it ends its plugins' programs too.
Your plugin and FrameX's licence
FrameX's licence lets you write plugins that work through this API and share or sell them on your own terms. They're your own work, as long as they contain no part of FrameX.