AI and agents

How it works

One contract defines every action in Cue. The editor window, the in-app chat and the MCP server all call it, so an agent's edit is recorded, attributed and undoable exactly like yours, and it shows up in the history as made by an agent. Every agent also receives the same written guide to editing in Cue (the get_guide tool returns it).

There are two ways in:

• The Agent panel inside Cue, which runs a coding agent you already have installed.
• Any MCP client on your Mac, such as Claude Code in a terminal.

The Agent panel

Open the Agent panel in the sidebar, or the Agent workspace (⌥6), which puts the chat in a wide sidebar with the history docked beside the viewer. Cue finds these agents if they are installed, and runs them signed in with your own account:

• Each message tells the agent what you see: the playhead, your selection and the in and out points. So "make this clip black and white" works on the selected clip.
• Right-click a clip and choose Ask the agent about this… to start a message about it.
• The agent gets Cue's tools only: no shell, no files and no web. Tool calls show in the chat as they run, and the cost of the conversation shows under the box when the agent reports it.
• Use the stop button to end a run, and the new-conversation button to start fresh.

Connect an MCP client

With Cue in Applications, pick your agent and run its command once in a terminal (or add the config it shows):

claude mcp add --scope user cue -- node "/Applications/Cue.app/Contents/Resources/mcp/cue-mcp.mjs"

Claude Code, Codex, Gemini CLI, VS Code, Cursor, Claude Desktop and any other MCP client start the same server: node with the path to cue-mcp.mjs inside the app. You can copy it from Settings → Agent or from the Agent panel's Connect & activity tab, which also shows whether an agent is connected and what it has done.

• The server starts Cue when needed.
• It talks to Cue on this Mac only, over a loopback connection with a token that changes every launch.
• Settings → Agent → Allow agents to control Cue turns outside access off entirely.

Reviewing agent edits

Turn on Review edits under the chat box (or Settings → Agent → Review agent edits) to have an agent's changes arrive as a proposal instead of being kept straight away.

• A bar over the timeline says the agent proposes changes, with how many clips were added, changed and removed.
• Added clips are outlined green and changed clips amber. Removed clips stay visible as dashed outlines.
• Hover a clip to keep or undo that one change (for a removed clip: keep it removed, or bring it back), or use Keep all and Undo all.

Agents cannot accept their own proposals. They are told to summarise what they changed and leave the decision to you.

What agents can and cannot do

They can

• Do anything the editor does: every tool is listed in the tool reference.
• Look at the result: render_frame renders exactly what the viewer shows at a moment, so the agent can check a title or framing.
• Point things out: select clips, move the playhead, set in and out, add markers and switch workspace.
• Read the history of the project, including your edits from earlier sessions, and go back to any step.
• Export video, audio, stills, captions, stems, variants and timelines for other editors.

They cannot

• Change API keys or whether agents are allowed in; those are yours to set.
• Accept their own proposed edits in review mode.
• Edit locked tracks (unless they unlock them, which they are told to do only when you ask).
• Overwrite files outside the project's export folder. They can create new files elsewhere, with the right file type.
• From the Agent panel: run shell commands, read your files or use the web.

Example prompts

• "Transcribe the interview, cut the filler words and add captions."
• "Find where they talk about pricing and make that the opening."
• "Mark the beats in the music and move the cuts onto them."
• "Make a vertical version and follow the speaker with the crop."
• "Render the frame at 1:15 and tell me if the title is readable."
• "Tighten the edit: cut pauses longer than a second."
• "Add captions to the voiceover and keep them to two short lines."
• "Look at the frame at the playhead and tell me what could look better."
• "Which script lines are over their time slot?"
• "Split this long take at its shot changes and put a marker on each shot."
• "Make 15 and 30 second vertical cuts from the best moments."

Local and cloud models

The agent is separate from the models Cue itself uses for voices, transcription, writing and images. Choose those per task in Settings → AI & models:

1. For local transcription, install whisper.cpp (brew install whisper-cpp) and download a model in Settings. Cue can also use the copy that comes with Recordly.
2. For local writing, run Ollama or LM Studio's local server and pick a model in Settings.
3. For cloud models, paste an OpenAI API key. It is stored encrypted in your macOS keychain.

Your footage never leaves your Mac unless you choose a cloud model. Editing itself needs no model at all. Agents can check what is set up with get_ai_status and tell you when a provider is missing.