Teach by recording

Teaching by recording is Agivar's most distinctive ability: demonstrate a workflow once, and the AI can learn it, remember it, and follow it later.

A picture beats a thousand words; a demo beats a thousand pictures. Instead of laboriously describing "click here, then click there", record a screen demo — how the mouse moves, what the keyboard typed, how the screen changed, all captured precisely. The AI doesn't have to guess.

What problem it solves

Teaching a GUI workflow with text usually gets stuck in two places:

1. Can't write it precisely — "click that submit button" — which one? what color? in which area? Text easily leaves ambiguity.
2. Can't write it completely — small steps the demonstrator takes for granted (click the field first to activate it, scroll to the bottom of the page…) often get left out.

A recording solves both at once:

• A precise action sequence — while recording, mouse clicks/drags/scrolls, keyboard input, and hotkeys are captured in sync, with a timeline. Nothing is missed.
• The matching frames — after recording, the system extracts keyframes automatically: the frame just before every action that visibly changes the screen (click, double-click, drag, pressing Enter to submit…) is kept, plus periodic samples during quiet stretches. The AI can both "see" what the screen looked like before an action and compare what it became after.
• Your extra explanations — add text annotations (Alt+I on Windows, Cmd+Shift+I on macOS), or use the green waveform button for voice annotations; these are merged into the recording's timeline.
• Distilled into reusable know-how — the recording is processed into a structured "operation description" (Overview / Initial state / Step-by-step / a "stage result" after each step), which goes straight into the memory base; the entry also records the ID and key clips of the "demo recording", so Task Mode can later ask targeted questions about details in it.

The full flow

Step 1: Start recording

1. Start a new conversation and turn on Teach Mode.
2. Click Record → Start screen recording in the input bar, or click Start recording from Select recording file.
3. A floating recorder control bar appears on screen — semi-transparent, draggable, always on top, with a timer, voice annotation, text annotation, history, pause / continue, and End.

The control bar isn't recorded

By default, the control bar and annotation box are excluded from the captured footage, so the video contains whatever is behind them. They only become visible in recordings when Full Screenshot Mode is enabled.

Step 2: Demonstrate (add text or voice explanations)

Just go do the operations you want to demonstrate. Recommendations:

• Slow down a bit — leave a small pause between steps so keyframes can reliably catch the pre-action screen.
• Demonstrate one complete workflow at a time — from a clear starting point (e.g. "browser is open") to a clear endpoint ("coins given successfully"). Don't record several unrelated things together.

When you want to add a text note for a step, click Annotate on the control bar or use the shortcut (Alt+I on Windows, Cmd+Shift+I on macOS). An annotation input opens beside the floating bar:

• Type your note, e.g. "wait for the page to fully load before clicking here, otherwise the button won't respond".
• Enter sends, Shift+Enter inserts a newline, Esc cancels.
• The note is merged into the recording's action timeline at the moment it happened.
• While the annotation input is open, what you type into it is not recorded as an action (the shortcut that opened it, plus the following modifier-key releases, are filtered out too).

If you want to talk while demonstrating, click the green waveform button or use the shortcut (Alt+D on Windows, Cmd+Shift+D on macOS) to start voice annotation; click it again or use the same shortcut again to stop. Agivar turns your speech into timestamped text notes and shows them in the annotation history.

When should you annotate? For anything "you can't tell from the footage": why you waited, why you picked this and not that, what the step is for, what the gotcha is.

Step 3: End recording

Click End on the control bar. If you need to pause, click Pause, then Continue when ready; paused time is not captured and does not count toward the recording length.

After stopping, the system packages what you recorded and starts background processing — this step needs the network, and how long it takes depends on the recording length, typically tens of seconds to a few minutes. During processing:

• you can keep typing in the input bar and add other attachments;
• but the Send button is temporarily disabled until processing finishes, so you can't send the message (with the recording) yet;
• the recording attachment shows elapsed time / estimated time remaining;
• if you do not want to send this recording with the current message, click X on the attachment card. The recording can still be found from Select recording file, and you can delete it from history if you truly want to discard it.

Step 4: Preview (optional)

After processing (or before), open the recording preview dialog to play back what you just recorded and confirm the demo is complete:

• ▶ Play / ⏸ Pause / ⟲ Replay.
• Not happy with it? Delete the attachment and re-record.

Step 5: Send it — over to the Teach Agent

Once processing is done and the Send button is back, send the message (you can also type a few notes alongside it). When the Teach Agent receives it:

• it first sees a summary of the recording and the structured operation description — so it knows the whole workflow without any extra lookups;
• if it needs more detail from inside the recording, it can ask targeted questions about the recording content (internally via a dedicated "video Q&A" capability, e.g. "what does the popup in that frame say");
• when something is incomplete or ambiguous, it asks you to confirm.

Step 6: Distilling into a memory entry

The Teach Agent organizes the workflow into the memory base (filed as platform/topic). The entry looks roughly like:

# Overview
This demonstrates the user opening the "非十科技" video on Bilibili and giving it two coins.

# Initial state
The browser is open.

# Steps

## Step 1: Open a new tab and navigate to Bilibili
1. Click the "+" button to the right of the browser tab bar to open a new tab.
2. Click the address bar at the top of the browser to activate input.
3. Type bilibili.com in the address bar.
4. Press Enter to navigate.
Stage result: successfully reached the bilibili.com homepage, showing the recommended video list and the top search bar.

## Step 2: …
…
Stage result: …

# Related demo recording
Recording ID: 00046; summary: …; key clips: …

A few details to note:

• Steps are split into several abstract steps, each with its concrete operations underneath, and each ends with a "stage result" — when Task Mode follows along it can use this to judge whether the current stage is complete.
• On-screen targets are written unambiguously — "click the blue 'Confirm' button at the bottom-right of the popup", "click the magnifier icon to the right of the search box", rather than just "click confirm".
• The entry records the demo recording's ID at the end, so later, when Task Mode runs a related task, it can pull up this recording and ask about details.

What actually happens during processing

For your understanding, "background processing" is roughly four stages (you don't do anything):

1. Upload keyframes — the keyframes picked from the recording are uploaded to the cloud.
2. Annotate frame by frame — the AI looks at these keyframes, combines them with the mouse/keyboard action timeline, and produces a detailed description with frame references.
3. Generate a summary — condenses the detailed description into a short video summary.
4. Generate the operation description — produces, in the "Overview / Initial state / Step-by-step / stage result" structure, the "operation description" that can go straight into a memory entry.

The recording's raw frames and mouse/keyboard events stay locally under ~/.agivar/; the processed result (keyframes, summary, operation description) is stored in the cloud, tied to your account.

Voice annotations (optional)

The green waveform button on the control bar records voice annotations. Turn it on and speak naturally; Agivar transcribes your speech in real time and places the recognized text into the annotation history at the right timestamps. When generating the operation description, the AI prefers the terminology and structure you spoke out loud, so for longer workflows you do not need to type every note manually.

The first time you use it, your system may request microphone permission. If microphone access is unavailable, screen recording continues normally, just without voice annotations.

Best practices for teaching by recording

• ✅ One workflow per recording, with clear start and end.
• ✅ Slow down, leave a pause between steps.
• ✅ Use text or voice annotations at key moments to explain the "why" — that's the information the recording can't give and only you can.
• ✅ Set up the environment before demonstrating (log in / open what's needed beforehand), and state it clearly in the "Initial state".
• ✅ Preview once before sending.
• ⚠️ Don't record sensitive screens like CAPTCHAs or passwords — when a step like that is truly needed, pause there, add a text annotation such as "user logs in here", and skip the actual input.
• ⚠️ With multiple monitors, the primary monitor is recorded — do the demo on the primary screen.

Screen Recording and Voice Notes

Screen recordings can be attached to Agivar messages. They are most commonly used in Teach Mode as demonstrations of how to complete a task, but you can also send a recording in Task Mode to give the AI a process to reference. A floating control bar appears while recording. Use it to pause, continue, or end the recording, and to add text or voice annotations.

Here, "voice recording" mainly refers to voice annotations: as you speak during a screen recording, Agivar converts your speech into timestamped text notes in real time and sends them with the recording for the AI to understand.

Start recording

1. If you want to teach Agivar a workflow, start a new conversation and switch to Teach Mode first.
2. Click Record in the input bar.
3. Choose Start screen recording from the menu, or open Select recording file and click Start recording in the selector.
4. On first use, follow the system prompts to grant screen recording, accessibility, input monitoring, microphone, and other required permissions.
5. A floating control bar appears near the bottom-right of the screen and displays REC 00:00/10:00. You can then begin the demonstration.

tip

The recording captures the primary display, mouse and keyboard operations, and annotations you submit. If you use multiple displays, move the window you want to demonstrate to the primary display.

How to use the floating control bar

Control	What it does
Six-dot handle / empty area	Drag the control bar. When released, it automatically returns to the visible screen area if necessary.
REC, red dot, and timer	Shows the elapsed recording time and limit. While paused, the red dot stops blinking and the timer freezes.
Green waveform button	Starts / stops voice annotation. The shortcut is Alt+D on Windows and Cmd+Shift+D on macOS. Recognized text automatically appears in the annotation history.
Annotate button	Opens the text annotation input. The shortcut is Alt+I on Windows and Cmd+Shift+I on macOS. Enter sends, Shift+Enter inserts a newline, and Esc closes the input.
History count	Opens / closes the annotation history. The arrow next to the title only expands or collapses the history content. Sent annotations can be edited or deleted.
Pause / Continue	While paused, Agivar does not capture frames or operations, and paused time does not count toward the recording length. Continuing resumes from the time before the pause.
End	Stops the recording and starts background processing.

Hover over a button to see its tooltip. If voice annotation is active when you click Pause, Agivar first saves the current voice draft. When recording continues, the voice annotation that was active before the pause resumes automatically.

Are floating widgets captured?

By default, no. The recording control bar, annotation box, task progress overlay, and similar floating widgets are excluded from AI screenshots and recordings, so the final capture contains whatever is behind them.

If you enable Streamer Mode under Settings → Advanced, these floating widgets become visible in recordings and AI screenshots, which is useful for demonstrations or livestreams. Disable Streamer Mode to restore the default hidden behavior.

Text annotations

Text annotations are useful for short notes, such as:

• "Wait for the page to finish loading before clicking here."
• "Choose the second option because we only process paid orders."
• "The user enters the password and CAPTCHA here."

Click Annotate, or use the shortcut to open the input: Alt+I on Windows and Cmd+Shift+I on macOS. The annotation is inserted into the action timeline at the recording time when it was submitted, and the AI treats it as your explanation when generating the operation description.

Voice annotations

Voice annotations are useful when you want to explain a process while demonstrating it, especially for longer workflows where frequent typing would be inconvenient:

1. Click the green waveform button, or use the shortcut to start: Alt+D on Windows and Cmd+Shift+D on macOS.
2. Speak your explanation normally.
3. As Agivar recognizes the speech, it shows a live draft in the annotation history and automatically divides it into annotations at pauses.
4. Click the green waveform button again, or use the same shortcut again, to stop.

The first time you use voice annotations, the system may request microphone permission. If permission is unavailable or the microphone cannot be used, screen recording continues normally, but this recording will not have voice annotations.

note

The microphone in the input bar converts speech into chat input. The green waveform button in the recording widget converts speech into recording annotations. They serve different purposes.

What happens after you click End?

When you click End, the control bar closes, a recording attachment card appears above the input bar, and background processing begins.

• You can keep typing and add other attachments while processing continues.
• The Send button is temporarily disabled and becomes available after processing finishes.
• The attachment card shows processing progress, elapsed time, and estimated remaining time.
• Click the attachment thumbnail to preview the recording with play, pause, and replay controls.
• If you do not want to send the recording with the current message, click X on the attachment to remove it. Background processing continues, and you can still find the recording later under Select recording file.

If you are sure you do not want the recording, delete it under Select recording file or in Recording Files management. A processed recording can also be attached to a later message from Select recording file.

Recording Files

Recording Files can be opened in two ways. When opened from the chat input bar, it is mainly used to attach recordings to the current message. When opened from Settings, it is mainly used to manage saved recordings. Both locations show the same recordings, but each is suited to different operations.

How to open it

• Select recording file dialog: click Record → Select recording file in the chat input bar. Use it to attach saved recordings to the current message.
• Recording Files management: click the gear icon in the window, then open Settings → Advanced Settings → Recording Files. Use it to rename, delete, and batch-delete recordings.

Search and preview

If you have many recordings, you can search by recording name. Clear the search box to return to the full list.

Recordings are sorted in reverse chronological order, so the most recent recordings appear near the top and are easier to find.

Before sending or deleting a recording, double-click it to preview and confirm that the content is complete and does not contain passwords, verification codes, private keys, or other sensitive information.

If a recording is marked Recording interrupted, the preview plays only the part that was saved.

Start a new recording

If none of the saved recordings is suitable, you can start a new recording directly from the current window. This is useful when you intended to use an older recording but realize you need to demonstrate the workflow again.

When recording begins, the current window closes and the recording control bar appears. You can reopen Recording Files while recording to continue finding and attaching existing recordings.

Rename and delete

You can rename and delete recordings in either place:

• Select recording file dialog: hover over a recording to show the Edit and Delete buttons. This is suitable for quickly renaming or deleting a single recording.

• Recording Files management: an Edit button appears next to each recording. Select recordings and click Delete. This is suitable for managing recordings and makes batch deletion easier.

When renaming, press Enter or click outside the input box to save. Press Esc to cancel. Names can be up to 18 characters.

Agivar displays a confirmation prompt before deletion. Once confirmed, the recordings are removed from Recording Files.

warning

Make sure you no longer need a recording before deleting it. Deleting a recording that is still processing also stops the remaining processing work.

Special statuses

Status	Recommendation
Processing / Queued	You can attach it to the input bar first and wait for processing to finish before sending. If you are sure you do not need it, you can delete it.
Not processed	If you still want to use the recording, attach it to the input bar and let Agivar process it again.
Recording interrupted	Preview the saved clip. If later steps are missing, continue recording from the Select recording file dialog.
Local file missing	The corresponding file cannot be found on this computer, so the recording usually cannot be used here.

Select Recording File dialog

The Select recording file dialog is mainly used to attach saved recordings to the current message.

Attach to a message

1. Select the recording you want to send.
2. If you need several recordings, you can continue selecting other recordings while one is uploading.
3. Click Select.

You can select up to 5 recordings at a time.

Processed recordings can be sent with the message. Recordings that are still processing must finish before the message can be sent.

If you selected the wrong recording, click the selected recording again to deselect it.

Continue or handle an interrupted recording

If a recording is marked Recording interrupted and later steps are missing, click the green continue recording button on the recording to continue it.

Continuing appends new actions to the original recording. The total recording length is limited to 10 minutes, and a recording cannot be continued after reaching the limit. If a demonstration cannot be completed within 10 minutes, record several videos and upload them together. You can select up to 5 recordings at a time.

Recording Files management

Recording Files management is mainly used to manage saved recordings and is especially useful for deleting several recordings.

Batch delete

To delete several recordings at once, select the recordings you want to delete, then click Delete.