Skip to content

feat: add semantic browser recording via bsk record#28

Merged
shnpd merged 4 commits into
Tencent:mainfrom
shnpd:feat/semantic-record-v3
Jul 22, 2026
Merged

feat: add semantic browser recording via bsk record#28
shnpd merged 4 commits into
Tencent:mainfrom
shnpd:feat/semantic-record-v3

Conversation

@shnpd

@shnpd shnpd commented Jul 16, 2026

Copy link
Copy Markdown
Collaborator

Capture user flows as LLM-readable trace.json (v1) with intent, page context, and semantic targets. Wire record_start/stop/await through CLI, daemon, and extension; bump CLI to 0.1.8 and extension to 0.1.4.

@BB-fat

BB-fat commented Jul 17, 2026

Copy link
Copy Markdown
Collaborator

Suggestion: Add a recording entry point in the Popup UI (generate a prompt for the Agent)

This PR’s recording capability (bsk record) is primarily meant to be invoked by an Agent via the CLI. There is currently no user-facing entry point in the Extension UI. BrowserSkill itself cannot call a large language model, and end users will not type CLI commands themselves.

Proposal: add a recording-related entry in the Popup, e.g. “Record a flow / Teach the Agent how to do this”:

  1. The user clicks the entry in the Popup
  2. The extension generates a Prompt that can be pasted directly to an Agent (with one-click copy)
  3. The Prompt must include the current BrowserID (instance_id), so the Agent can target the correct browser instance and run bsk record start …
  4. The user pastes the Prompt to the Agent → the Agent starts recording → the user demonstrates in the Agent Window → clicks Finish → the Agent receives trace.json

This would turn the collaboration model—“humans only demonstrate; the Agent starts recording and saves the textbook”—from CLI-only documentation into an in-product discovery path, lowering first-time discovery and onboarding cost.

Could this be a follow-up to this PR, or a minimal Popup entry included in this PR?

@BB-fat

BB-fat commented Jul 17, 2026

Copy link
Copy Markdown
Collaborator

RecordOverlay doesn't match our existing overlay style (ControlOverlay):

  • Dark card vs white pill + logo
  • Red rectangular button vs orange pill CTA with icon
  • No fade-in / breathe treatment

Please align with the existing bottom pill pattern before merge.

@shnpd
shnpd force-pushed the feat/semantic-record-v3 branch 2 times, most recently from 3e179ae to d56e8d0 Compare July 21, 2026 09:24
“haonan” added 2 commits July 22, 2026 11:27
Capture user actions in the Agent Window and export a record-only
trace (pages[] + navigate/click/fill/select/press steps with target
descriptors). Adds the record overlay UI aligned with the control
overlay, and wires record start/stop/await through CLI, daemon,
protocol, and extension.
…optional

Add a popup launcher + action-recording panel that builds a copyable
`bsk record start` prompt, add `--browser` targeting, and default `--url`
to https://example.com/ when omitted. Drop the popup browser-alias field.
@shnpd
shnpd force-pushed the feat/semantic-record-v3 branch from d56e8d0 to 585308d Compare July 22, 2026 05:28
@shnpd
shnpd merged commit bc8d6fa into Tencent:main Jul 22, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants