# Browser preview and inspection > In-window webview navigation, CDP attachment per session, DOM inspector injection, snapshots, console/network capture, and Stagehand integration for agent-driven browsing. - Repository: Parcha-ai/build - GitHub: https://github.com/Parcha-ai/build - Human docs: https://grok-wiki.com/public/docs/parcha-ai-build-bea5702b371b - Complete Markdown: https://grok-wiki.com/public/docs/parcha-ai-build-bea5702b371b/llms-full.txt ## Source Files - `src/main/services/browser.service.ts` - `src/main/services/cdp-proxy.service.ts` - `src/main/services/stagehand.service.ts` - `src/main/ipc/browser.ipc.ts` - `src/renderer/components/preview/BrowserPreview.tsx` - `.notes/cdp-proxy-hardening-summary.md` --- --- title: "Browser preview and inspection" description: "In-window webview navigation, CDP attachment per session, DOM inspector injection, snapshots, console/network capture, and Stagehand integration for agent-driven browsing." --- Build embeds a per-session Electron `` in `BrowserPreview`, registers each instance with `browser.service` for CDP control, and exposes a localhost CDP proxy (`cdp-proxy.service`) so Playwright and Stagehand can attach over `connectOverCDP`. Agents reach the stack through the in-process `claudette-browser` MCP server in `claude.service`, which prefers Stagehand for natural-language automation while `browser.service` and `computer-use.service` provide direct CDP paths for snapshots, selectors, console/network capture, and coordinate-based interaction. ## Architecture ```mermaid flowchart TB subgraph renderer["Renderer — BrowserPreview.tsx"] WV["webview partition persist:browser-{sessionId}"] INS["DOM inspector script GREP_INSPECTOR"] end subgraph main["Main process"] BS["browser.service"] CDP["cdp-proxy.service :9223"] SH["stagehand.service"] CU["computer-use.service"] CS["claude.service — claudette-browser MCP"] end WV -->|register-webview IPC| BS BS -->|notifyNewTarget / unregisterWebview| CDP SH -->|connectOverCDP ws://localhost:9223/devtools/browser| CDP CDP -->|debugger.attach 1.3| WV BS -->|sendCommand Runtime/Page/Input/Network| WV CS --> SH CS --> BS CS --> CU CU --> BS ``` | Layer | Module | Responsibility | |-------|--------|----------------| | UI | `src/renderer/components/preview/BrowserPreview.tsx` | Toolbar navigation, webview lifecycle, inspector injection, IPC fallbacks for snapshots/actions | | Session CDP | `src/main/services/browser.service.ts` | `sessionId` ↔ `webContentsId` map, debugger attach, navigate/click/type, console/network buffers | | External CDP | `src/main/services/cdp-proxy.service.ts` | HTTP `/json/*`, browser/page WebSockets, Target domain for Playwright | | AI automation | `src/main/services/stagehand.service.ts` | Stagehand V3 LOCAL, webview CDP or headless fallback | | Visual automation | `src/main/services/computer-use.service.ts` | 1024×768 virtual coordinates via CDP Input/Page | | Agents | `src/main/services/claude.service.ts` | MCP tools, `ensureBrowserPanelOpen`, `BROWSER_UPDATE` events | | IPC invoke | `src/main/ipc/browser.ipc.ts` | Snapshot capture, navigate, get snapshot, clear storage | On app ready, main starts the CDP proxy before agents connect: ```typescript await cdpProxyService.start(); // default port from CDP_PROXY_PORT or 9223, binds 127.0.0.1 only ``` ## In-window webview Each running session can show `BrowserPreview` when the browser panel is open (`ui.store` → `isBrowserPanelOpen`, toggled from the globe toolbar control). `persist:browser-${session.id}` — isolated cookies and storage per session. `session.lastBrowserUrl`, else last `localhost` URL from chat transcript, else `http://localhost:${session.ports?.web || 3000}`. Navigation uses `webview.loadURL` when `url` state changes; `lastBrowserUrl` is persisted on `did-navigate`. Connection failures to local dev servers (`ERR_CONNECTION_REFUSED`, etc.) retry reload up to three times at 3s intervals. The webview stays mounted when the panel is hidden (`isVisible` toggles CSS `display` only). Pop-out uses `window.electronAPI.app.openBrowserWindow()`. ## Webview registration and CDP attachment When the webview fires `dom-ready`, the renderer calls `window.electronAPI.browser.registerWebview(sessionId, webContentsId)`. Main stores bidirectional maps and notifies the CDP proxy so late-connecting Playwright clients receive `Target.attachedToTarget`. Unregister on unmount sends `browser:unregister-webview`; main detaches the debugger, calls `cdpProxyService.unregisterWebview`, and broadcasts `Target.targetDestroyed` to browser-level WebSocket clients. Automation APIs return an error until at least one webview is registered: *"Browser panel is not open. Please open the browser panel (use the globe icon in the toolbar)…"* `claude.service.ensureBrowserPanelOpen` sends `BROWSER_OPEN_PANEL` and polls up to 5s for registration when agents invoke browser tools before the user opens the panel. ## CDP proxy (Playwright / Stagehand bridge) `CdpProxyService` implements Chrome discovery endpoints and WebSocket routing: | Endpoint | Purpose | |----------|---------| | `GET /json/version` | Returns `webSocketDebuggerUrl: ws://localhost:{port}/devtools/browser` | | `GET /json/list` | Page targets keyed by `sessionId` | | `GET /json/cookies` | Cookies from `persist:browser-{firstSessionId}` | | `ws://localhost:{port}/devtools/browser` | Browser-level connection; Target domain (`setAutoAttach`, `attachToTarget`, …) | | `ws://localhost:{port}/devtools/page/{sessionId}` | Direct page CDP forwarding | Optional env override; default `9223`. If the port is in use, the service increments and retries. Security: the HTTP server listens on `127.0.0.1` only. CDP proxy hardening: - `sendCommandWithTimeout` — 30s cap on `debugger.sendCommand` to avoid hangs after detach - `debugger.on('detach')` — emits `Target.detachedFromTarget` / `Target.targetDestroyed` - `notifyNewTarget` — resolves race when Stagehand calls `setAutoAttach` before the webview exists - `unregisterWebview` — cleans sessions and closes stale page WebSockets Stagehand connects with `localBrowserLaunchOptions.cdpUrl = cdpProxyService.getBrowserWebSocketUrl()`. If no target exists after retries, it launches a headless browser (`headless: true`) that does not update the visible webview. ## Browser service (direct CDP) `BrowserService` attaches Electron `webContents.debugger` at protocol `1.3` and routes commands per session. | Method area | CDP / behavior | |-------------|----------------| | `navigate` | `Page.navigate`; IPC fallback `browser:navigate` to renderer | | `click` / `type` | `Runtime.evaluate` + `Input.dispatchMouseEvent` / `Input.insertText` | | `captureScreenshotCDP` | `Page.captureScreenshot` (PNG base64) | | `getDOM` | `Runtime.evaluate` → `document.documentElement.outerHTML` | | `enableConsoleCapture` | `Runtime.enable`; buffers last 500 `Runtime.consoleAPICalled` / `Runtime.exceptionThrown` | | `enableNetworkCapture` | `Network.enable`; buffers last 200 requests with timing | | `enableDebugging` | Both Runtime and Network domains | | `captureSnapshot` | Parallel CDP screenshot + HTML + page info; 10s IPC fallback via `browser:capture-snapshot` | | `saveSnapshotToDisk` | Writes PNG under `{tmpdir}/claudette-snapshots/` | Automation visual feedback: `emitAutomationEvent` pushes `browser:automation-event` (`start` / `position` / `end`) for click ripples and status overlays in `BrowserPreview`. Session deletion calls `cleanupSession` from `session.ipc` to drop maps, logs, and debugger state. ## DOM inspector Inspector mode toggles from the target (crosshair) toolbar button. `injectInspector` runs a large `executeJavaScript` bundle that: - Adds purple element overlay and yellow text-node highlights - Resolves React component names via `__REACT_DEVTOOLS_GLOBAL_HOOK__` or fiber keys - Builds CSS selectors walking up to `document.body` - On element click: posts `GREP_INSPECTOR:{json}` through `console-message` → renderer captures element screenshot, dispatches `grep-insert-chat` with selector/HTML/React metadata - On text click: inline editor; Enter sends `type: 'text_replacement'` to trigger `sendMessage` with an edit prompt - Shift+drag region select: grid-samples `elementsFromPoint`, attaches region screenshot + element list to chat Per-session inspector state lives in `ui.store` (`sessionInspectorActive`, `sessionSelectedElement`). ## Snapshots | Path | Trigger | Output | |------|---------|--------| | CDP | `browserService.captureSnapshot` | `{ url, screenshot, html, timestamp }` | | Renderer IPC | Main sends `browser:capture-snapshot` | `webview.capturePage()` + `outerHTML` → `browser:snapshot-captured` | | Stagehand | `stagehandService.captureSnapshot` | Playwright page screenshot + HTML + title | | User | Camera toolbar button | Viewport PNG attached to chat via `grep-insert-chat` | | MCP | `BrowserSnapshot` tool | Stagehand navigate + snapshot; image in tool result | Snapshots for agent tools are capped at 10 000 characters of HTML in the text portion of MCP responses. ## Console and network capture Console and network data are collected only after explicit enable per session (`enableConsoleCapture`, `enableNetworkCapture`, or `enableDebugging`). Retrieval APIs: - `getConsoleLogs(sessionId, { type?, limit?, since? })` - `getNetworkRequests(sessionId, { urlPattern?, method?, status?, limit? })` - `getResponseBody(sessionId, requestId)` via `Network.getResponseBody` SSH/local chrome-devtools bridging in `claude.service.executeLocalChromeDevtool` exposes `get_console_logs` against `browserService.getConsoleLogs`. Cookie read/write uses the session partition or `/json/cookies` on the proxy. ## Stagehand integration `StagehandService` wraps `@browserbasehq/stagehand` in `LOCAL` mode. Default `google/gemini-2.5-flash` (requires Google API key via settings or `GOOGLE_API_KEY` / `GOOGLE_GENERATIVE_AI_API_KEY`). Optional; copied to `process.env.ANTHROPIC_API_KEY` when set on the service. `syncCookiesFromWebview` copies Electron partition cookies into the Playwright context before navigation so agent and user sessions share auth. Operations include `navigate`, `act`, `observe`, `agent`, `extract`, `captureSnapshot`, selector `click`/`type`, and `reconnectToWebview`. Timeouts: 30s for navigate/act/observe, 60s for agent. Destroyed-target errors trigger one re-init retry. UI sync: successful tool runs call `emitBrowserUpdate` → `BROWSER_UPDATE` with screenshot and optional URL; `BrowserPreview` navigates the visible webview to match Stagehand when connected to the shared CDP target. ## Agent MCP tools (`claudette-browser`) Fresh MCP server per query (`getBrowserMcpServer`). Tool groups: `BrowserAct`, `BrowserObserve`, `BrowserAgent`, `BrowserExtractData`, `BrowserNavigate`, `BrowserSnapshot` — natural language and structured extraction. `BrowserClick`, `BrowserType`, `BrowserExtract`, `BrowserGetInfo`, `BrowserGetDOM` — CSS selectors via Stagehand page APIs. `computer` — screenshot-driven 1024×768 coordinate actions via `computer-use.service`. Remote SSH sessions can execute browser tools locally on the machine running Build (`executeLocalBrowserTool`, `executeLocalChromeDevtool`) while the agent runs elsewhere. Auto Build routing treats browser-related chat signals (`browser`, `screenshot`, `dom`, `playwright`, `stagehand`, `webview`, `localhost`, DOM attachments) as `needsBrowser` in `auto-router.service`. ## IPC and preload surface **Invoke handlers** (`browser.ipc.ts` + `preload.ts`): | Channel | Direction | Role | |---------|-----------|------| | `browser:capture-snapshot` | invoke | CDP/IPC snapshot | | `browser:navigate-to` | invoke | CDP navigate | | `browser:get-snapshot` | invoke | Last cached snapshot | | `browser:clear-storage` | invoke | Clears `persist:browser` partition storage | **Push channels** (main → renderer unless noted): | Channel | Role | |---------|------| | `browser:register-webview` / `browser:unregister-webview` | Renderer → main registration | | `browser:navigate` | Main → renderer URL load | | `browser:capture-snapshot` | Main → renderer capture request | | `browser:snapshot-captured` | Renderer → main snapshot payload | | `browser:action` / `browser:action-result` | Renderer executes click/type/script fallbacks | | `browser:automation-event` | CDP automation overlays | | `browser:update` | Stagehand screenshot/URL sync | | `browser:open-panel` | Open browser panel for session | Full channel names are listed under browser domains in the IPC channels reference page. ## Operational constraints Toggle the globe control or let an agent tool send `browser:open-panel`. The session must be `running`. After load, logs should show webview registration; `/json/list` should return a target whose `id` equals `sessionId`. Set Google API key in settings for `BrowserAct` / `BrowserAgent`; Anthropic key is optional depending on model wiring. Commands abort after 30s with a detach/timeout message. Close and reopen the browser panel to re-register the webview; restart the app if port 9223 is stuck. No webview was registered when Stagehand initialized. Reconnect via a browser tool after opening the panel, or call flows that invoke `reconnectToWebview`. `browserService.navigate` is called after Stagehand navigation when possible; if Stagehand fell back to its own browser, only the headless page updates until CDP attaches to the webview. Trash icon in toolbar: `clearStorage` IPC plus in-page `localStorage` / IndexedDB wipe, then hard reload. ## Related pages Main vs renderer boundaries, service layout, and CDP port startup in the main process. How `electronAPI.browser` maps to `ipcMain` handlers and push events. Complete `browser:*` channel catalog with invoke vs push semantics. Anthropic, Google, and other keys used by Stagehand and harness tools. How `claudette-browser` is loaded into the Claude Agent SDK per session. CDP timeouts, MCP harness sync, and dev instance issues.