# CLI browser and computer reference

> Browser automation commands (navigation, interaction, cookies, storage, capture) and `computer` desktop accessibility commands with capability flags.

- Repository: stablyai/orca
- GitHub: https://github.com/stablyai/orca
- Human docs: https://grok-wiki.com/public/docs/stablyai-orca-2036d532bf1c
- Complete Markdown: https://grok-wiki.com/public/docs/stablyai-orca-2036d532bf1c/llms-full.txt

## Source Files

- `src/cli/specs/browser-basic.ts`
- `src/cli/specs/browser-advanced.ts`
- `src/cli/specs/computer.ts`
- `src/cli/specs/orchestration.ts`
- `src/cli/handlers/computer.ts`
- `src/cli/handlers/browser-capture.ts`
- `skills/computer-use/SKILL.md`

---

---
title: "CLI browser and computer reference"
description: "Browser automation commands (navigation, interaction, cookies, storage, capture) and `computer` desktop accessibility commands with capability flags."
---

The Orca CLI exposes two automation surfaces on top of the Orca runtime RPC layer: **browser** commands drive Orca’s embedded Chromium tabs through an `agent-browser` bridge, and **`orca computer`** commands drive native desktop apps through a platform computer-use provider. Command signatures live in `src/cli/specs/browser-basic.ts`, `src/cli/specs/browser-advanced.ts`, and `src/cli/specs/computer.ts`; handlers resolve targets in `src/cli/selectors.ts` and call `browser.*` or `computer.*` RPC methods.

```mermaid
sequenceDiagram
  participant CLI as orca CLI
  participant RT as Orca runtime RPC
  participant BR as agent-browser bridge
  participant CU as computer-use provider

  CLI->>RT: browser.snapshot / browser.click / ...
  RT->>BR: CDP session + queued command
  BR-->>RT: BrowserSnapshotResult / action result
  RT-->>CLI: JSON or formatted text

  CLI->>RT: computer.getAppState / computer.click / ...
  RT->>CU: accessibility + input + screenshot
  CU-->>RT: ComputerSnapshotResult / ComputerActionResult
  RT-->>CLI: JSON or formatted text
```

## Prerequisites

| Requirement | Verification |
| --- | --- |
| Orca desktop app or `orca serve` runtime reachable | `orca status --json` |
| Browser tab in a worktree (for browser commands) | `orca tab list --json` or `orca tab create --url <url>` |
| Computer-use permissions (macOS) | `orca computer permissions --json` |
| Provider feature support | `orca computer capabilities --json` |

Browser commands default to the **active tab** in the worktree enclosing the CLI’s current directory. Computer commands default the same worktree context unless `--session` is used.

## Global flags

All commands in this reference accept the global CLI flags from `GLOBAL_FLAGS` in `src/cli/args.ts`:

| Flag | Purpose |
| --- | --- |
| `--help` | Show command-specific help (same as `orca help <command>`) |
| `--json` | Emit the full RPC success envelope as JSON |
| `--environment <selector>` | Target a saved remote runtime |
| `--pairing-code <code>` | Pair to a runtime for this invocation |

Add `--worktree <selector>` on browser and most computer commands to pin a worktree. Selectors follow the grammar documented on the selectors page (`id:`, `path:`, `branch:`, `active`, `current`). On a **remote** runtime, `active` / `current` are rejected because they resolve from the client machine’s cwd, not the server.

## Browser targeting

| Flag | Applies to | Behavior |
| --- | --- | --- |
| `--worktree <selector>` | Most browser commands | Resolves tab scope; `all` on `tab list` / `tab current` lists across worktrees |
| `--page <browserPageId>` | Tab-targeted commands (see `supportsBrowserPageFlag`) | Targets a specific tab globally; optional `--worktree` validates ownership |
| *(default)* | Local CLI inside a managed worktree | Auto-resolves enclosing worktree from cwd |
| *(default)* | Remote CLI | Omits worktree; runtime uses server-side focus |

Interaction commands take **`--element <ref>`** where `ref` comes from the latest `orca snapshot` (for example `@e1`, `@e2`). Refs are valid only for the snapshot that produced them.

### Snapshot → act loop

<Steps>
<Step title="Capture state">
Run `orca snapshot` to get an accessibility tree, URL, title, and `refs` array.
</Step>
<Step title="Act on a ref">
Use `orca click --element @e3`, `orca fill --element @e2 --value "..."`, or similar.
</Step>
<Step title="Re-snapshot after navigation">
Re-run `orca snapshot` after navigation, DOM changes, or tab switches — stale refs return element errors.
</Step>
</Steps>

<RequestExample>

```bash
orca snapshot --json
orca click --element @e1 --json
orca wait --text "Dashboard" --timeout 30000 --json
```

</RequestExample>

## Browser command inventory

### Observation and navigation

| Command | Usage summary |
| --- | --- |
| `snapshot` | `orca snapshot [--worktree <selector>] [--json]` |
| `screenshot` | `orca screenshot [--format png\|jpeg] [--worktree <selector>] [--json]` |
| `full-screenshot` | Full-page capture beyond viewport |
| `goto` | `orca goto --url <url>` — 60s RPC timeout for network idle |
| `back` / `forward` / `reload` | History navigation |
| `eval` | `orca eval --expression <js>` in page context |
| `wait` | Wait for selector, text, URL pattern, load state, JS (`--fn`), visibility (`--state hidden\|visible`), or `--timeout` ms |
| `pdf` | Export active tab as PDF (base64 in JSON) |
| `scroll` | `orca scroll --direction up\|down [--amount <pixels>]` |
| `scrollintoview` | `orca scrollintoview --element <ref>` |

`wait` uses an extended RPC timeout (default 60s client-side, plus `--timeout` when set) so slow page conditions are not misreported as runtime unreachable.

### Element interaction

| Command | Required flags | Notes |
| --- | --- | --- |
| `click` | `--element <ref>` | |
| `dblclick` | `--element <ref>` | |
| `hover` | `--element <ref>` | |
| `fill` | `--element`, `--value` | Clear + fill |
| `type` | `--input <text>` | Types at current focus (no element) |
| `inserttext` | `--text` | Insert without key events |
| `select` | `--element`, `--value` | Dropdown |
| `check` / `uncheck` | `--element` | Checkbox/radio |
| `focus` / `clear` / `select-all` | `--element` | |
| `keypress` | `--key <name>` | Enter, Tab, Escape, ArrowDown, … |
| `drag` | `--from <ref>`, `--to <ref>` | |
| `upload` | `--element`, `--files <path,...>` | Comma-separated paths |
| `get` | `--what <property>` | `text`, `html`, `value`, `url`, `title`, `count`, `box`; optional `--element` |
| `is` | `--what <state>`, `--element` | `visible`, `enabled`, `checked` |
| `find` | `--locator`, `--value`, `--action` | Locators: `role`, `text`, `label`; optional `--text` |
| `highlight` | `--selector <ref>` | Visual highlight |
| `download` | `--selector`, `--path` | Trigger download to path |

### Mouse

| Command | Flags |
| --- | --- |
| `mouse move` | `--x`, `--y` |
| `mouse down` / `mouse up` | Optional `--button left\|right\|middle` |
| `mouse wheel` | `--dy` required; optional `--dx` |

### Tabs and session profiles

| Command | Purpose |
| --- | --- |
| `tab list` | List tabs; `--show-profile` adds profile column; `--worktree all` |
| `tab show` | `orca tab show --page <id>` |
| `tab current` | Active tab for worktree |
| `tab switch` | `--index <n>` or `--page <id>`; optional `--focus` to surface browser pane |
| `tab create` | Optional `--url`, `--profile <id>` |
| `tab close` | Optional `--index` |
| `tab profile list` | All session profiles |
| `tab profile create` | `--label <name>`; `--scope isolated\|imported` |
| `tab profile delete` | `--profile <id>` |
| `tab profile set` | `--page` or `--worktree` + `--profile` |
| `tab profile show` / `use-default` / `clone` | Profile binding and cloning |

Profile scopes are `default`, `isolated`, and `imported` (see `BrowserSessionProfileScope` in shared types).

### Cookies, storage, and environment

| Group | Commands |
| --- | --- |
| Cookies | `cookie get [--url]`, `cookie set` (`--name`, `--value`, `--domain`, `--path`, `--secure`, `--httpOnly`, `--sameSite`, `--expires`), `cookie delete` |
| Viewport | `viewport --width --height [--scale] [--mobile]` |
| Geolocation | `geolocation --latitude --longitude [--accuracy]` |
| Offline / device / headers | `set offline [--state on\|off]`, `set device --name`, `set headers --headers <json>`, `set credentials --user --pass`, `set media [--color-scheme] [--reduced-motion]` |
| Clipboard | `clipboard read`, `clipboard write --text` |
| Dialogs | `dialog accept [--text]`, `dialog dismiss` |
| Storage | `storage local get\|set\|clear`, `storage session get\|set\|clear` (each needs `--key`; set needs `--value`) |

### Capture and interception

| Command | Behavior |
| --- | --- |
| `capture start` / `capture stop` | Toggle console + network event capture |
| `console` | Show captured console entries; `--limit <n>` |
| `network` | Show captured network log; `--limit <n>` |
| `intercept enable` | Pause matching requests; `--patterns <glob,...>` |
| `intercept disable` | Disable interception |
| `intercept list` | List paused requests (`[id] METHOD url`) |

<Warning>

Per-request `intercept continue` / `intercept block` are not exposed in the CLI yet — the advanced spec notes they depend on future `agent-browser` support. Today interception is URL-pattern based only.

</Warning>

### Passthrough

```bash
orca exec --command "<agent-browser subcommand>" [--worktree <selector>] [--json]
```

`exec` forwards arbitrary `agent-browser` CLI syntax against the resolved tab. Prefer typed top-level `orca` commands when they exist — they have explicit flags, timeouts, and structured JSON types.

## Computer command reference

Computer commands are grouped under `orca computer <subcommand>`. Action commands return a fresh **`ComputerActionResult`** (snapshot + optional screenshot + action metadata) so agents can verify UI state without a separate `get-app-state` call.

### Capabilities

```bash
orca computer capabilities [--json]
```

Human output lists provider name, platform, protocol version, and enabled support groups. The JSON shape is `ComputerProviderCapabilities`:

| Group | Fields | Meaning |
| --- | --- | --- |
| `supports.apps` | `list`, `bundleIds`, `pids` | App discovery modes |
| `supports.windows` | `list`, `targetById`, `targetByIndex`, `focus`, `moveResize` | Window targeting |
| `supports.observation` | `screenshot`, `annotatedScreenshot`, `elementFrames`, `ocr` | Snapshot extras |
| `supports.actions` | `click`, `typeText`, `pressKey`, `hotkey`, `pasteText`, `scroll`, `drag`, `setValue`, `performAction` | Allowed mutations |
| `supports.surfaces` | `menus`, `dialogs`, `dock`, `menubar` | Platform UI surfaces |

Call `capabilities` before scripting portable automation — providers on macOS, Linux, and Windows differ.

### Discovery and permissions

| Command | Usage |
| --- | --- |
| `computer permissions` | macOS only — opens Orca Computer Use permission setup; no-op message on other platforms |
| `computer list-apps` | `orca computer list-apps [--worktree <selector>] [--json]` |
| `computer list-windows` | `orca computer list-windows --app <name\|bundle\|pid:N> [--worktree \| --session]` |
| `computer get-app-state` | Accessibility tree + screenshot for one app/window |

### App and window selectors

<ParamField body="--app" type="string" required>
Target application. Prefer bundle IDs from `list-apps` (for example `com.spotify.client`). Human names work when unambiguous. Use `pid:<number>` only when bundle/name matching fails.
</ParamField>

<ParamField body="--window-id" type="number">
Target a specific window by platform window id.
</ParamField>

<ParamField body="--window-index" type="number">
Target by zero-based index from `list-windows`.
</ParamField>

<ParamField body="--session" type="string">
Computer session id instead of worktree context (used with `list-windows` and action commands).
</ParamField>

### Action commands

| Command | Targeting | Notable flags |
| --- | --- | --- |
| `computer click` | `--element-index <n>` **or** `--x` + `--y` | `--click-count`, `--mouse-button left\|right\|middle` |
| `computer perform-secondary-action` | `--element-index`, `--action <name>` | Uses accessibility action names from snapshot |
| `computer scroll` | element index **or** coordinates | `--direction up\|down\|left\|right`, `--pages` |
| `computer drag` | `--from-element-index` + `--to-element-index` **or** coordinate pairs | |
| `computer type-text` | literal keystrokes | `--text` or `--text-stdin` |
| `computer press-key` | xdotool-style key names | `--key Return`, etc. |
| `computer hotkey` | platform-aware combos | `--key CmdOrCtrl+A` |
| `computer paste-text` | clipboard paste path | `--text` or `--text-stdin` |
| `computer set-value` | direct AX value write | `--value` or `--value-stdin` (empty allowed) |

### Observation flags on actions

| Flag | Effect |
| --- | --- |
| `--restore-window` | Ask provider to restore minimized/off-screen windows before acting |
| `--no-screenshot` | Skip screenshot capture in the returned state (faster; tree-only) |

Coordinates are **window-local** — use positions from the latest snapshot/screenshot for the same window.

### Snapshot–act loop (desktop)

<Steps>
<Step title="List apps">
`orca computer list-apps --json`
</Step>
<Step title="Capture app state">
`orca computer get-app-state --app <bundle> --json` — read `snapshot.treeText` and `element-index` values.
</Step>
<Step title="Perform one action">
`orca computer click --app <bundle> --element-index 42 --json` (or `set-value`, `scroll`, …).
</Step>
<Step title="Inspect action result">
Action responses include an updated snapshot; re-fetch with `get-app-state` if the UI changes asynchronously.
</Step>
</Steps>

<Element indexes go stale after navigation, focus changes, scrolling, or re-renders — refresh state before reusing an index.

### JSON screenshots

With `--json`, screenshot bytes are written to a temp file under the system temp directory (`orca-computer-use/`) and the JSON payload replaces inline `data` with `screenshot.path`, `dataOmitted: true`, and `expiresAt` (24h TTL). Read pixels from `path`, not from the RPC body.

### Computer error codes

| Code | Typical cause |
| --- | --- |
| `app_not_found` | Bad `--app`; rerun `list-apps` |
| `element_not_found` | Stale `--element-index` |
| `window_not_found` / `window_stale` | Invalid window target |
| `unsupported_capability` | Action not in `capabilities` |
| `permission_denied` | macOS Accessibility / Screen Recording |
| `action_not_supported` / `value_not_settable` | Wrong action for element role |
| `invalid_argument` | Conflicting flags (for example `--text` and `--text-stdin`) |
| `screenshot_failed` | Hidden/minimized/off-screen window |

## JSON output shape

`--json` prints the full `RuntimeRpcSuccess` envelope via `printResult` in `src/cli/format.ts`:

<ResponseField name="ok" type="boolean">
Always `true` on success paths.
</ResponseField>

<ResponseField name="id" type="string">
RPC request id.
</ResponseField>

<ResponseField name="result" type="object">
Command-specific payload (`BrowserSnapshotResult`, `ComputerActionResult`, etc.).
</ResponseField>

Failures throw `RuntimeClientError` / `RuntimeRpcFailureError` with `code` and `message` (for example `selector_not_found`, `invalid_argument`).

## Command help

Every spec entry supports:

```bash
orca help snapshot
orca snapshot --help
```

Help text is generated from `CommandSpec` records in `src/cli/specs/` — usage strings there are authoritative when flags differ between help groupings.

## Related pages

<CardGroup>
<Card title="Browser automation" href="/browser-automation">
Conceptual guide to Orca’s embedded browser, accessibility snapshots, and agent-browser integration.
</Card>
<Card title="CLI core reference" href="/cli-core-reference">
`open`, `status`, `worktree`, `terminal`, and other non-browser command groups.
</Card>
<Card title="Selectors and JSON output" href="/selectors-json-output">
Worktree selector grammar, `--json` errors, and remote-runtime resolution rules.
</Card>
<Card title="Runtime environments" href="/runtime-environments">
Target a remote Orca runtime with `--environment` or `--pairing-code`.
</Card>
<Card title="CLI scripting workflow" href="/cli-scripting-workflow">
End-to-end agent automation combining terminals, worktrees, and browser/computer commands.
</Card>
</CardGroup>