# Extend agent helpers

> Edit agent-workspace/agent_helpers.py for task-specific primitives, PH_AGENT_WORKSPACE override, auto-load into the script namespace, and the tap_icon Home Screen pattern.

- Repository: ShawnPana/phone-harness
- GitHub: https://github.com/ShawnPana/phone-harness
- Human docs: https://grok-wiki.com/public/docs/shawnpana-phone-harness-bf80173a2a2e
- Complete Markdown: https://grok-wiki.com/public/docs/shawnpana-phone-harness-bf80173a2a2e/llms-full.txt

## Source Files

- `agent-workspace/agent_helpers.py`
- `src/phone_harness/helpers.py`
- `src/phone_harness/run.py`
- `SKILL.md`
- `README.md`

---

---
title: "Extend agent helpers"
description: "Edit agent-workspace/agent_helpers.py for task-specific primitives, PH_AGENT_WORKSPACE override, auto-load into the script namespace, and the tap_icon Home Screen pattern."
---

Task-specific phone primitives live in `agent-workspace/agent_helpers.py`. On import, `phone_harness.helpers` loads that file (or the path from `PH_AGENT_WORKSPACE`) and injects every public name into the helper module globals. The CLI then execs stdin Python with those names already in scope, so new functions work the same way as core helpers without changing `src/phone_harness/`.

## Ownership boundary

| Layer | Path | Who edits |
| --- | --- | --- |
| Protected core | `src/phone_harness/helpers.py` and siblings | Package maintainers; pre-imported primitives |
| Agent workspace | `agent-workspace/agent_helpers.py` | Agents and users; task-specific helpers |
| CLI namespace | `phone_harness.run` | Builds the exec dict from public helper names |

Core helpers stay thin and stable. When a flow needs a reusable pattern (Home Screen icons, a multi-step navigation, a project-specific OCR filter), define it in the agent workspace rather than forking the package.

:::files
repo/
├── agent-workspace/
│   └── agent_helpers.py    # edit here; auto-loaded
└── src/phone_harness/
    ├── helpers.py          # _load_agent_helpers() at import
    └── run.py              # exec(stdin, non-_ helpers)
:::

## Load path and `PH_AGENT_WORKSPACE`

```text
REPO_ROOT = <checkout root>          # parent of src/
AGENT_WORKSPACE = $PH_AGENT_WORKSPACE
                | default: REPO_ROOT/agent-workspace
load file     = AGENT_WORKSPACE/agent_helpers.py
```

| Item | Value |
| --- | --- |
| Env var | `PH_AGENT_WORKSPACE` |
| Default | `<repo>/agent-workspace` (resolved from `helpers.py` → `REPO_ROOT`) |
| Loaded file | `$PH_AGENT_WORKSPACE/agent_helpers.py` |
| Module name (importlib) | `phone_harness_agent_helpers` |

Canonical install keeps the tree at `~/.phone-harness`, so the default workspace is `~/.phone-harness/agent-workspace`. Point `PH_AGENT_WORKSPACE` at another directory when you want a per-project helper set without relocating the package.

<ParamField body="PH_AGENT_WORKSPACE" type="path" optional>
Directory that contains `agent_helpers.py`. Not the file path itself. Unset → `<repo>/agent-workspace`.
</ParamField>

## Auto-load rules

`_load_agent_helpers()` runs once at the end of `helpers` import:

1. Resolve `p = AGENT_WORKSPACE / "agent_helpers.py"`.
2. If the file does not exist, return silently (core helpers still work).
3. Load via `importlib.util.spec_from_file_location` / `exec_module`.
4. For each `name, value` in `vars(module)` where `name` does **not** start with `_`, assign `globals()[name] = value` on the helpers module.

The CLI then mirrors that public surface into the script namespace:

```python
from . import helpers
g = {k: v for k, v in vars(helpers).items() if not k.startswith("_")}
g["__name__"] = "__main__"
exec(code, g)
```

<Note>
Each `phone-harness` process is self-contained. Edits to `agent_helpers.py` apply on the next invocation; there is no in-process hot reload.
</Note>

### What becomes available

| Defined in `agent_helpers.py` | In `phone-harness` scripts? |
| --- | --- |
| `def tap_icon(...)` | Yes |
| `FOO = 1` | Yes |
| `_private_helper` | No (leading `_` filtered) |
| Imports used only inside the file | Not re-exported as names unless bound at module top level |

Names from agent helpers can shadow core helper names if they collide. Prefer new identifiers.

## Shipped example: `tap_icon`

Home Screen app labels are OCR-visible but not the touch target. Tapping the label center is a no-op; the icon hit box sits about **35 screen points above** the label. The stock agent helper encodes that offset.

```python
# agent-workspace/agent_helpers.py
def tap_icon(label, index=0):
    """Tap a Home-Screen app icon by its label."""
    from phone_harness.helpers import find_text, tap
    hits = find_text(label)
    if not hits:
        raise RuntimeError(f"no Home-Screen label matching {label!r}")
    h = hits[index]
    tap(h["x"], h["y"] - 35)
    return h
```

### Signature

| Parameter | Type | Default | Role |
| --- | --- | --- | --- |
| `label` | str | required | Substring match via `find_text` (case-insensitive) |
| `index` | int | `0` | Which OCR hit when several labels match |

### Behavior

| Step | Action |
| --- | --- |
| Find | `find_text(label)` → list of OCR hits with `x`, `y`, … |
| Fail | No hits → `RuntimeError(f"no Home-Screen label matching {label!r}")` |
| Tap | `tap(h["x"], h["y"] - 35)` |
| Return | The selected hit dict `h` |

### Home Screen vs in-app

| Situation | Use |
| --- | --- |
| Home Screen app icon | `tap_icon("Weather")` |
| In-app button, row, or labeled control | `tap_text("New Note")` (core helper; taps text center) |

SKILL guidance: `tap_text("Weather")` on the Home Screen hits the caption and does nothing; use `tap_icon` there.

### Example

```bash
phone-harness <<'PY'
home()
wait_stable()
tap_icon("Weather")
wait_stable()
print([o["text"] for o in ocr()][:15])
PY
```

## How to add a helper

<Steps>
  <Step title="Open the workspace file">
    Edit `agent-workspace/agent_helpers.py` under the install root (or `$PH_AGENT_WORKSPACE/agent_helpers.py` if overridden).
  </Step>
  <Step title="Define a public function">
    Use a non-`_` name. Import core primitives inside the function or at module top as needed:

    ```python
    def dismiss_keyboard():
        from phone_harness.helpers import press, wait_stable
        press("return")  # or a known Done/Hide label via tap_text
        wait_stable()
    ```
  </Step>
  <Step title="Keep side effects out of import">
    Module import runs on every `phone-harness` start. Do not open apps, tap, or capture at import time—only define callables and constants.
  </Step>
  <Step title="Verify in a one-shot script">
    ```bash
    phone-harness <<'PY'
    print("dismiss_keyboard" in dir())  # or call your helper
    print(tap_icon)  # stock helper still present
    PY
    ```
  </Step>
</Steps>

### Patterns that fit the workspace

- **Offset / geometry quirks** — like `tap_icon`’s −35 pt Home Screen correction.
- **Multi-step flows** — e.g. open app → wait → assert OCR, wrapped as one name.
- **Domain extractors** — OCR filters and parsers reused across scroll_collect calls.
- **Project conventions** — fixed app names, safe consent gates before outward actions.

Prefer composing `ocr`, `find_text`, `tap`, `wait_stable`, navigation helpers, and scroll helpers over reimplementing capture or HID input.

## Constraints and failure modes

| Symptom | Cause | Fix |
| --- | --- | --- |
| `NameError: name '…' is not defined` | Helper not loaded (wrong path, leading `_`, or syntax error at import) | Confirm file path; avoid `_` prefix; fix import-time errors |
| Stock `tap_icon` missing | Workspace file missing or replaced without re-exporting it | Restore `agent_helpers.py` or re-add the function |
| Edits seem ignored | Different `PH_AGENT_WORKSPACE` or non-editable install copy | Check env; edit the tree bound by `pip install -e` (often `~/.phone-harness`) |
| Home Screen launch fails with `tap_text` | Label is not the icon hit target | Use `tap_icon` |
| `no Home-Screen label matching …` | OCR miss, wrong page, or app name mismatch | `home()`, swipe pages, print `ocr()` / `find_text` results; pass `index` if duplicates |

<Warning>
Agent helpers run with the same privileges as core helpers: full screen capture and input on the real phone. Do not encode irreversible outward actions (send, purchase, delete) without an explicit consent gate in the calling script.
</Warning>

## Relation to core API

Core primitives stay in `helpers.py` (`connection_state`, `ocr`, `tap_text`, `scroll_collect`, `open_app`, …). Agent helpers do not replace that module; they extend the same namespace for scripts and for anything that does `from phone_harness import helpers` after import-time load.

For the full public surface of core helpers, see the Helpers API page. For `PH_AGENT_WORKSPACE` among other env vars, see Environment variables.

## Related pages

<CardGroup>
  <Card title="Helpers API" href="/helpers-api">
    Core pre-imported helpers: signatures, defaults, return shapes, and errors.
  </Card>
  <Card title="Navigate apps and type text" href="/navigate-and-type">
    home, open_app, type_text, and when to use tap_icon versus tap_text.
  </Card>
  <Card title="Environment variables" href="/environment-variables">
    PH_AGENT_WORKSPACE path override and other runtime env keys.
  </Card>
  <Card title="CLI reference" href="/cli-reference">
    stdin exec model that exposes helper names in script scope.
  </Card>
  <Card title="See, act, and verify" href="/see-act-verify">
    OCR-first loop and wait_stable verification after custom helpers.
  </Card>
  <Card title="Register as an agent skill" href="/register-agent-skill">
    Skill text that points agents at agent-workspace for task-specific edits.
  </Card>
</CardGroup>