Skip to content
Menu

How it works

Write a plan

The Markdown grammar qaren walks, and what a check reads from the screen.

Your agent writes and runs this; here is what it means.

A plan is a Markdown file. qaren check and qaren pr take it with --plan-file.

md
# Test App smoke

## QA

### Onboarding to the tasks tab

1. Tap "onboarding-skip"
2. Tap "onboarding-done"
3. Wait for "Welcome" to appear
✓ "Welcome"
4. Tap "tab-tasks"
5. Wait for "task-header" to appear
✓ "Tasks ("

Structure

  • When the plan has a ## QA section, only the lines under it are read, up to the next ## heading. Without one, the whole file is read.
  • Each ### heading starts a named block. Steps before the first ### heading form a block named after the plan's # title, or plan when there is none.
  • Other headings are skipped. Every other non-empty line is a numbered step (1. or 1)), a check that starts with ✓ or ✔ (optionally after a - or * bullet), or a Starts on: line. Any other line refuses the plan with "not a numbered step or a ✓ line".
  • HTML comments are ignored.
  • A block with no steps, two blocks with the same title, or two titles that produce the same block ID refuse the plan.

Open a block that continues a journey with starts on: <testID or text> before its steps. The value is a literal testID or visible text, with optional quotes. The start line alone does not count as a block's steps. For start waits, refusals and replay, see the authoritative saved-block contract.

The block ID is the heading in lowercase, with every run of characters outside a-z and 0-9 replaced by a hyphen. It names the saved block file.

Linked Git worktrees can share saved steps through a .qaren/actions link to the main copy; see the saved-block contract.

Steps

KindVerbsExample
Presstap, press, click, open, select, choose, toggle, hit, go toTap "tab-tasks"
Filltype, enter, input, write, fill inType "Buy milk" into "task-input"
Fillthe same verbs, target firstFill "task-input" with "Buy milk"
Scrollscroll, scroll down, scroll up, optionally until a targetScroll down until "Settings"
Waitwait for, wait until, optionally ending to appear or is visibleWait for "Welcome" to appear
Backback, go back, navigate back, press back, tap back buttonGo back
Dialogaccept, allow, dismiss, deny, decline, cancel with dialog, prompt, alert or permissionAccept the permission dialog
Echoexpect input NAME on screen, expect input NAME on this screenExpect input USERNAME on screen

For plain-scroll no-op results and when to use Scroll down until "Settings", see the plain scroll contract.

A fill needs its text in quotes, or a runtime input. For iOS entry methods, already-set skipping, handler recovery, verification and the (any method) and (always retype) labels, see the authoritative Fill ladder contract. A dialog step needs exactly one unconditional action: Accept the dialog if it appears and Do not dismiss the alert refuse.

For real-tap verification, including already-selected controls and selection toggles, see the press evidence contract.

Runtime inputs

A fill can take its value from the environment instead of the plan, so a login block with staging credentials can be saved and replayed:

md
1. Fill "staging-user" with input USERNAME
2. Fill "staging-password" with input PASSWORD
3. Expect input USERNAME on screen
4. Tap "sign-in"

The name is 1–32 characters of A-Z, 0-9 and _, starting with a letter. qaren reads the value of QAREN_INPUT_<NAME> once when the run starts, here QAREN_INPUT_USERNAME and QAREN_INPUT_PASSWORD. Resolve the value before you run qaren, for example with your password manager's CLI; qaren does not read .env files or resolve references.

Every input has one of three tiers:

  • SECRET (the default; passwords, OTP codes, API keys, payment data). A runtime input with no annotation, or with (secret), declares this tier.
  • TEST-PII (staging test e-mails, test addresses and similar test-account data). Opt in per Fill with (test-pii); (non-secret) is the older spelling of the same tier.
  • PUBLIC: literal text in the plan, such as Fill "Search" with "shoes".
md
1. Fill "email-field" with input TEST_EMAIL (test-pii)
2. Fill "password-field" with input PASSWORD

For secure and unreadable field protection, private-label handling, SECRET collisions, readback, log redaction and media withholding, follow the CLI guide's runtime-input tier contract.

Declared-value redaction only masks the values you declared. Other personal data the app shows next to them, such as the account holder's name, phone number or address, is not covered and can appear in the logs and failure text of a TEST-PII run. Use TEST-PII only for accounts whose on-screen data is test data.

TEST-PII also relaxes execution hash and echo protection for that input. Its labels, identifiers and types keep their original hash operands, including at the filled field; an Expect input TEST_EMAIL on screen line is unnecessary and does not constant-mask its echo. Every use of a name across the plan and configured login block must agree; conflicts refuse before device selection. When values overlap, SECRET inputs win, so a TEST-PII alias or echo declaration cannot exempt a SECRET value. Input tiers have no global, config or environment-based opt-in, and unused environment variables declare no policy. The separate video policy controls video publication without changing input tiers or text masking.

For saved-slot encoding, run-wide policy hashes and refreshing policy-stale login blocks, follow the CLI guide's runtime-input persistence contract. Never mark real credentials TEST-PII: qaren cannot identify a production backend or prove that a value is test data, and a misclassified secret is read and only redacted by value.

Expect input NAME on screen (also on this screen, with case-insensitive keywords) declares an expected label echo; it does not assert that the value is visible. Its row passes without capturing the screen or dispatching an action. The name must follow the input-name rules above and be used by a fill in this plan. Malformed declarations and names unused by any fill refuse PLAN_UNPARSEABLE, without falling back to Jev. There is no no longer form or heading-level declaration.

  • Every input the plan or the configured login block names must be set and non-empty, even when login recovery never happens. Otherwise the run refuses PLAN_INPUT_MISSING, naming every missing input, before a device is selected.
  • At most 64 distinct inputs may be required. A value is one line of printable text, at least three characters after trimming, at most 4096 bytes and 64 KiB across all required inputs. Whitespace is kept exactly. A value that looks like an unresolved reference, starting with a scheme such as op:// or vault: or containing ${, $( or {{, refuses CORE_REFUSED. Invalid values refuse with runtime inputs are invalid.
  • The value is masked in text output, logs and reports, omitted from saved blocks and redacted before sending to Jev; SECRET inputs are also protected from execution hashing. For screenshot and video eligibility, follow the declared media policy. For log and media handling and the value-free events retained in logs/core-diagnostics.jsonl, see the CLI guide's runtime-input diagnostics.
  • The iOS simulator's own system log may record text the app displays, including a filled field's value, as it does for every XCTest-driven run; erase a dedicated QA simulator (xcrun simctl erase <udid>) when that matters. The runner's XCTest result bundles are removed after every runner session.
  • For masking input echoes on later screens, optional echo declarations and privacy refusals before hashing, follow the CLI guide's runtime-input hash protections.
  • QAREN_INPUT_* variables required by neither the plan nor the compatible configured login block are not used for fills. The whole namespace is removed from every process qaren starts; its presence still withholds subprocess logs.
  • Fill "note" with "input PASSWORD" types the quoted text.

A qaren runtime without input support refuses a plan that names inputs.

Quoted and phrase targets

A quoted target, such as Tap "tab-tasks", resolves locally against the labels and test IDs on screen. Two or more distinct matches refuse with TARGET_AMBIGUOUS; that refusal is never retried or re-walked. For Fill targets addressed by an input's React placeholder, see the targeting rules, including the conditions for naming an otherwise unnamed native input.

A phrase target, such as Tap the tasks tab, has no quotes. Jev, QaReN's screen judge, picks the element, which must carry the phrase's naming words or the step refuses TARGET_NOT_FOUND.

Checks

A check is literal only when its whole payload is one quoted string: ✓ "Welcome". It passes when that text is visible on screen as a substring. Text hidden under the keyboard, React-only text and images do not count.

Any other check is a phrase check, such as ✓ The tasks screen shows the task count. Jev judges it against the captured screen.

A failing check or an unmet quoted wait can enter step recovery before failing the plan. A literal check that still cannot be decided after a fresh capture ends CHECK_UNSURE if recovery cannot resolve it. For quoted-wait deadline limits and fields mounted in React but absent from the native screen, see the CLI reference's wait evidence and recovery contract.

When the plan needs TYPESAFE_API_KEY

See Jev: how QaReN judges phrase steps for which lines need a key, how to write a key-free plan, and the readiness and recovery contracts.

Refusals before the walk

An unparseable line refuses the plan with PLAN_UNPARSEABLE before a device is leased. The refusal lists each line and its reason. A screen target that does not resolve refuses during the walk instead.

Saved plans

After a fully passing qaren check, QaReN saves the plan into the app's .qaren/plans/ as a canonical copy: the # title, ## QA, and each block's ### title and plan lines, with prose and comments dropped. A copy that would hold a private value is not saved. An existing file is never overwritten: a file that matches apart from its qaren:broken marker lines is left alone, and a different plan with the same name is saved as <name>-2.md, <name>-3.md and so on. qaren pr does not save plans.

qaren plans (or qaren plans --json), run from the app directory, lists the saved plans with their titles, blocks and the platforms they are flagged broken on. Pass a listed file to --plan-file to run it again.

When a saved plan runs on a clean checkout whose HEAD is the default-branch commit (origin/HEAD), a step failure inserts <!-- qaren:broken ios --> (or android) as a new first line, and the next such PASS removes it. Only a definite failure flags the plan: a check judged to fail, or an action that demonstrably did not take effect. An uncertain check, an unverified action, a system dialog in front of the app, an outage of the judgment service, a device or runner error, or a refusal leaves it unflagged. On any other commit or with local changes the file is left alone. The receipt's recipe field and the plan: line on stderr report what happened. Set savePlans: false in configuration to turn saving and flagging off. The CLI guide's saved plans section lists every outcome.

Platform notes

  • On iOS, an accept or dismiss step against a system alert taps one button by exact English label, so Allow never matches Allow Once. A localized alert fails DIALOG_BUTTON_NOT_FOUND. Assert the permission's effect with the next step.
  • For Android dialog and modal limits, snapshot visibility and covered-target recovery, see the CLI guide's walk contract and occluded taps and focus.
  • For how fills affect video eligibility under the selected policy, see Test a pull request.

Edit this page on GitHub