# Runner, reporters and types

> Look up the browser check runner API, its options, result fields, report files and exported TypeScript types.

The programmatic runner behind `blop test run`, the files it writes, and the types `@blopai/test` exports. Import from `@blopai/test`, never `@blopai/cli` or files under `dist/`; coverage helpers come from `@blopai/test/coverage`.

## Runner functions

| Function | Use |
|---|---|
| `runBlopTest(options)` | Run one spec file (`options.specFile`) |
| `runBlopTests(options)` | Run several spec files (`options.specFiles`) in a browser, write reports and optionally upload |

Both return `Promise<BlopRunResult>`.

```ts
import { runBlopTests } from "@blopai/test";

const result = await runBlopTests({
  specFiles: ["tests/homepage.blop.ts"],
  baseUrl: "http://localhost:3000",
  reporter: "all",
});
if (result.status !== "passed") process.exit(1);
```

## Runner options

`BlopRunOptions` fields. Every field is optional; the `blop.config.ts` fields in [Configuration](/reference/configuration/) use the same names and defaults.

| Field | Type | Purpose |
|---|---|---|
| `specFile`, `specFiles` | `string`, `string[]` | What to run |
| `baseUrl` | `string` | App under test; relative `agent.goto()` paths resolve against it |
| `browserContext` | Playwright `BrowserContextOptions` | Extra context options |
| `browser`, `viewport`, `headed`, `containerized` | See [Configuration](/reference/configuration/#browser-options) | Browser choice and display |
| `provider`, `model`, `apiKey`, `reasoningEffort` | `string`; `"none" \| "low" \| "medium" \| "high" \| "max"` | [Provider](/reference/glossary/#provider) settings; the effort applies when the API supports it |
| `cwd` | `string` | Working directory for the agent |
| `maxSteps`, `timeoutMs`, `retries`, `workers` | `number` | Step cap, per-check time cap, retries, concurrency |
| `reportDir`, `reporter`, `verbose` | `string`, `BlopReporter`, `boolean` | Reports |
| `progressFile` | `string` | Append live NDJSON progress (`test_start`, `action`, `frame`, `test_finish`) |
| `captureStepScreenshots`, `streamFrames`, `frameIntervalMs` | `boolean`, `boolean`, `number` | Per-action screenshots and live Chromium frames |
| `coverage`, `coverageEndpoint` | `boolean`, `string` | Chromium JavaScript coverage and the collector that receives it |
| `platformUrl`, `platformApiKey` | `string` | Blop Cloud upload; also needs `BLOP_PROJECT_ID` |
| `otelEndpoint`, `otelHeaders`, `otelServiceName`, `otelEnvironment`, `otelPropagateToApp`, `otelPropagateAllowlist` | See [Configuration](/reference/configuration/#opentelemetry-options) | OpenTelemetry export |
| `evidence` | `BlopEvidence` | Launch provenance from a trace or finding; never proof of a fix |
| `agentStream` | `BlopAgentStreamRunner` | Replace the model to test the runner; see [Run browser checks without a provider](/checks/providers/#run-browser-checks-without-a-provider) |

## Result fields

`BlopRunResult` has `runId`, `status`, `startedAt`, `finishedAt` (ISO 8601), `durationMs` and `results: BlopTestResult[]`. Its `status` is `error` if any check errored, else `failed` if any failed, else `passed`.

| `BlopTestResult` field | Meaning |
|---|---|
| `id`, `name`, `specFile` | Check identity; `specFile` is repo-relative or `null` |
| `status`, `reason` | Outcome and the agent's explanation |
| `startedAt`, `finishedAt`, `durationMs` | Timing |
| `attempts`, `firstAttemptStatus` | `1` plus retries used; the first attempt's status when retried, else `null` |
| `resumes` | Agent recoveries in the same live browser; not retries |
| `synthetic` | `true` for a record the runner made for a spec load error or aborted run |
| `baseUrl`, `provider`, `model` | What ran, or `null` |
| `ci` | `BlopCiMetadata` |
| `screenshots`, `screenshotArtifacts` | Screenshot paths and their details |
| `criticalPoints` | Requirements the agent recorded with `record_critical_point` |
| `browserLogs` | Console messages, page errors and failed requests |
| `actions` | `BlopAction`: `name`, `input`, `output`, `metadata`, `timestamp`, `durationMs` for each tool call |
| `events` | `BlopAgentEvent`: `event_type`, `content`, `metadata`, `timestamp` |

| Status | Meaning |
|---|---|
| `passed` | The agent called `finish_test` with `passed` |
| `failed` | The agent finished with an unmet goal; check the assertions and evidence before blaming the app |
| `error` | The run did not finish normally: timeout, browser crash, provider failure or invalid spec |

A final `passed` with a non-passing `firstAttemptStatus` is flaky.

`BlopCiMetadata` has `provider`, `runId`, `jobId`, `branch`, `commitSha`, `pullRequest` (raw ref such as `refs/pull/42/merge`), `pullRequestNumber`, `repositoryUrl`, `workflowName`, `runAttempt`, `runUrl` and `refType` (`branch` or `tag`). It is filled on GitHub Actions (`provider: "github-actions"`) and `null` elsewhere.

## Report files

Every run writes to `reportDir` (default `.blop/`), whether started from the CLI, watch mode or `runBlopTests`.

| File | Written |
|---|---|
| `results.json` | Always; the `BlopRunResult` |
| `events.jsonl` | Always; one `BlopAgentEvent` per line |
| `report.xml` | With reporter `junit` or `all` |
| `screenshots/` | When the check takes screenshots or per-action capture is on |
| `coverage.json` | With `coverage`; the Inspector shows the latest report at `/inspector?view=coverage` |

| Reporter | Files |
|---|---|
| `all` (default) | `results.json`, `events.jsonl`, `report.xml` |
| `junit` | Same as `all` |
| `json`, `basic` | `results.json`, `events.jsonl` |

| `event_type` | Tells you |
|---|---|
| `step_start`, `step_finish` | One reasoning step |
| `tool_result` | What a browser tool returned |
| `text_delta`, `text` | The agent's narration |

`report.xml` is one flat `<testsuite>` with a `<testcase>` per check. Both `failed` and `error` become `<failure>` with the reason as its body, and `time` is in seconds.

Use a fresh `--report-dir` per run, such as `.blop/reports/$GITHUB_RUN_ID`, because old screenshots stay. Never delete all of `.blop/`: it can also hold the project's collector store and configuration. Add `.blop/` to `.gitignore`.

## Exported types

| Types | Reference |
|---|---|
| `BlopAgent`, `BlopAgentTest`, `BlopAgentTestContext`, `BlopAgentTestHandler`, `BlopAgentStep` | [Test DSL](/reference/test-dsl/) |
| `BlopRunOptions`, `BlopRunResult`, `BlopTestResult`, `BlopTestStatus`, `BlopAction`, `BlopAgentEvent`, `BlopCiMetadata`, `BlopAgentStreamRunner`, `BlopAgentStreamEvent` | This page |
| `BlopConfig`, `BlopBrowserName`, `BlopReporter` | [Configuration](/reference/configuration/) |

Exported functions include `agentTest`, `describe`, `defineAgentTest`, `loadAgentTests`, `runBlopTest`, `runBlopTests`, `runBrowserAgentStream`, `runNativeAgentStream` and `loadBlopConfig`; `BLOP_BROWSER_NAMES` lists the browser names.