# Browser checks in CI

> Run browser checks in GitHub Actions, export their telemetry over OTLP, and upload Vitest or JUnit results to Blop Cloud.

Run browser checks on every pull request, keep their evidence, and send test results to Blop Cloud.

## Prerequisites

- `@blopai/cli` and `@blopai/test` committed as dev dependencies with their lockfile ([Browser checks](/checks/)).
- A preview or staging URL for the app, stored as the repository variable `APP_URL`.
- The provider key stored as the repository secret `BLOP_AGENT_API_KEY`.
- Node.js 22.18 or newer; the workflow below uses the latest Node.js 22.
- To upload results to Blop Cloud: the ingest secret from your project's **Settings → Hosted evidence ingest**, the project ID, and your Blop Cloud address followed by `/api/ingest`. Store them as `BLOP_INGEST_SECRET` (secret), `BLOP_PROJECT_ID` and `BLOP_INGEST_URL` (variables).

## Run browser checks in GitHub Actions

1. Add this workflow:

   ```yaml
   # .github/workflows/blop.yml
   name: Browser checks
   on:
     pull_request:
     push:
       branches: [main]

   permissions:
     contents: read
     checks: write

   jobs:
     checks:
       runs-on: ubuntu-latest
       steps:
         - uses: actions/checkout@v4
         - uses: pnpm/action-setup@v4
           with:
             version: 9
         - uses: actions/setup-node@v4
           with:
             node-version: 22
         - run: pnpm install --frozen-lockfile
         - run: pnpm dlx playwright@1.61.1 install --with-deps chromium
         - name: Run browser checks
           run: pnpm exec blop test run --base-url ${{ vars.APP_URL }} --max-steps 75
           env:
             BLOP_AGENT_PROVIDER: openai
             BLOP_AGENT_MODEL: gpt-5
             BLOP_AGENT_API_KEY: ${{ secrets.BLOP_AGENT_API_KEY }}
         - name: Upload reports
           if: always()
           uses: actions/upload-artifact@v4
           with:
             name: blop-results
             path: .blop/
         - name: Publish JUnit report
           if: always()
           uses: dorny/test-reporter@v1
           with:
             name: Browser checks
             path: .blop/report.xml
             reporter: java-junit
   ```

2. Push a branch and open a pull request.

`dorny/test-reporter` needs the `checks: write` permission. Pull requests from forks get no secrets or variables, so checks there cannot reach the provider.

The step fails when any check fails or errors, so branch protection needs no extra wiring. `if: always()` keeps the reports when checks fail, which is when you need them. Install Chromium on every fresh runner, keep keys in secrets, and pin the model: a new model version can change what the agent does. Seed isolated test data before the checks start.

On GitHub Actions, each result in `results.json` carries CI metadata: branch, commit, pull request, workflow and run URL ([`BlopCiMetadata`](/reference/runner/)). Other CI systems get `null` for these fields.

Other CI systems read the same JUnit file: GitLab with `artifacts.reports.junit: .blop/report.xml`, CircleCI with `store_test_results` on `.blop`, Buildkite with `buildkite-agent artifact upload .blop/**/*`.

## Shard browser checks and control cost

```yaml
strategy:
  matrix:
    shard: [1, 2, 3, 4]
steps:
  - run: pnpm exec blop test run "tests/shard-${{ matrix.shard }}/**/*.blop.ts" --report-dir .blop/${{ matrix.shard }}
```

`--workers <n>` runs up to n checks at once in one job; shards give separate failures and artifacts. Give each shard its own report directory and merge the JUnit files afterwards. A pass on retry is flaky, not a pass; see [retries](/checks/agent-authored/#retries-and-flaky-browser-checks). `--coverage` adds `coverage.json` to the reports but is not a CI gate.

`--max-steps` caps agent tool steps, not money, and does not count retries or resumes. There is no default cap. Set one that is tight but realistic, run the full suite on `main` and a smoke subset on pull requests, and cache `~/.cache/ms-playwright`.

## Export browser check telemetry with OpenTelemetry

```yaml
      - run: pnpm exec blop test run --base-url ${{ vars.APP_URL }}
        env:
          OTEL_EXPORTER_OTLP_ENDPOINT: http://collector.internal:4318
          OTEL_SERVICE_NAME: blop-runner
```

The CLI exports straight from the job to your OTLP/HTTP collector, and nothing goes to Blop Cloud. It supports `http/protobuf` (the default) and `http/json`, not gRPC. Without an endpoint nothing is exported. Export failures are logged and never fail the check, and shutdown waits at most five seconds for an unreachable collector. All variables are in [Environment](/reference/environment/#browser-check-opentelemetry-export).

Each check execution is one trace: a `run` span, a `scenario` span per check, and a `step` span per tool call. Spans carry the OpenTelemetry test, CI/CD and VCS attributes, plus `blop.journey.id` (the root `describe`), `blop.scenario.path`, `blop.failure.category` and `blop.step.tool`. They never carry screenshots, DOM snapshots, typed text or tool output, and URL credentials and token query parameters are redacted.

Three metrics, without execution or scenario IDs to keep cardinality low: `blop.scenario.duration` (histogram, seconds, by journey, status and failure category), `blop.agent.recoveries` (counter, by journey and `resume` or `retry`) and `blop.agent.tokens` (counter, by journey, kind and team). Tokens are not a bill. The events `qa.run.started.v1`, `qa.run.step.finished.v1` and `qa.run.finished.v1` are also sent as log records in the trace.

To link a failed step to the backend span behind it, send W3C `traceparent` to the app:

```yaml
        env:
          OTEL_EXPORTER_OTLP_ENDPOINT: http://collector.internal:4318
          BLOP_OTEL_PROPAGATE_TO_APP: "true"
          BLOP_OTEL_PROPAGATE_ALLOWLIST: staging.example.com
```

Propagation is off by default and has no wildcard: only listed hosts and their subdomains get the header. Use non-production targets; a production target needs the [production safeguards](/checks/production/). Service worker requests carry no header, and with Camoufox the header is a fingerprinting signal, which Blop warns about.

## Upload browser check results to Blop Cloud

```yaml
        env:
          BLOP_INGEST_URL: ${{ vars.BLOP_INGEST_URL }}
          BLOP_INGEST_SECRET: ${{ secrets.BLOP_INGEST_SECRET }}
          BLOP_PROJECT_ID: ${{ vars.BLOP_PROJECT_ID }}
```

With all three set on the `blop test` step, each execution becomes a [check run](/reference/glossary/#check-run) in your [Blop Cloud](/cloud/) project, with a zipped bundle of `results.json`, `events.jsonl` and screenshots. Upload failures do not fail the step.

## Upload Vitest results to Blop Cloud

1. Install the reporter:

   ```bash
   pnpm add -D @blopai/ingest
   ```

2. Add it to `vitest.config.ts`:

   ```ts
   import { defineConfig } from "vitest/config";

   export default defineConfig({
     test: {
       reporters: ["default", "@blopai/ingest/vitest"],
     },
   });
   ```

3. Set the three ingest variables on the CI step that runs Vitest:

   ```yaml
         - run: pnpm test
           env:
             BLOP_INGEST_URL: ${{ vars.BLOP_INGEST_URL }}
             BLOP_INGEST_SECRET: ${{ secrets.BLOP_INGEST_SECRET }}
             BLOP_PROJECT_ID: ${{ vars.BLOP_PROJECT_ID }}
   ```

The reporter sends counts, up to 25 failures (2,000 characters each), CI metadata and, when artifact storage is configured, `vitest-report.json`. If any of the three variables is unset, it does nothing and never fails Vitest. In watch mode, each rerun is uploaded as a separate check run in Blop Cloud unless `BLOP_RUN_ID` is set. Optional: `BLOP_RUN_ID` (generated if unset) and `BLOP_TRIGGER` (`ci`, `manual`, `schedule` or `healer_verify`).

## Upload JUnit XML from any framework to Blop Cloud

1. Make your framework write JUnit XML, for example `pytest --junitxml=report.xml`, `gotestsum --junitfile report.xml -- ./...` or `phpunit --log-junit report.xml`.
2. Add an upload step after the tests:

   ```yaml
         - name: Upload to Blop
           if: always()
           run: npx @blopai/ingest upload --junit report.xml
           env:
             BLOP_INGEST_URL: ${{ vars.BLOP_INGEST_URL }}
             BLOP_INGEST_SECRET: ${{ secrets.BLOP_INGEST_SECRET }}
             BLOP_PROJECT_ID: ${{ vars.BLOP_PROJECT_ID }}
   ```

Without the three variables, or `--ingest-url`, `--secret` and `--project-id`, the upload exits 1. `--junit` takes a glob and repeats; `--artifact <file>` uploads extra files, not directories. `BLOP_TRIGGER` defaults to `ci` in CI and `manual` elsewhere. Run `npx @blopai/ingest upload --help` for every flag.

Each test gets a stable identity from suite, class and name; a test that failed then passed on rerun is recorded as flaky; failures with the same signature are grouped. Results are filed under their branch, so a feature branch never affects the default branch's pass rate.

## Check it worked

The pull request shows a **Browser checks** status and a JUnit summary, and the `blop-results` artifact holds `results.json`, `events.jsonl`, `report.xml` and screenshots.

## Next

- [Reverify a fix with a browser check](/checks/agent-authored/)
- [Blop Cloud](/cloud/)