CI integration

Blop is designed CI-first. The default reporter writes JUnit XML and a JSON result that any pipeline can pick up.

GitHub Actions (recommended)

.github/workflows/blop.yml

name: Blop E2E
on:
  pull_request:
  push:
    branches: [main]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: oven-sh/setup-bun@v1

      - run: bun install

      - run: bunx playwright install --with-deps chromium

      - name: Run Blop
        run: bunx blop test --base-url ${{ vars.APP_URL }} --report-dir .blop --reporter all
        env:
          BLOP_AGENT_PROVIDER: openai
          BLOP_AGENT_MODEL: gpt-5
          BLOP_AGENT_API_KEY: ${{ secrets.BLOP_AGENT_API_KEY }}

      - name: Upload artifacts
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: blop-results
          path: .blop/

      - name: Publish JUnit report
        if: always()
        uses: dorny/test-reporter@v1
        with:
          name: Blop tests
          path: .blop/report.xml
          reporter: java-junit

Key points:

• Install browsers explicitly. bunx playwright install is required on fresh runners.
• Use repository secrets for API keys. Never inline them.
• if: always() on artifact upload. You want the bundle even when tests fail; that’s when it matters most.
• Pin the model. Floating model versions can change agent behavior between runs.

CI metadata captured automatically

When running on GitHub Actions, Blop populates BlopCiMetadata on every test result:

{
  "provider": "github-actions",
  "runId": "1234567890",
  "jobId": "test",
  "branch": "feat/checkout-flow",
  "commitSha": "deadbeef...",
  "pullRequest": "42"
}

This is included in results.json and uploaded to Blop Platform if configured.

Other CI providers

The same shape works on any provider that supports JUnit ingestion. Fill in the equivalent steps:

• GitLab CI → artifacts.reports.junit: .blop/report.xml
• CircleCI → store_test_results: path: .blop
• Buildkite → buildkite-agent artifact upload .blop/**/*

CI metadata auto-detection currently covers GitHub Actions only; on other providers BlopCiMetadata.provider will be null.

Parallelization

Blop runs tests serially within a single process so the agent has predictable control of the browser. To parallelize, shard at the workflow level:

strategy:
  matrix:
    shard: [1, 2, 3, 4]
steps:
  - run: bunx blop test "tests/shard-${{ matrix.shard }}/**/*.blop.ts" --report-dir .blop/${{ matrix.shard }}

Pair with actions/upload-artifact and merge JUnit reports downstream.

Cost control

Each test costs up to maxSteps model calls. To keep CI bills sane:

• Pin a --max-steps budget that’s tight but realistic.
• Run the full suite on main only; on PRs, run a smoke subset.
• Cache ~/.cache/ms-playwright to skip re-downloading browsers.

Uploading runs to Blop Platform

Blop uploads results to the platform using CloudEvents (qa.run.started.v1 + qa.run.finished.v1) via the @blopai/ingest client. Set:

env:
  BLOP_INGEST_URL: ${{ vars.BLOP_INGEST_URL }}
  BLOP_INGEST_SECRET: ${{ secrets.BLOP_INGEST_SECRET }}
  BLOP_PROJECT_ID: ${{ vars.BLOP_PROJECT_ID }}

The runner emits started + finished events automatically when BLOP_INGEST_URL and BLOP_INGEST_SECRET are present, and uploads a zipped report bundle (results.json, events.jsonl, screenshots) as an artifact. CI metadata (branch, commit, run URL) is attached automatically on GitHub Actions.

This is the same wire protocol used by every other test runner adapter — see Vitest integration and any framework via JUnit XML.

Failing the build

blop test exits non-zero if any test failed or errored. No extra wiring needed — let the step fail and your branch protection / required checks kick in.