TypeScript API

The package exports controlled browser tools and session helpers from one public entry point. Use this API when your application owns the agent loop and needs explicit tool contracts for a Playwright page.

Install the package

Install the harness alongside Playwright in a Node.js 22 or newer project:

npm install @blopai/browser-harness playwright

Create browser tools

Pass a Playwright page and mutable evidence collections to createBrowserTools:

import { chromium } from "playwright";
import {
  createBrowserTools,
  type HarnessAction,
} from "@blopai/browser-harness";

const browser = await chromium.launch({ headless: true });
const page = await browser.newPage();
const actions: HarnessAction[] = [];

const tools = await createBrowserTools({
  page,
  testId: "checkout",
  screenshotDir: ".harness-screenshots",
  actions,
  screenshots: [],
  finishState: { status: null, reason: null },
});

const goto = tools.find((tool) => tool.name === "browser_goto")!;
await goto.execute({ url: "https://example.com" });

await browser.close();

Each NativeToolBridge contains a name, description, JSON parameter schema, prompt snippet, and execute function. The harness records successful and failed actions in the collection supplied by the host.

Target elements

Prefer an opaque reference returned by browser_snapshot, then semantic locators, labels, test IDs, and CSS selectors in that order:

await click.execute({
  target: { ref: "e1" },
});

await type.execute({
  target: { label: "Email", exact: true },
  text: "ada@example.com",
});

await expectValue.execute({
  target: { selector: "input[name='email']" },
  value: "ada@example.com",
});

Structured targets preserve Playwright strictness. Ambiguous targets fail unless you explicitly set first: true. Stale, hidden, detached, or occluded references also fail instead of silently retargeting another element.

Use bounded observation tools

The tool catalog includes navigation, mouse, keyboard, forms, tabs, assertions, extraction, diagnostics, screenshots, batching, and lifecycle records. Observation results are bounded to keep model context predictable:

• browser_snapshot returns at most 60 semantic controls by default.
• browser_extract returns 30 records by default and at most 100.
• browser_run_steps executes at most 20 controlled steps.
• Screenshots default to a maximum dimension of 2,000 pixels.

For the tools used by Blop tests, see the browser tools reference.

Stream screencast frames

startScreencast streams bounded JPEG frames from Chromium through CDP:

import { startScreencast } from "@blopai/browser-harness";

const screencast = await startScreencast({
  page,
  quality: 50,
  maxWidth: 1280,
  maxHeight: 800,
  everyNthFrame: 1,
  onFrame(frame) {
    console.log(frame.seq, frame.timestamp);
  },
});

const latestFrame = screencast?.latest();
await screencast?.stop();

The function returns null when CDP screencasting isn’t available, including Firefox and WebKit sessions.

Next steps

Use container sessions when the browser must run outside your application process. Review the security model before exposing tools to an agent.