TypeScript API

Embed Browser Harness tools in an application that runs its own agent loop. Everything is exported from @blopai/browser-harness. Requires Node.js 22 or newer.

npm install @blopai/browser-harness playwright

createBrowserTools

createBrowserTools(options) returns the tool set for one Playwright page.

import { chromium } from "playwright";
import { createBrowserTools, type HarnessAction } from "@blopai/browser-harness";

const browser = await chromium.launch({ headless: true });
const page = await browser.newPage();
const actions: HarnessAction[] = [];

const tools = await createBrowserTools({
  page,
  testId: "checkout",
  screenshotDir: ".harness-screenshots",
  actions,
  screenshots: [],
  finishState: { status: null, reason: null },
});

const goto = tools.find((tool) => tool.name === "browser_goto")!;
await goto.execute({ url: "https://example.com" });
await browser.close();

Option	Type	Purpose
page	Playwright Page	Page the tools drive
testId	string	Identifier for this run’s evidence
screenshotDir	string	Where screenshots are written
actions	HarnessAction[]	Receives every successful and failed action
screenshots	string[]	Receives screenshot paths
finishState	{ status, reason }	Receives the finish_test outcome
safety	BrowserSessionPolicy	Optional. { mode: "read-only" } denies pointer, keyboard, form, upload and page-closing tools; denied calls throw BrowserSafetyError and are still recorded.

NativeToolBridge

Each returned tool is a NativeToolBridge:

Field	Purpose
name	Tool name, for example browser_click
description	Text for the model
parameters	JSON Schema for the input
promptSnippet	One-line usage hint for a system prompt
execute(input)	Run the tool and record the action

Targets

Tools that act on an element take a target. In order of preference:

Target	Example
Reference from browser_snapshot	{ ref: "e13" }
Role and accessible name	{ role: "button", name: "Save" }
Label, placeholder or text	{ label: "Email", exact: true }
Test ID	{ testId: "submit" }
CSS selector	{ selector: "input[name='email']" }

An ambiguous target fails unless you set first: true. A stale, detached, hidden, inert or occluded reference fails rather than silently retargeting another element.

Tool bounds

Tool	Bound
browser_snapshot	At most 60 controls (default 60)
browser_extract	30 records by default, at most 100
browser_run_steps	At most 20 steps; cannot include finish_test
browser_screenshot	Largest dimension 2,000 pixels by default

The tool set also covers navigation, mouse, keyboard, forms, tabs, assertions, diagnostics and lifecycle. The browser tools reference lists the tools browser checks use.

startScreencast

startScreencast(options) streams JPEG frames from Chromium over CDP. It returns null when CDP screencasting is unavailable, including Firefox and WebKit.

import { startScreencast } from "@blopai/browser-harness";

const screencast = await startScreencast({
  page,
  quality: 50,
  maxWidth: 1280,
  maxHeight: 800,
  everyNthFrame: 1,
  onFrame(frame) {
    console.log(frame.seq, frame.timestamp);
  },
});

const latestFrame = screencast?.latest();
await screencast?.stop();

Member	Purpose
quality	JPEG quality, 0 to 100
maxWidth, maxHeight	Frame size limit
everyNthFrame	Keep every Nth repaint; 1 keeps all
onFrame(frame)	Called with { data, timestamp, seq } for each frame
latest()	Most recent frame, or null before the first
frameCount()	Frames received so far
stop()	End the screencast

For browsers in Docker, see Container sessions.