Skip to content
Blopblop docs

Writing tests

A Blop test is a TypeScript file that ends in .blop.ts (or .blop.tsx) and exports one or more agent tests. The runtime is intentionally small — three exports cover almost every case.

The two shapes

Section titled “The two shapes”

Function form: describe + agentTest

Section titled “Function form: describe + agentTest”

Group related tests, write goals inline:

import { agentTest, describe } from "blop";


describe("onboarding", () => {
  agentTest("user can create a project", async ({ agent }) => {
    await agent.goto("/");
    await agent.goal(`
      Sign in as the seeded user.
      Create a project called "Checkout QA".
      Verify the project appears in the dashboard.
    `);
  });
});

Each describe adds to the test name with > — nest as deeply as you like.

Object form: defineAgentTest

Section titled “Object form: defineAgentTest”

For programmatically generated specs or array exports:

import { defineAgentTest } from "blop";


export default defineAgentTest({
  name: "smoke",
  goal: "Open the homepage and verify the primary CTA renders.",
});


export const tests = [
  defineAgentTest({
    name: "checks page title",
    goal: "Go to /, get the page title, and verify it contains 'Blop'.",
  }),
];

Each spec file may export default (single test) and/or tests (array).

What agent actually does

Section titled “What agent actually does”

agent.goto(url) and agent.goal(text) don’t run anything immediately. They build up a numbered goal list. When the runner reaches the test, the agent receives that list along with browser tools and tries to fulfill it.

agentTest("checkout flow", async ({ agent }) => {
  await agent.goto("/cart");                // step 1
  await agent.goal("Add the first item.");  // step 2
  await agent.goto("/checkout");            // step 3
  await agent.goal("Pay with the test card and verify success.");
});

That ordering matters: the agent treats the list as a sequence.

Writing a goal the agent can act on

Section titled “Writing a goal the agent can act on”

The single biggest factor in test reliability is the wording of goal. Treat it like a clear ticket for a careful junior engineer.

Good goals

Section titled “Good goals”
  • State outcomes, not selectors.

    “Verify the dashboard heading reads My projects and the New project button is visible in the top-right.”

  • Demand explicit verification.

    “Submit the form. Verify the URL becomes /thanks and the heading Order confirmed is on the page.”

  • Force a strict pass condition.

    “Finish the test as passed only if all of the above checks succeed. Otherwise finish as failed and explain why.”

Antipatterns

Section titled “Antipatterns”
  • “Test the homepage.” → too vague; the agent will optimistically pass.
  • “Make sure the button works.” → “works” is undefined.
  • Multi-flow goals like “Sign up, then post a comment, then delete the account.” → split into separate tests.

See Troubleshooting for more.

Naming and organization

Section titled “Naming and organization”
  • One describe per user-visible feature (onboarding, billing, auth).
  • Test names finish the sentence: agentTest("can create a project", ...).
  • Co-locate specs with the feature when possible: apps/web/app/(dashboard)/projects/__tests__/projects.blop.ts.
  • Or keep them in a top-level tests/ directory — Blop discovers either.

Per-test overrides

Section titled “Per-test overrides”

defineAgentTest accepts baseUrl and timeoutMs for a single test:

defineAgentTest({
  name: "external integration smoke",
  goal: "Hit the staging API health endpoint and verify it returns 200.",
  baseUrl: "https://staging.example.com",
  timeoutMs: 60_000,
});

For function-form tests, use blop.config.ts or the CLI flags.

Testing your specs without a real model

Section titled “Testing your specs without a real model”

When you need to unit-test the runner or your goal templates, pass a mock agentStream to runBlopTests:

import { runBlopTests, type BlopAgentStreamRunner } from "blop";


const mockStream: BlopAgentStreamRunner = async function* ({ nativeTools }) {
  const goto = nativeTools.find((t: any) => t.name === "browser_goto");
  const finish = nativeTools.find((t: any) => t.name === "finish_test");


  yield { event_type: "step_start", metadata: {} };
  await goto!.execute({ url: "https://example.com" });
  yield { event_type: "step_finish", metadata: {} };


  await finish!.execute({ status: "passed", reason: "page loaded" });
};


const result = await runBlopTests({
  specFiles: ["tests/example.blop.ts"],
  agentStream: mockStream,
});

No API keys required, no network calls. Useful for CI smoke checks of the framework itself.

Where to go next

Section titled “Where to go next”