Troubleshooting

If a Blop run goes sideways, this page covers the failure modes you’ll hit most often. The structured output in .blop/ is designed to be diagnostic — start there before anything else.

“No Blop spec files found”

Error: No Blop spec files found for: **/*.blop.ts, **/*.blop.tsx

Blop only picks up files matching *.blop.ts(x) or *.agent.ts(x). Either:

• Rename your spec file to match (tests/homepage.blop.ts).
• Pass an explicit pattern: blop test tests/my-spec.ts.
• Adjust include in blop.config.ts.

Agent hits --max-steps

ERROR  homepage > loads - exhausted max steps (100)

The agent ran out of tool calls before calling finish_test. Two common causes:

1. Goal is underspecified. Rewrite the goal with concrete verifications (“verify the heading ‘Welcome’ appears and the ‘Sign up’ button is visible in the header”). See Writing tests for goal-writing tips.
2. The flow is genuinely long. Raise the budget: --max-steps 150 or maxSteps: 150 in blop.config.ts.

Provider auth: “Missing API key”

Error: Missing API key for provider "openai".

Set one of (in order of precedence):

1. CLI flag: --api-key sk-...
2. BLOP_AGENT_API_KEY env var
3. Provider-native env var: OPENAI_API_KEY, ANTHROPIC_API_KEY, etc.
4. apiKey in blop.config.ts

Per-provider details live in Agent Providers.

Browser launch failures

If you see Playwright errors like browserType.launch: Executable doesn't exist at ..., install the browsers:

bunx playwright install chromium
# or for all engines
bunx playwright install

In CI, add this step before blop test. See CI integration.

The agent finished as failed — why?

A failed status means the agent called finish_test with status: "failed". That is a product-level failure, not a Blop bug. To debug:

1. Read reason in .blop/results.json — the agent explains why it failed.
2. Open .blop/events.jsonl and scan the tool_result events near the end.
3. Look at .blop/screenshots/ — most agent runs capture state at decision points.

Compare with error, which means a runtime failure (timeout, crash, no agent output). Use --verbose to stream events to stderr while you reproduce.

Reading events.jsonl for a stuck run

events.jsonl is one event per line. The most useful types:

event_type	What it tells you
step_start / step_finish	Reasoning iterations
tool_result	What a browser tool returned
text_delta / text	The agent’s running narration

Tail it live during a run:

bunx blop test --reporter all &
tail -f .blop/events.jsonl

Goal-writing antipatterns

Antipattern	Why it fails	Fix
“Test the homepage.”	Too vague — agent will pass on anything.	“Verify the heading Welcome appears and the Sign up button is in the header.”
“Click the button and check it works.”	“Works” is undefined.	“Click Submit and verify the URL changes to /thanks and the heading reads Order confirmed.”
Multi-flow goals in one test	Hard to debug; agent may bail early.	Split into one test per user-visible outcome.
“Don’t fail.”	Agents tend to optimistically pass.	Always add explicit verifications and “finish as passed only if the page is usable.”

Stale screenshots in .blop/screenshots/

Blop never deletes the report directory between runs — old screenshots hang around. Clean it manually or in CI:

rm -rf .blop && bunx blop test

Or set a unique --report-dir per run.

JUnit XML quirks for CI consumers

.blop/report.xml is a flat <testsuite> of every test in the run.

• A Blop error status maps to JUnit <error>; failed maps to <failure>.
• The time attribute is in seconds (Blop tracks ms internally).
• system-out includes the agent’s reason string.

If your CI ingestor refuses the file, validate against the JUnit XSD — truncated events.jsonl does not invalidate the XML.

Still stuck?

• Re-run with --verbose to stream every event to stderr.
• Read packages/blop/AGENTS.md for design constraints.
• File an issue on the GitHub repo.