Agent providers

Blop’s native agent loop supports the providers listed below through their OpenAI-compatible or translated tool-call APIs.

Setting a provider

Sources, from highest to lowest precedence:

1. CLI flags: --provider, --model, --api-key
2. blop.config.ts: provider, model, apiKey
3. Environment variables: BLOP_AGENT_*
4. Provider-native variables such as OPENAI_API_KEY

# CLI
blop test --provider anthropic --model claude-sonnet-4-20250514

# Env
export BLOP_AGENT_PROVIDER=anthropic
export BLOP_AGENT_MODEL=claude-sonnet-4-20250514
export BLOP_AGENT_API_KEY=sk-ant-...

# Config
# provider: 'anthropic', model: 'claude-sonnet-4-20250514'

Supported providers

Provider	--provider value	Native env var
OpenAI	openai	OPENAI_API_KEY
Anthropic	anthropic	ANTHROPIC_API_KEY
Google	google	GEMINI_API_KEY
Groq	groq	GROQ_API_KEY
xAI	xai	XAI_API_KEY
OpenRouter	openrouter	OPENROUTER_API_KEY
Mistral	mistral	MISTRAL_API_KEY
Cerebras	cerebras	CEREBRAS_API_KEY
NVIDIA	nvidia	NVIDIA_API_KEY

When BLOP_AGENT_API_KEY is unset, Blop falls back to the provider-native env var.

Choosing a model

The agent must be capable enough to read the page, decide on actions, and produce a structured finish_test call. Practical recommendations:

• OpenAI: gpt-5 for production runs; cheaper models work for smoke tests but can plateau on multi-step flows.
• Anthropic: claude-sonnet-4 and newer Claude 4.x family handle Blop’s tool-call patterns well.
• Google: latest Gemini 2.5+ models.
• Groq / Cerebras: very fast, good for tight CI feedback loops on simple flows; may struggle with long goals.

If you don’t know what to pick, start with openai + gpt-5 and switch based on cost and latency.

Running without keys

For unit testing the runner or CI smoke checks of the framework itself, pass a mock agentStream to runBlopTests — see Writing tests → Testing your specs without a real model.

Cost and latency

Every step counts as one round-trip to the provider. With --max-steps, budget for up to the configured cap per check. Two ways to keep cost predictable:

• Tighten goals. A precise goal often resolves in 6–10 steps.
• Lower maxSteps. Force tests to fail loudly instead of grinding.

For long flows, raising --max-steps 150 is fine. The budget exists to catch runaway agents, not to optimize cost.

Troubleshooting auth failures

See Troubleshooting → Provider auth.