Verifies that a calendar event was created with correct title, date, and participants via HTTP API query.

Opens IMAP connection to Sent folder and searches for a matching message by recipient and subject fragment. Proves the 'latent state' insight that UI confirmation does not equal actual delivery.

Executes a sandboxed read-only shell command and compares stdout to an expected value, probing whether an agent action actually changed system state.

Verifies that an HTTP response contains expected headers.

Verifies that an HTTP response body contains expected substrings.

Verifies that an HTTP endpoint returns the expected status code.

Verifies that agent-generated Python code passes ruff lint checks with configurable violation thresholds.

Writes agent-generated Python code to a temp directory and runs pytest via the sandbox runner. Score is based on the fraction of tests that pass.

Verifies that a row matching given criteria exists in a database table.

Verifies that a database row was updated to contain expected values.

Verifies that a database table has the expected number of rows.

Verifies that a CSV file has the expected number of data rows.

Verifies that a file contains valid JSON with optional type and key checks.

Verifies that a PDF has the expected number of pages.

Verifies that a text file contains all expected substrings.

Verifies that a file contains valid YAML with optional key checks.

Verifies that a file was created at the expected path with optional size and content hash checks.

Verifies that a specific git commit exists in a repository by SHA prefix or message substring.

LLM-judge rubric verifier scoring code logic correctness on 4 criteria: algorithm, edge cases, logic errors, requirements.

4-component rubric scored by LLM judge: greeting, formality, key info, no inappropriate content. Score = sum/4. MUST be composed with vr/aiv.email.sent_folder_confirmed.

Scores summary faithfulness to source text via a 3-component LLM rubric: factual accuracy, key points coverage, no hallucinations.

Queries airline API to confirm flight rebooking fields match expected values (date, cabin class, passengers).

Pure-logic verifier checking agent action traces against domain policy rules. Works for any domain with codifiable constraints.

Queries retail API to confirm that a product SKU has the expected quantity in inventory after an agent action.

Queries retail API to confirm an order is in cancelled state with matching reason code.

Queries mock/real retail API to confirm a refund has been processed with the expected amount and status. Catches agents that claim refunds were issued but the actual state shows otherwise.

Verifies that a customer's telecom plan was changed to the expected plan with correct effective date via CRM API.

Navigates to a URL via headless browser and checks whether a specific CSS selector is present in the DOM. Catches agents that claim UI actions succeeded but never actually modified the page.

Captures a live screenshot via browser automation and compares it to a reference image using SSIM (Structural Similarity Index).

Verifies that an e-commerce order was placed with correct items and total via HTTP API query.

Verifies that a specific GitHub Actions workflow completed successfully.

Verifies that all CI check runs passed for a given commit SHA.

Verifies that a GitHub Pull Request was merged to the target branch.

Verifies that a message containing expected text exists in a Slack channel.

Verifies that a specific reaction was added to a Slack message.

Verifies that a Stripe charge was completed successfully.

Verifies that a Stripe refund was processed successfully.

Verifies that a Jira ticket has been transitioned to the expected status.