Integration Guide

LangChain / LangGraph

Install with LangChain support:

pip install vrdev[langchain]

Option 1: Verify Tool (agent self-verification)

Give your agent a tool it can call to verify its own work:

from vrdev import get_verifier, compose
from vrdev.adapters.langchain import VrdevVerifyTool

pipeline = compose([
    get_verifier("vr/tau2.retail.order_cancelled"),
    get_verifier("vr/rubric.email.tone_professional"),
], policy_mode="fail_closed")

tool = VrdevVerifyTool(pipeline)

# Add to any LangChain agent
from langchain.agents import create_react_agent
agent = create_react_agent(llm, tools=[search_tool, api_tool, tool])

The tool accepts JSON input with completion and ground_truth keys and returns the verdict, score, and evidence.

Option 2: Callback Handler (automatic post-action verification)

Auto-verify when the agent finishes, with results injected into run metadata (visible in LangSmith):

from vrdev import get_verifier
from vrdev.adapters.langchain import VrdevCallbackHandler

verifier = get_verifier("vr/tau2.retail.order_cancelled")
handler = VrdevCallbackHandler(
    verifier,
    ground_truth={"order_id": "ORD-42"},
)

result = agent.invoke(
    {"input": "Cancel order ORD-42"},
    config={"callbacks": [handler]},
)

# Check the verdict
print(handler.last_result)
# {"verdict": "PASS", "score": 1.0, "passed": True, ...}

Option 3: LangGraph Verification Node

Add verification as a step in your LangGraph workflow:

from langgraph.graph import StateGraph
from vrdev import get_verifier, compose
from vrdev.adapters.langgraph import verify_node

pipeline = compose([
    get_verifier("vr/tau2.retail.order_cancelled"),
], policy_mode="fail_closed")

graph = StateGraph(dict)
graph.add_node("act", act_fn)
graph.add_node("verify", verify_node(
    pipeline,
    completion_key="output",
    ground_truth_key="expected",
))
graph.add_edge("act", "verify")
graph.set_entry_point("act")

# The "verify" node reads state["output"] and state["expected"],
# runs the pipeline, and writes the result to state["verification"].

When to use which

| Pattern | When to use | Runs | |---------|-------------|------| | VrdevVerifyTool | Agent decides when to verify (self-check) | On agent tool call | | VrdevCallbackHandler | Auto-verify every agent run | On agent finish | | verify_node | Verification is a graph step with routing logic | As LangGraph node | | pipeline.verify() | Direct call in any Python code | Anywhere |

TRL (Transformer Reinforcement Learning)

Export verified rewards directly to TRL format:

from vrdev import get_verifier, compose, VerifierInput, export_to_trl
from vrdev.core.types import PolicyMode

pipeline = compose(
    [get_verifier("vr/tau2.retail.order_cancelled"),
     get_verifier("vr/rubric.email.tone_professional")],
    require_hard=True,
    policy_mode=PolicyMode.FAIL_CLOSED,
)

# Run verification across your eval set
results = [pipeline.verify(episode) for episode in eval_set]

# Export TRL-compatible reward dataset
export_to_trl(results, output="rewards.jsonl")

The output JSONL contains one record per verification with the reward signal, evidence hash, and trajectory reference.

VERL

For VERL-based training:

from vrdev import export_to_verl

export_to_verl(results, output="verl_rewards/")

This generates the directory structure VERL expects, including reward shaping metadata.

OpenClaw Adapter

The openclaw.cancel_and_email skill in the registry shows how to integrate vr.dev verifiers as the reward function for an OpenClaw skill:

from openclaw import Skill
from vrdev import compose

reward_fn = compose([
    "tau2.retail.order_cancelled",
    "aiv.email.sent_folder_confirmed",
    "rubric.email.tone_professional",
], policy_mode="fail_closed")

skill = Skill(
    name="cancel_and_email",
    reward=reward_fn,
    max_steps=15,
)

Custom Training Loops

For arbitrary training frameworks, use the raw result objects:

from vrdev import get_verifier, VerifierInput

v = get_verifier("vr/code.python.tests_pass")
result = v.verify(VerifierInput(
    completions=["agent output here"],
    ground_truth={"repo": ".", "test_cmd": "pytest"},
))

reward = result[0].score          # float 0.0-1.0
passed = result[0].passed         # bool
evidence = result[0].evidence     # dict
hash = result[0].evidence_hash    # str, sha256:...

Map these to your framework's reward format as needed.

CI/CD Integration

Add verification to your CI pipeline:

# .github/workflows/verify.yml
- name: Verify agent outputs
  run: |
    pip install vrdev
    vr compose \
      --verifiers code.python.lint_ruff,code.python.tests_pass \
      --ground-truth '{"repo": "."}' \
      --policy fail_closed \
      --output json