Motivation
A convincing final answer does not tell us whether an agent used the right tools to reach it.
Tool-using agents make multiple decisions between receiving a prompt and producing a response. They select tools, supply arguments, inspect outputs, and generate claims based on information they have seen. A final-answer-only test can miss an unnecessary tool call, a skipped lookup, or a response that includes unsupported details.
AgentBench treats the agent run as the unit of evaluation. Rather than checking only the last message, it combines the execution trace with the final answer, then compares that evidence against scenario-level expectations. The goal is to make failures easier to locate and regressions easier to notice after a model or prompt change.
Core design choice. Evaluate both what the agent did and what it said. A pass requires the configured thresholds for all three metrics, rather than a single aggregate score.
System architecture
AgentBench is organized around six components. YAML files define expected behavior. The harness starts a run and invokes an agent runner. An interceptor wraps LangChain tools and records calls. The scoring engine evaluates the resulting trajectory. SQLite keeps run history, and a dashboard presents trends and regression flags.
The flow reflects the repository’s evaluation harness, scoring engine, interceptor and persistence modules.
Trajectory capture
The TrajectoryInterceptor wraps synchronous and asynchronous tools. Each invocation records the tool name, input arguments, returned value or error, execution latency, and a timestamp. Optional reasoning text and token counts can be supplied by the agent runner. Captured tool nodes include parent identifiers, forming a directed graph representation.
The current implementation links each new tool call to the previous one. This preserves sequential execution history, but does not yet capture a general parallel branching graph. The interceptor also cannot infer hidden model reasoning. It records only traces explicitly supplied through its interface.
Local storage and review
SQLite stores four kinds of records: runs, trajectories, scenario results, and regression flags. Run metadata includes model and prompt-version labels, allowing comparisons over time. The Streamlit interface reads this history to show pass rates, metric distributions, and flagged changes without requiring an external database.
Evaluation methodology
Each scenario defines a prompt, an expected tool sequence, a reference answer, supporting context, and optional metric thresholds. The harness executes the agent, finalizes its trace, and calculates three scores concurrently with a ThreadPoolExecutor.
Did the agent invoke the expected tools with the expected arguments?
How closely does the final answer match the reference answer in meaning?
How much of the final answer is supported by supplied context or tool output?
Tool-call accuracy
The implementation normalizes each tool invocation into a tool name and argument set, then computes Levenshtein edit distance between actual and required call sequences. A match requires the same tool name, with expected arguments treated as a required subset. Optional calls are excluded from the required sequence and can contribute a small bonus when present.
Semantic similarity
The system compares the final response with a gold response using cosine similarity. When available, it uses all-MiniLM-L6-v2 embeddings from sentence-transformers. If that dependency cannot load, it falls back to a token-frequency cosine score. The two methods are useful for local evaluation but are not equivalent, so results should be compared with the scoring method held constant.
Faithfulness
The faithfulness heuristic splits the answer into sentence-like claims, then checks for support in the scenario context and captured tool outputs. A claim counts as supported when token overlap reaches 0.4 or a fuzzy trigram check succeeds. The final score is the proportion of supported claims. This is an inexpensive grounding proxy, not a full logical entailment check.
A scenario passes only when all three scores meet their corresponding thresholds. In the source implementation, default thresholds are 0.70 for tool-call accuracy, 0.60 for semantic similarity, and 0.50 for faithfulness unless the YAML scenario overrides them.
Regression detection
A passing run can still be worse than earlier runs. To catch those changes, AgentBench compares each new score with prior values for the same scenario and metric. The default window contains up to 30 historical results and requires at least three values before statistical comparison begins.
flag when z ≤ −1.5
When prior scores have essentially zero variance, the implementation treats a measurable drop below the historical mean as a regression rather than dividing by zero. The resulting flag records the scenario, metric, current value, baseline statistics, model label, prompt version, and run identifier.
Interpretation. A z-score flag is an alert about a change from a local baseline. It is not, by itself, statistical proof of a production failure. Small histories, repeated deterministic runs, and shifts in test composition affect how meaningful the baseline is.
Reported demo results
The project website reports a seeded demonstration with four scenarios and nine runs per scenario, giving 36 scenario-level results. It includes normal, degraded, and broken agent variants. These figures come from the project’s published demonstration; they are not an independently rerun benchmark or a measurement of real-world production traffic.
| Scenario | Pass | Tool | Semantic | Faithfulness |
|---|---|---|---|---|
| weather_austin | 8 / 9 | 0.89 | 0.88 | 0.89 |
| company_lookup | 8 / 9 | 0.89 | 0.87 | 0.89 |
| company_tax | 6 / 9 | 0.89 | 0.86 | 0.33 |
| order_status | 8 / 9 | 0.89 | 0.87 | 0.89 |
Scores are published averages, rounded to two decimal places. Pass counts total 30 of 36 scenario evaluations.
The tax scenario stands out. It has similar average tool-call accuracy to the other cases but substantially lower faithfulness. That separation is useful: a system can follow the expected tool path and still produce claims that the available evidence does not support.
The demo page also reports a 100% pass rate for a good run and 75% for a degraded run. Those illustrative figures show that the harness distinguishes selected failure cases. They do not establish detection accuracy across unseen models, tasks, or prompt distributions.
Reproducibility & integration
The repository provides a Python package, demo agent, YAML scenarios, command-line entry points, and automated tests. A developer can install the project, seed example results, run an evaluation, and launch the Streamlit dashboard.
Commands are drawn from the repository README. The virtual-environment activation line is for Unix-like shells.
Connecting another agent
Integration centers on a small callable interface. The agent receives a scenario and interceptor, wraps the tools it plans to use, and returns its final answer. The harness handles scoring, persistence, and baseline comparison. This keeps evaluation concerns separate from application logic, although developers still need to supply test cases and trustworthy reference data.
Limitations & next steps
Current constraints
- Sequential parent links do not model true parallel tool fan-out.
- Overlap-based grounding can mistake shared vocabulary for factual support.
- Semantic scores depend on embedding availability and reference-answer quality.
- Demo data is small and partly deterministic, limiting generalization.
Useful next steps
- Add first-class provider adapters and stronger usage accounting.
- Support CI gates for new failures and meaningful score changes.
- Offer entailment-based faithfulness checks as an optional metric.
- Expand scenarios to multi-hop work, safety cases, latency, and cost.
These directions align with the project’s published roadmap. A particularly useful improvement would be a held-out benchmark with manually reviewed trajectory and grounding labels. That would help quantify false-positive and false-negative behavior in the scoring heuristics.
Conclusion
AgentBench makes agent evaluation more informative by looking beyond a final answer. It records what tools were called, evaluates whether the output matches expectations and supporting evidence, and compares results with prior runs. Its local architecture makes it practical for small experiments and repeatable development checks.
The main contribution is not a new benchmark score. It is a workflow for turning agent execution into inspectable evidence. With broader scenarios, better grounding checks, and more realistic baselines, the same design could support stronger quality controls during agent development.
References & source material
[1] Sharma, D. AgentBench, source repository and README. github.com/b4batunde/AgentBench.
[2] AgentBench project website, architecture, methodology, demonstration results, and roadmap. agentbench.netlify.app.
[3] AgentBench source modules: interceptor.py, scoring.py, store.py, and harness.py, main branch of [1].