What benchmark actually predicts agent quality?

tau-bench (customer service tool use), WebArena (browser tasks), GAIA (general assistants), and SWE-bench Verified (repo-level code agents). Single-turn benchmarks like MMLU do not predict agent performance well.

Should I use a cheap model inside my agent loop?

Yes, for most steps. Route the hard planning step to a frontier model (Claude Opus, GPT-5), then use a cheap model for routine tool calls, retries, and tool-result summarization. See the agent stack for the full pattern.

Which providers support reliable tool calling?

Anthropic, OpenAI, Google, Mistral, Together, and Fireworks all have stable JSON-mode tool calling. DeepSeek and Qwen work for most cases but occasionally hallucinate tool names. Test on your schema.

How do I keep an agent under a budget cap?

Set a hard token cap per run, enable prompt caching (Anthropic, OpenAI, DeepInfra all support it), and log every call to detect runaway loops.

Use case · Agents

Cheapest agent LLMs

The cheapest LLMs that score well on agentic benchmarks. Ranked by price per 1M input tokens.

Models30

Cheapest$0.00

ScopeSWE · tau · WebArena

All pricing Pricing home

What this page is

This page ranks every LLM with meaningful agentic scores (SWE-bench, tau-bench, WebArena, GAIA, AgentBench) by input price. Agent workloads spike token consumption because of looping, tool schemas, and retries. Choosing a cheaper model for routine tool calls while reserving frontier models for planning can cut agent bills by 50 to 90 percent.

Ranked by input price

Models with agent benchmark scores, cheapest first.

#	Model	Provider	In $/1M	Out $/1M	Context	avg score	Type
1	Qwen3 Next 80B A3B Instruct (free)	Alibaba Qwen	$0.00	$0.00	262K	0.0	OSS
2	gpt-oss-120b (free)	OpenAI	$0.00	$0.00	131K	68.7	OSS
3	LFM2.5-1.2B-Thinking (free)	liquid	$0.00	$0.00	33K	0.0	OSS
4	Gemma 4 31B (free)	Google DeepMind	$0.00	$0.00	262K	0.0	OSS
5	gpt-oss-20b (free)	OpenAI	$0.00	$0.00	131K	66.4	OSS
6	LFM2.5-1.2B-Instruct (free)	liquid	$0.00	$0.00	33K	0.0	OSS
7	Gemma 4 26B A4B (free)	Google DeepMind	$0.00	$0.00	262K	0.0	OSS
8	Qwen3 Coder 480B A35B (free)	Alibaba Qwen	$0.00	$0.00	1.0M	0.0	OSS
9	DeepSeek V4.1 Flash 1017	DeepSeek	$0.01	$0.50	1.0M	64.5	OSS
10	GLM 5.2	z-ai	$0.01	$6.80	1.0M	56.3	OSS
11	Granite 4.0 Micro	ibm-granite	$0.02	$0.11	131K	0.0	OSS
12	LFM2-24B-A2B	liquid	$0.03	$0.12	128K	0.0	OSS
13	GLM 5.3 Flash	z-ai	$0.04	$0.08	1.0M	48.1	Closed
14	Granite 4.1 8B	ibm-granite	$0.05	$0.10	131K	0.0	OSS
15	GPT-5 Nano	OpenAI	$0.05	$0.40	400K	39.1	Closed
16	DeepSeek V4.1 Flash	DeepSeek	$0.05	$0.57	1.0M	54.1	OSS
17	GLM 5.3	z-ai	$0.07	$1.40	1.0M	56.7	Closed
18	Nemotron 3 Super	NVIDIA	$0.07	$0.40	262K	32.5	OSS
19	I Ring-2.6-1T	inclusionai	$0.07	$0.63	262K	0.0	Closed
20	MiMo-V2-Flash	xiaomi	$0.09	$0.29	262K	73.3	OSS
21	Qwen3 Next 80B A3B Instruct	Alibaba Qwen	$0.09	$1.10	262K	54.4	OSS
22	DeepSeek V4 Flash	DeepSeek	$0.09	$0.18	1.0M	61.6	OSS
23	Llama 4 Scout	Meta	$0.10	$0.30	1.3M	19.5	OSS
24	Gemini 2.5 Flash Lite	Google DeepMind	$0.10	$0.40	1.0M	48.6	Closed
25	Gemini 2.0 Flash	Google DeepMind	$0.10	$0.40	1.0M	48.0	Closed
26	Qwen3.6 35B A3B	Alibaba Qwen	$0.10	$0.90	262K	42.2	OSS
27	GPT-6 Luna	OpenAI	$0.10	$0.50	1.1M	49.8	Closed
28	Claude Haiku 5.5	Anthropic	$0.10	$0.50	1.0M	57.6	Closed
29	Step 3.5 Flash	stepfun	$0.10	$0.30	262K	76.9	OSS
30	DeepSeek V4 Pro 0813	DeepSeek	$0.11	$4.44	1.0M	58.7	Closed

Top 3 cheapest agent LLMs

Cheapest agent LLM

Qwen3 Next 80B A3B Instruct (free)

Qwen3 Next 80B A3B Instruct (free) has agentic scores (SWE-bench, tau-bench, WebArena) and lands at $0.00 per 1M input tokens. Useful when agent loops spike token spend.

gpt-oss-120b (free) has agentic scores (SWE-bench, tau-bench, WebArena) and lands at $0.00 per 1M input tokens. Useful when agent loops spike token spend.

Third choice

LFM2.5-1.2B-Thinking (free)

LFM2.5-1.2B-Thinking (free) has agentic scores (SWE-bench, tau-bench, WebArena) and lands at $0.00 per 1M input tokens. Useful when agent loops spike token spend.

The price gap · cheapest vs most expensive

Cheapest

Qwen3 Next 80B A3B Instruct (free)

$0.00/M

$ per 1M input tokens

Why the gap

Premium agent models pay for more reliable tool use, fewer hallucinated tool names, and longer-horizon planning. For simple tool loops, the cheap end works fine.

Most expensive

DeepSeek V4 Pro 0813

$0.11/M

$ per 1M input tokens

Frequently asked questions

Agents loop. A single task may produce 10 to 100 LLM calls with heavy context re-use. That amplifies input costs (because the system prompt and tool schema repeat) and multiplies output costs for every thinking step.

Cheapest agent LLMs

Ranked by input price

Top 3 cheapest agent LLMs

The price gap · cheapest vs most expensive

Frequently asked questions

See also

Related pricing

Stacks

Compare