Gecko Tests · Tokenizer TaxLatest run 2026-10-04 · 49 models · highest average tax Kimi K2.5 +100%
Tokenizer Tax · Same Text, Different Price
APIs bill by token, and tokenizers favor some languages. We send the same passage in nine languages to each model and read its own token count. 1.00 means as cheap as English; 1.80 means 80% more tokens for the same meaning.
| Model | en | fr | es | pt | it | de | ja | zh | ko | Avg tax | $ / 1k words · ja |
|---|---|---|---|---|---|---|---|---|---|---|---|
| DeepSeek V4 Flash | 1.00 | 1.03 | 1.04 | 1.03 | 1.39 | 1.09 | 1.09 | 0.65 | 1.27 | +7% | $0.0000 |
| DeepSeek V4 Pro 0813 | 1.00 | 1.03 | 1.43 | 1.03 | 1.39 | 1.48 | 1.09 | 0.65 | 1.27 | +17% | $0.0017 |
| MiniMax M2.7 | 1.00 | 1.28 | 1.30 | 1.28 | 1.56 | 1.64 | 1.20 | 1.01 | 1.49 | +35% | $0.0003 |
| Qwen3.8 Max Prime | 1.00 | 1.42 | 1.45 | 1.41 | 1.27 | 1.40 | 1.44 | 1.03 | 1.41 | +35% | $0.0063 |
| Qwen3.8 Max (0902) | 1.00 | 1.42 | 1.45 | 1.41 | 1.27 | 1.40 | 1.44 | 1.03 | 1.41 | +35% | $0.0032 |
| Qwen3.8 Flash | 1.00 | 1.42 | 1.45 | 1.41 | 1.27 | 1.40 | 1.44 | 1.03 | 1.41 | +35% | $0.0002 |
| MiniMax M2-her | 1.00 | 1.30 | 1.32 | 1.30 | 1.58 | 1.67 | 1.20 | 1.02 | 1.52 | +36% | $0.0004 |
| MiniMax M2.1 | 1.00 | 1.30 | 1.32 | 1.30 | 1.58 | 1.67 | 1.20 | 1.02 | 1.52 | +36% | $0.0004 |
| MiniMax M2 | 1.00 | 1.30 | 1.32 | 1.30 | 1.58 | 1.67 | 1.20 | 1.02 | 1.52 | +36% | $0.0004 |
| Qwen3.8 2.4T A95B | 1.00 | 1.49 | 1.52 | 1.41 | 1.27 | 1.47 | 1.44 | 1.10 | 1.41 | +39% | $0.0032 |
| Qwen3.8 27B | 1.00 | 1.42 | 1.45 | 1.41 | 1.27 | 1.40 | 1.85 | 1.03 | 1.41 | +41% | $0.0009 |
| Llama 4 Scout | 1.00 | 1.41 | 1.40 | 1.41 | 1.35 | 1.42 | 1.63 | 1.29 | 1.51 | +43% | $0.0002 |
| Grok 4.3 | 1.00 | 1.37 | 1.40 | 1.37 | 1.23 | 1.60 | 1.56 | 1.18 | 1.77 | +44% | $0.0022 |
| Grok 4.20 | 1.00 | 1.37 | 1.40 | 1.37 | 1.23 | 1.60 | 1.56 | 1.18 | 1.77 | +44% | $0.0022 |
| Gemini 3.8 Flash | 1.00 | 1.45 | 1.41 | 1.43 | 1.38 | 1.42 | 1.48 | 1.28 | 1.64 | +44% | $0.0012 |
| Gemini 3.7 Flash | 1.00 | 1.45 | 1.41 | 1.43 | 1.38 | 1.42 | 1.48 | 1.28 | 1.64 | +44% | $0.0012 |
| Gemini 3.6 Flash | 1.00 | 1.45 | 1.41 | 1.43 | 1.38 | 1.42 | 1.48 | 1.28 | 1.64 | +44% | $0.0012 |
| Gemini 3.5 Flash Lite | 1.00 | 1.45 | 1.41 | 1.43 | 1.38 | 1.42 | 1.48 | 1.28 | 1.64 | +44% | $0.0005 |
| Gemma 4 26B A4B | 1.00 | 1.45 | 1.41 | 1.43 | 1.38 | 1.42 | 1.48 | 1.28 | 1.64 | +44% | $0.0001 |
| Llama 4 Maverick | 1.00 | 1.40 | 1.39 | 1.40 | 1.38 | 1.42 | 1.62 | 1.28 | 1.64 | +44% | $0.0003 |
| Grok 4.7 | 1.00 | 1.50 | 1.49 | 1.48 | 1.42 | 1.48 | 1.43 | 1.23 | 2.00 | +50% | $0.0031 |
| Grok 4.6 | 1.00 | 1.50 | 1.49 | 1.48 | 1.42 | 1.48 | 1.43 | 1.23 | 2.00 | +50% | $0.0031 |
| Grok 4.5 | 1.00 | 1.50 | 1.49 | 1.48 | 1.42 | 1.48 | 1.43 | 1.23 | 2.00 | +50% | $0.0031 |
| GPT-6 Sol | 1.00 | 1.37 | 1.40 | 1.35 | 1.49 | 1.40 | 2.08 | 1.43 | 1.74 | +53% | $0.0048 |
| Mistral Small 4 | 1.00 | 1.37 | 1.42 | 1.48 | 1.40 | 1.41 | 2.05 | 1.66 | 1.47 | +53% | $0.0003 |
| Devstral 2 2512 | 1.00 | 1.37 | 1.42 | 1.48 | 1.40 | 1.41 | 2.05 | 1.66 | 1.47 | +53% | $0.0009 |
| Ministral 3 14B 2512 | 1.00 | 1.37 | 1.42 | 1.48 | 1.40 | 1.41 | 2.05 | 1.66 | 1.47 | +53% | $0.0005 |
| Ministral 3 3B 2512 | 1.00 | 1.37 | 1.42 | 1.48 | 1.40 | 1.41 | 2.05 | 1.66 | 1.47 | +53% | $0.0002 |
| GPT-6 Luna | 1.00 | 1.39 | 1.42 | 1.37 | 1.52 | 1.42 | 2.14 | 1.45 | 1.78 | +56% | $0.0002 |
| GPT-6 Luna Pro | 1.00 | 1.39 | 1.42 | 1.38 | 1.52 | 1.42 | 2.14 | 1.46 | 1.79 | +57% | $0.0007 |
| MiniMax M2.5 | 1.00 | 2.39 | 1.32 | 1.30 | 1.58 | 1.67 | 2.29 | 1.02 | 1.54 | +64% | $0.0007 |
| DeepSeek V4.1 Flash | 1.00 | 1.69 | 1.71 | 1.69 | 1.65 | 1.78 | 1.79 | 1.06 | 2.07 | +68% | $0.0003 |
| DeepSeek V4 Flash 0731 | 1.00 | 1.69 | 1.71 | 1.69 | 1.65 | 1.78 | 1.79 | 1.06 | 2.07 | +68% | $0.0000 |
| DeepSeek V4 Pro | 1.00 | 1.69 | 1.71 | 1.69 | 1.65 | 1.78 | 1.79 | 1.06 | 2.07 | +68% | $0.0004 |
| GLM 5.3 Prime | 1.00 | 1.67 | 1.64 | 1.68 | 1.60 | 1.60 | 1.89 | 1.09 | 2.40 | +70% | $0.0058 |
| GLM 5.3 FlashX | 1.00 | 1.67 | 1.64 | 1.68 | 1.60 | 1.60 | 1.89 | 1.09 | 2.40 | +70% | $0.0008 |
| GLM 5.3 Flash | 1.00 | 1.67 | 1.64 | 1.68 | 1.60 | 1.60 | 1.89 | 1.09 | 2.40 | +70% | $0.0003 |
| GLM 5.1 | 1.00 | 1.67 | 1.69 | 1.73 | 1.60 | 1.60 | 1.89 | 1.09 | 2.40 | +71% | $0.0020 |
| Claude Sonnet 5.5 | 1.00 | 1.69 | 1.76 | 1.67 | 1.66 | 2.04 | 1.58 | 1.30 | 2.11 | +73% | $0.0052 |
| Claude Opus 5.5 | 1.00 | 1.69 | 1.76 | 1.67 | 1.66 | 2.04 | 1.58 | 1.30 | 2.11 | +73% | $0.0105 |
| Claude Fable 5.1 | 1.00 | 1.69 | 1.76 | 1.67 | 1.66 | 2.04 | 1.58 | 1.30 | 2.11 | +73% | $0.0262 |
| Claude Opus 5 | 1.00 | 1.69 | 1.76 | 1.67 | 1.66 | 2.04 | 1.58 | 1.30 | 2.11 | +73% | $0.0131 |
| Claude Opus 5 (Fast) | 1.00 | 1.69 | 1.76 | 1.67 | 1.66 | 2.04 | 1.58 | 1.30 | 2.11 | +73% | $0.0262 |
| GLM 5.3 | 1.00 | 1.84 | 1.80 | 1.85 | 2.00 | 1.75 | 2.11 | 1.11 | 2.75 | +90% | $0.0026 |
| Kimi K2 0905 | 1.00 | 1.94 | 1.94 | 1.90 | 1.85 | 2.05 | 2.13 | 0.99 | 2.55 | +92% | $0.0014 |
| Kimi K2.6 | 1.00 | 1.94 | 1.94 | 1.91 | 1.85 | 2.05 | 2.13 | 0.99 | 2.55 | +92% | $0.0022 |
| Kimi K2 Thinking | 1.00 | 1.94 | 1.94 | 1.90 | 1.85 | 2.05 | 2.13 | 1.02 | 2.55 | +92% | $0.0014 |
| Kimi K3 | 1.00 | 1.94 | 1.94 | 1.94 | 1.85 | 2.05 | 2.15 | 1.02 | 2.55 | +93% | $0.0017 |
| Kimi K2.5 | 1.00 | 1.94 | 2.09 | 2.06 | 1.85 | 2.20 | 2.28 | 0.99 | 2.55 | +100% | $0.0011 |
Green · within 15% of English. Amber · up to 50% more. Pink · more than 50% more tokens.
How it works
- One passage of everyday prose (about 110 English words), translated once into eight languages and frozen in the dataset, so every model sees identical text.
- Each passage is sent as a single message with a 16-token reply cap. The token count comes from the model's own API usage, minus a one-character baseline that removes the chat template.
- Ratio = tokens in a language divided by tokens in English. The average tax is the mean extra percentage across the eight other languages.
- Cost per 1,000 words uses the model's list input price. Tokenizers rarely change, so each model is measured once per dataset version.
Open dataset · v1
9 items, every model answer and every score are public under CC BY 4.0. Cite as “BenchGecko Gecko Tests, Tokenizer Tax v1” with a link.
Download JSONFrequently Asked Questions
Tokenizers are trained mostly on English text, so other scripts and longer words split into more pieces. Chinese, Japanese and Korean often need far more tokens per sentence on older tokenizers.