Benchmarks/Fiction.LiveBench

Fiction.LiveBench

Fiction.LiveBench — a continuously updated benchmark using recently published fiction to test reading comprehension and reasoning, preventing data contamination.

knowledge

Models Tested

97.2

Top Score

60.4

Average Score

Rankings

#	Model	Score
1	GPT-5 ChatOpenAI	97.2
2	GPT-5OpenAI	97.2
3	o3 ProOpenAI	97.2
4	Grok 4 FastxAI	94.4
5	Grok 4xAI	94.4
6	Gemini 2.5 ProGoogle DeepMind	91.7
7	o3OpenAI	88.9
8	M Kimi K2.5moonshotai	86.1
9	DeepSeek V3.2 ExpDeepSeek	83.3
10	Claude 3.7 SonnetAnthropic	83.3
11	Claude 3.7 Sonnet (thinking)Anthropic	83.3
12	o1OpenAI	83.3
13	o4 MiniOpenAI	77.8
14	Qwen3 235B A22B Thinking 2507Alibaba Qwen	75.0
15	DeepSeek-R1 (May 2025)DeepSeek	75.0
16	GPT-5 MiniOpenAI	69.4
17	DeepSeek-R1DeepSeek	69.4
18	Qwen3 235B A22BAlibaba Qwen	67.7
19	Qwen3 MaxAlibaba Qwen	66.7
20	M Kimi K2 0905moonshotai	66.7
21	Grok 3 MinixAI	66.7
22	Grok 3 Mini BetaxAI	66.7
23	Qwen2.5-MaxAlibaba	66.7
24	GPT-4.1OpenAI	63.9
25	GPT-4.5OpenAI	63.9
26	M Kimi K2 0711moonshotai	61.1
27	Claude Opus 4Anthropic	61.1
28	Grok 3xAI	58.3
29	Grok 3 BetaxAI	58.3
30	Qwen3-235B-A22B-Instruct (Jul 2025)Alibaba	52.9
31	DeepSeek V3.1DeepSeek	52.8
32	Gemini 2.0 Flash Thinking (Jan 2025)Google	52.8
33	o3 MiniOpenAI	50.0
34	Gemini 2.5 FlashGoogle DeepMind	47.2
35	Claude Sonnet 4Anthropic	46.9
36	Llama 4 MaverickMeta	46.2
37	GPT-5 NanoOpenAI	44.4
38	gpt-oss-120b (free)OpenAI	44.4
39	gpt-oss-120bOpenAI	44.4
40	gpt-oss-20b (free)OpenAI	44.4
41	gpt-oss-20bOpenAI	44.4
42	GPT-4.1 MiniOpenAI	44.4
43	Gemini 2.0 ProGoogle	41.7
44	Llama 4 ScoutMeta	36.0
45	Gemma 3 4B (free)Google DeepMind	33.3
46	Gemma 3 4BGoogle DeepMind	33.3
47	Gemma 3 12B (free)Google DeepMind	33.3
48	Gemma 3 12BGoogle DeepMind	33.3
49	Gemma 3 27B (free)Google DeepMind	33.3
50	Gemma 3 27BGoogle DeepMind	33.3
51	Llama 3.3 70B Instruct (free)Meta	33.3
52	Llama 3.3 70B InstructMeta	33.3
53	GPT-4.1 NanoOpenAI	25.0

Related Benchmarks