Benchmarks/OpenBookQA

OpenBookQA

OpenBookQA — science questions that require combining a given core fact with broad common knowledge, mimicking an open-book exam setting.

knowledge

Models Tested

84.0

Top Score

48.4

Average Score

Rankings

#	Model	Score
1	phi-3-mini 3.8BMicrosoft	84.0
2	phi-3-small 7.4BMicrosoft	84.0
3	phi-3-medium 14BMicrosoft	83.2
4	Mixtral 8x7B InstructMistral AI	81.1
5	Llama 3 8B InstructMeta	76.8
6	Mistral 7B v0.1Mistral	73.1
7	Gemma 7BGoogle	71.5
8	Phi-2Microsoft	64.8
9	T Falcon-180BTII	52.3
10	LLaMA-65BMeta	46.9
11	Llama 2-70BMeta	46.9
12	LLaMA-33BMeta	44.8
13	Llama 2-7BMeta	44.8
14	Llama 2-34BMeta	44.3
15	LLaMA-7BMeta	42.9
16	Llama 2-13BMeta	42.7
17	T Falcon-40BTII	42.1
18	LLaMA-13BMeta	41.9
19	U MPT-30Bunknown	36.0
20	T Falcon-7BTII	35.5
21	U MPT-7Bunknown	35.2
22	Llama 3.1-405BMeta	32.3
23	Llama 3 70B InstructMeta	30.1
24	U XGen-7Bunknown	20.3
25	U Dolly 2.0-12bunknown	18.9
26	Phi-1.5Microsoft	16.3
27	Cerebras-GPT-13BOpenAI	14.4