Inspect results for a specific evaluation. Source coverage, test suite, and prompting configuration matter as much as the score.
Loading explorer…
Published evidence. Clear sources. No sponsored rank placement.
How to read these results →125 models · HumanEval+ · Higher is better
Python function generation tested against EvalPlus expanded tests. Higher pass@1 is better. Ranks apply only to this source snapshot and selected metric. Arena ranks are preserved from the source; other ranks use exact scores. Small differences do not establish statistical superiority.
| Compare | Rank | Model / evaluated label | HumanEval+ | Mode | Parameters | Source |
|---|---|---|---|---|---|---|
| #1 | O1 Mini (Sept 2024)Not specified | 89.0% | Instruction | Not provided | Evidence | |
| #1 | O1 Preview (Sept 2024)Not specified | 89.0% | Instruction | Not provided | Evidence | |
| #3 | GPT 4o (Aug 2024)Not specified | 87.2% | Instruction | Not provided | Evidence | |
| #3 | Qwen2.5-Coder-32B-InstructQwen | 87.2% | Instruction | 32B | Evidence | |
| #5 | DeepSeek-V3 (Nov 2024)deepseek-ai | 86.6% | Instruction | 37B | Evidence | |
| #5 | GPT-4-Turbo (April 2024)Not specified | 86.6% | Instruction | Not provided | Evidence | |
| #7 | DeepSeek-V2.5 (Nov 2024)deepseek-ai | 83.5% | Instruction | 21B | Evidence | |
| #7 | GPT 4o Mini (July 2024)Not specified | 83.5% | Instruction | Not provided | Evidence | |
| #9 | DeepSeek-Coder-V2-Instructdeepseek-ai | 82.3% | Instruction | 21B | Evidence | |
| #10 | Claude Sonnet 3.5 (June 2024)Not specified | 81.7% | Instruction | Not provided | Evidence | |
| #10 | GPT-4-Turbo (Nov 2023)Not specified | 81.7% | Instruction | Not provided | Evidence | |
| #12 | Grok BetaNot specified | 80.5% | Instruction | Not provided | Evidence | |
| #13 | Gemini 1.5 Pro 002Not specified | 79.3% | Instruction | Not provided | Evidence | |
| #13 | GPT-4 (May 2023)Not specified | 79.3% | Instruction | Not provided | Evidence | |
| #15 | CodeQwen1.5-7B-ChatQwen | 78.7% | Instruction | 7B | Evidence | |
| #16 | claude-3-opus (Mar 2024)Not specified | 77.4% | Instruction | Not provided | Evidence | |
| #16 | OpenCoder-8B-Instructinfly | 77.4% | Instruction | 8B | Evidence | |
| #18 | Gemini 1.5 Flash 002Not specified | 75.6% | Instruction | Not provided | Evidence | |
| #19 | DeepSeek-Coder-33B-instructdeepseek-ai | 75.0% | Instruction | 33B | Evidence | |
| #20 | Codestral-22B-v0.1mistralai | 73.8% | Instruction | 22B | Evidence | |
| #20 | OpenCodeInterpreter-DS-33Bm-a-p | 73.8% | Instruction | 33B | Evidence | |
| #22 | WizardCoder-33B-V1.1WizardLM | 73.2% | Instruction | 33B | Evidence | |
| #23 | Artigenz-Coder-DS-6.7BArtigenz | 72.6% | Instruction | 6.7B | Evidence | |
| #24 | Llama3-70B-instructmeta-llama | 72.0% | Instruction | 70B | Evidence | |
| #24 | Mixtral-8x22B-Instruct-v0.1mistralai | 72.0% | Instruction | 44B | Evidence |
#12
Grok BetaEvalPlus contributors — results redistributed under Apache-2.0. Normalized by BuildnScale; no evaluations rerun. Parameter counts and instruction labels are source metadata. Namespace is shown where supplied; it is not a verified API provider or license classification.