Current coverage
BuildnScale imports Arena’s official open leaderboard dataset. The overview uses the overall Text board with style control; Coding uses the overall Web Development board; Agents uses the aggregate Agent board. Other categories and modalities are not included in this release. These boards can have different publication dates and model coverage. A newly released model may not yet have published results.
Arena dataset and schema · Dataset announcement · Arena methodology changes
Measurements and ranks
Text and web development use Arena preference ratings based on its Bradley–Terry methodology. Text uses the style-controlled subset, not the raw text subset. Agent Arena uses IPS scores, which can be negative. These scales are not percentages and are not combined into an overall BuildnScale score. Web development ratings do not measure general coding accuracy or SWE-bench issue resolution.
We preserve the source’s published rank, full-precision score, confidence bounds, counts, model label, and publication date. Display rounding does not change rank. Filtering preserves ranks from the complete selected board. Agent counts are observations and sessions; text/web development counts are votes. Confidence intervals describe uncertainty; small gaps and overlapping intervals do not establish superiority. Read Agent Arena’s methodology for its interpretation and assumptions.
Model identities and missing data
Only exact model labels and organisations are joined across Arena boards. Reasoning variants remain distinct. When the same source label has multiple records in one board, we retain separate records marked with their source rank; their configurations are unknown and their internal URLs can change if those ambiguous ranks change. We do not guess equivalent provider API IDs, release dates, prices, context limits, or model licenses from a name. Missing measurements stay unavailable and unranked, never zero. A source model label is not proof of production availability.
Freshness and failure handling
Scheduled checks target every four hours. Scheduling delays, provider outages, and publication delays can occur. Each import reads all three latest overall boards from immutable Parquet files at one dataset revision and verifies the published license. Source publication dates are shown for each measurement; the summary uses the oldest included board date.
Last successful check and source publication are separate: checking unchanged data does not make the scores new. A notice appears after eight hours without a successful check, or when an included board is older than seven days. The page cache can take up to one hour to reflect a freshness warning. Validation failures retain the previous snapshot and fail the refresh job. The bundled, dated Arena snapshot is used when storage is unavailable; its original check time is preserved.
We validate identities, dates, counts, numeric values, confidence bounds, coverage drops, and publication regressions before saving. Imported revisions are retained in the update history. These checks detect malformed or suspicious updates; they cannot independently establish that the source’s evaluations are correct.
Attribution and licensing
Arena / lmarena-ai publishes this dataset under CC BY 4.0. BuildnScale filters the dataset to selected overall boards and normalizes its format. We have not rerun the evaluations. Attribution, source revision links, and license links accompany the results. Arena does not endorse BuildnScale. We do not scrape Arena’s website or bundle model weights, task text, or logos.
Historical results and optional sources
The EvalPlus archive preserves the historical Apache-2.0 coding dataset, initially updated on 26 December 2024. HumanEval/HumanEval+ cover 164 Python function tasks; MBPP/MBPP+ use EvalPlus’s sanitized 399-task subset. Pass@1 percentages are separate from Arena ratings. Non-Arena ranks use exact-score competition ranking (1, 1, 3). Artificial Analysis remains optional and disabled unless public-display rights are recorded. Its data is not required for the current Arena leaderboard.
Pricing, independence, and corrections
Arena’s dataset does not provide token prices or output speed. Those fields remain unavailable until a separately sourced, permitted feed is connected. There is no invented value score or paid rank placement. Validate candidates against your own application and agent workloads.
Report a correction with the model label, metric, revision, and evidence through GitHub issues. Inspect published snapshots in the update history.