AI Model Rankings — Wilson Score Benchmarks for Coding, Reasoning & Planning

🔬 Test Methodology

What is "RAW" Testing?

RAW tests evaluate models with direct prompts — no augmentation, no prompt engineering, no external tools. This measures each model's baseline capability across coding, reasoning and planning tasks when given a pure task description.

Test Pipeline

1️⃣ Task Prompt

Send a coding, reasoning or planning challenge directly to the model with clear requirements

2️⃣ Model Response

Model returns implementation code + unit tests

3️⃣ Vitest Run

Execute unit tests to verify correctness

4️⃣ TSC Check

TypeScript Compiler checks type safety

5️⃣ Wilson Score

Calculate adjusted ranking score

📐 How Rankings Are Calculated

⚠️ The 100% Problem

Simple pass rate (passes / total) is misleading:

1 pass, 0 fail = 100% (but only 1 test!)
10 pass, 1 fail = 90.9% (but 11 tests!)

A model that passes 1/1 looks better than one that passes 10/11. That's wrong.

🎯 Solution: Adjusted Wilson Score

Wilson Score Lower Bound accounts for both pass rate AND sample size:

Wilson = (p + z²/2n - z√[p(1-p)/n + z²/4n²] / (1+z²/n)) × n

Where:

p = pass rate (passes / total)
n = total tests attempted
z = 1.96 (95% confidence)

Result: A model with 10/11 passes gets a higher Wilson score than 1/1 passes. More tests = more confidence = fair ranking.

Score Components

Metric	Description
Pass	All unit tests passed
Fail	Tests ran but failed (logic/implementation errors)
Wilson	Adjusted Wilson Score (primary ranking)
Pass Rate	For reference only (not used for ranking)

🏆 Overall Rankings (All Categories Combined)

Ranked by Adjusted Wilson Score — higher is better

Rank	Model	Pass	Fail	Wilson	Rate
1	deepseek-v3.2	47	6	41.03	88.7%
2	qwen3-coder-next	43	9	36.54	82.7%
3	grok-4	40	3	35.00	93.0%
4	minimax-m2.5	41	8	34.77	83.7%
5	glm-4.7	39	10	32.52	79.6%
6	hunter	34	0	30.55	100%
7	glm-5	34	1	29.91	97.1%
7	healer	34	1	29.91	97.1%
9	qwen3-coder	31	3	26.19	91.2%
10	haiku-4.5	27	1	23.04	96.4%

Rank	Model	Pass	Fail	Wilson	Rate
11	gemini-2.5-pro	26	0	22.65	100%
12	grok-4.1-fast-reason	23	5	18.03	82.1%
13	deepseek-v3	22	7	16.79	75.9%
14	kimi-k2.5	18	0	14.83	100%
15	gpt-5-mini	15	0	11.94	100%
16	gpt-oss-120b	15	1	11.47	93.8%
17	qwen3.5-397b	14	0	10.99	100%
18	gpt-5.2	14	5	10.52	73.7%
18	codestral	14	5	10.52	73.7%
20	o3	13	2	9.82	86.7%
21	qwen35-9b	15	20	9.79	42.9%
22	gemini-3.1-pro	13	0	9.67	100%
23	grok-code-fast-1	12	6	8.00	66.7%
23	o4-mini	12	3	8.00	80.0%
25	devstral-2	10	8	5.56	55.6%
26	devstral-medium	7	7	4.00	50.0%
27	gemini-3-pro	6	0	3.00	100%
28	gemini-3-flash	5	0	2.50	100%
29	grok-4.1-fast-noreas	5	8	2.00	38.5%
30	gpt-5-nano	1	2	0.33	33.3%

🦀 Rust Rankings

Ranked by Adjusted Wilson Score — raw tests only, 9 tasks

Rank	Model	Pass	Fail	Wilson
1	opus-4.5	96	1	91.56
2	opus-4.6	86	0	82.32
3	gpt-5.3	77	1	72.61
4	gemini-3-pro	63	0	59.38
5	gemini-3.1-pro	60	0	56.39
6	gpt-5.4	56	0	52.41
7	gpt-5.2	52	0	48.42
8	gpt-5.2-codex	48	0	44.44
9	haiku-4.5	45	1	40.79
10	healer	40	1	35.84

Rank	Model	Pass	Fail	Wilson
11	hunter	39	0	35.50
11	gemini-3-flash	39	0	35.50
13	o4-mini	38	0	34.51
14	deepseek-v3.2-exp	38	3	33.03
15	gpt-5.1	37	1	32.87
16	qwen3.5-397b	36	0	32.53
17	o3	35	0	31.54
17	gpt-5-nano	35	0	31.54
19	qwen3-coder	35	1	30.90
20	gpt-5.1-codex	34	0	30.55
21	mimo-v2-flash	33	1	28.93
22	glm-5	32	0	28.57
23	gpt-oss-120b	32	1	27.94
24	kimi-k2.5	31	0	27.58
25	deepseek-v3.2	30	1	25.98
26	qwen3-max-thinking	28	0	24.62
27	grok-code-fast-1	26	1	22.06
28	grok-4	24	0	20.69
28	gpt-5-mini	24	0	20.69
30	mistral-large-2512	22	1	18.17
31	glm-4.7	21	0	17.75
32	command-a	21	1	17.20
33	trinity-large	18	0	14.83
33	codestral	18	0	14.83
35	gemini-2.5-flash	19	3	14.67
36	deepseek-v3	17	0	13.87
37	step-3.5-flash	14	1	10.53
38	qwen3-coder-next	12	0	9.09
39	devstral-medium	10	0	7.22
40	minimax-m2.5	10	1	6.85
41	devstral-2	10	3	6.47
42	gemini-2.5-pro	9	0	6.31
43	llama-4-maverick	7	0	4.52
44	deepseek-v3.2-speciale	6	0	3.66
45	seed-2.0-mini	4	0	2.04
45	nova-premier	4	0	2.04
45	llama-4-scout	4	0	2.04
48	grok-4.1-fast	3	0	1.32

📝 TypeScript Rankings

Ranked by Adjusted Wilson Score — raw tests only, 3 tasks

Rank	Model	Pass	Fail	Wilson
1	minimax-m2.5	52	9	45.31
2	opus-4.5	34	1	29.91
3	grok-4.1-fast	35	4	29.80
4	opus-4.6	33	1	28.93
5	qwen3-coder	33	3	28.14
6	haiku-4.5	32	3	27.17
7	grok-4	31	1	26.96
8	qwen3-coder-next	28	4	23.02
9	glm-4.7	25	4	20.14
10	gpt-5.3	22	0	18.73

Rank	Model	Pass	Fail	Wilson
10	gpt-5.2-codex	22	0	18.73
10	gpt-5.2	22	0	18.73
13	gpt-5-mini	21	1	17.20
14	deepseek-v3	22	5	17.09
15	glm-5	18	0	14.83
16	gemini-3-flash	18	1	14.32
17	o3	18	3	13.73
17	llama-4-maverick	18	3	13.73
17	gpt-oss-120b	18	3	13.73
20	gpt-5.4	16	0	12.90
20	gemini-3.1-pro	16	0	12.90
22	command-a	17	4	12.60
23	codestral	17	5	12.44
24	o4-mini	17	6	12.31
25	gemini-2.5-pro	15	0	11.94
26	deepseek-v3.2	16	5	11.53
27	qwen3-max-thinking	15	1	11.47
27	mimo-v2-flash	15	1	11.47
29	devstral-medium	16	6	11.41
30	devstral-2	14	1	10.53
31	hunter	15	8	10.32
32	grok-code-fast-1	11	1	7.75
33	llama-4-scout	11	4	7.21
34	healer	11	5	7.10
35	gpt-5.1	9	0	6.31
35	gemini-3-pro	9	0	6.31
37	gpt-5.1-codex	7	0	4.52
38	qwen3.5-397b	6	0	3.66

🧠 Reasoning Rankings

Rank	Model	Pass	Fail	Wilson
1	sonnet-4.6	15	0	11.94
2	grok-code-fast-1	15	0	11.94
3	gpt-5-mini	15	0	11.94
4	seed-2.0-mini	15	0	11.94
5	gpt-5.4-pro	15	0	11.94
5	hunter	15	0	11.94
5	healer	15	0	11.94
8	minimax-m2.5	14	0	10.99
9	llama-4-maverick	14	1	10.53
9	glm-4.7	14	1	10.53

✍️ Non-Coding Rankings

Rank	Model	Pass	Fail	Wilson
1	deepseek-v3.2	40	0	36.50
2	gemini-3-flash	40	0	36.50
3	qwen3-coder	36	4	30.78
4	glm-5	32	4	26.89
5	kimi-k2.5	26	6	20.70
6	deepseek-v3.2-exp	20	0	16.78
7	gpt-5-mini	20	0	16.78
8	llama-4-maverick	20	0	16.78
9	hunter	19	0	15.80
10	sonnet-4.6	19	1	15.28
10	minimax-m2.5	19	1	15.28
10	healer	19	1	15.28

📋 Test Tasks by Category

📝 TypeScript

async-retry: Exponential backoff with jitter, configurable retry predicate

typed-emitter: Generic type-safe event emitter

rate-limiter: Token bucket algorithm

🦀 Rust

builder-pattern: Builder creational pattern

channel-mpmc: Multi-producer multi-consumer

functional-pipeline: Iterator combinators

generic-cache: Generic cache implementation

state-machine: State machine pattern

🧠 Reasoning

Logic puzzles, cryptarithmetic, spatial reasoning, tournament brackets

✍️ Non-Coding

Writing tasks, analysis, structured output, JSON extraction

📝 TypeScript Tasks — Detailed

mcp-server

Write a TypeScript MCP (Model Context Protocol) server implementation with tool registration, JSON-RPC request handling, and input validation against JSON Schema.

Features: Tool registration, tools/list, tools/call methods, type validation
Types: ToolDefinition, ToolResult, MCPRequest, MCPResponse

skill-router

Write a TypeScript skill/command router for an AI assistant with prefix, regex, and keyword matching.

Features: Multiple trigger types (prefix, regex, keyword), parameter extraction, priority handling
Matching: prefix > regex > keyword priority

tool-planner

Write a TypeScript tool execution planner that builds dependency graphs and executes tools in parallel.

Features: DAG-based planning, parallel execution, dependency resolution
Validation: Checks all inputs are satisfied from available data or previous steps

🦀 Rust Tasks — Detailed

trait-dispatch

Write a Rust plugin system using trait objects with chain and parallel execution modes.

Features: Plugin trait, PluginRegistry, chain() and parallel() execution
Error types: NotFound, ExecutionError, ChainError with step index

generic-cache

Write a generic LRU cache with TTL expiration using only std collections.

Features: LRU eviction, time-to-live, get/insert/remove/cleanup
Constraints: No external crates, use HashMap + Vec/VecDeque

state-machine

Write a type-safe state machine for order processing using typestate pattern.

States: Draft → Submitted → Processing → Shipped → Delivered / Cancelled
Feature: Compile-time enforcement of valid transitions

iterator-combinators

Write custom iterator combinators (windowed, interleave, unique_by, batched) without itertools.

Combinators: Windowed, Interleave, UniqueBy, Batched
Extension trait: IteratorExt with chainable methods

channel-mpmc

Write a multi-producer multi-consumer bounded channel using std::sync primitives.

Features: Bounded capacity, try_send/try_recv, proper disconnect handling
Primitives: Mutex, Condvar, Arc, VecDeque

builder-pattern

Write a type-safe HTTP request builder using typestate pattern.

States: NoUrl → HasUrl → HasMethod
Feature: Compile-time enforcement of required fields (url, method)

result-combinators

Write Result extension trait with error handling combinators.

Methods: map_err_with_context, or_retry, and_validate, tap, tap_err
Utilities: compose_results, fallback_chain

functional-pipeline

Write a functional data processing pipeline with closures and higher-order functions.

Pipeline: map, flat_map, filter, inspect, unwrap
Utilities: compose, pipe, apply_n, fold_while, scan_collect, group_by_key

tokio-task-pool

Write an async task pool using tokio with work stealing and graceful shutdown.

Features: spawn, spawn_blocking, shutdown, map_concurrent, race, timeout_race
Concurrency: Bounded parallelism, work stealing

🧠 Reasoning Tasks — Detailed

Logic Deduction

Solve logic puzzles involving suspects, truth-telling, and deductions.

Example: "Three suspects: Alex, Blake, Casey. Exactly one is guilty. The guilty person always lies; innocent people always tell the truth..."

Math Word Problems

Solve multi-step word problems involving rates, percentages, and work calculations.

Topics: Train meeting problems, discounts with tax, work collaboration

Sequence Prediction

Find patterns in number sequences and predict next values.

Example: "2, 3, 5, 9, 17, 33, __, __, __" (hint: look at differences)

Spatial Reasoning

Solve puzzles involving spatial arrangements, grids, and visual patterns.

Cryptarithmetic

Solve alphametic puzzles where letters represent digits.

Tournament Brackets

Determine tournament outcomes given constraints.

✍️ Non-Coding Tasks — Detailed

Writing Tasks

Generate coherent, well-structured text based on prompts.

Evaluated on: Grammar, coherence, relevance, style

Analysis Tasks

Analyze data or text and provide insights.

Structured Output

Generate output in specific formats (JSON, CSV, markdown tables).

Verification: Output must match exact schema

JSON Extraction

Extract structured data from unstructured text.

Classification

Categorize items based on provided criteria.

Summarization

Condense longer text into key points.

📖 Abbreviations

Abbreviation	Meaning
`RAW`	Direct model calls without any augmentation
`VITEST`	JavaScript/TypeScript test runner
`TSC`	TypeScript Compiler (type checking)
`Wilson`	Adjusted Wilson Score (confidence-weighted ranking)
`TS`	TypeScript
`Pass Rate`	passes / (passes + failures) — for reference only