Leaderboard
On-device LLM performance rankings powered by Glicko-2
iPhone 17 Pro Max
iOSRank
#6
Rating
1,987
±17 RD
Win Rate
97.0%
Conservative Rating
1,954
TG Rating
1,988
PP Rating
1,990
Matches
979
Record
950W – 29L
Models Tested
| Model | TG Median (tok/s) | PP Median (tok/s) | TG Best | PP Best | Runs |
|---|---|---|---|---|---|
| gemma-3-1b-it.Q2_K | 82.36 | 1097.30 | 83.59 | 1139.50 | 2 |
| qwen2.5-1.5b-instruct.Q4_K_M | 61.46 | 648.88 | 61.46 | 648.88 | 1 |
| Qwen3-1.7B-Q4_K_M | 40.82 | 315.69 | 52.00 | 586.66 | 2 |
| qwen2.5-1.5b-instruct-q8_0 | 38.13 | 661.60 | 38.13 | 661.60 | 1 |
| qwen2.5-3b-instruct-q5_k_m | 27.32 | 281.01 | 27.91 | 291.56 | 9 |
| gemma-2-2b-it-Q6_K | 26.63 | 367.54 | 29.57 | 402.86 | 7 |
| Qwen3.5-2B.Q8_0 | 24.45 | 488.58 | 24.45 | 488.58 | 1 |
| Qwen3-VL-4B-Thinking-Q4_K_M | 23.83 | 216.04 | 23.83 | 216.04 | 1 |
| Qwen3-4B-Instruct-2507-Q4_K_M | 23.82 | 223.93 | 23.82 | 223.93 | 1 |
| Llama-3.2-3B-Instruct-Q6_K | 23.43 | 281.22 | 23.82 | 296.95 | 5 |
| qwen2.5-1.5b-instruct-fp16 | 21.54 | 660.04 | 21.54 | 660.04 | 1 |
| Qwen3-4B.Q6_K | 18.85 | 207.65 | 18.85 | 207.65 | 1 |
| Phi-3.5-mini-instruct.Q4_K_M | 17.33 | 162.98 | 24.30 | 221.95 | 2 |
| Gemmasutra-Mini-2B-v1-Q6_K | 16.10 | 198.76 | 17.21 | 373.69 | 2 |
| SmolLM3-Q4_K_M | 15.58 | 22.43 | 15.58 | 22.43 | 1 |
| Qwen3-4B-Instruct-2507-Q5_K_S | 15.57 | 187.96 | 15.57 | 187.96 | 1 |
| Llama-3.2-8B-Instruct-Q3_K_M | 14.96 | 117.36 | 14.96 | 117.36 | 1 |
| Qwen3.5-2B-BF16 | 14.96 | 483.89 | 14.96 | 483.89 | 1 |
| Qwen3-4B-Instruct-2507-UD-Q5_K_XL | 14.68 | 207.90 | 14.68 | 207.90 | 1 |
| Qwen3-4B-Instruct-2507-UD-Q6_K_XL | 14.21 | 216.62 | 14.21 | 216.62 | 1 |
| Qwen3.5-4B-IQ4_NL | 14.05 | 180.92 | 14.21 | 185.05 | 2 |
| Qwen_Qwen3-4B-Thinking-2507-Q8_0 | 14.03 | 215.31 | 14.03 | 215.31 | 1 |
| Qwen_Qwen3-4B-Instruct-2507-Q8_0 | 13.78 | 214.16 | 13.78 | 214.16 | 1 |
| Qwen3.5-4B-Q4_K_M | 13.55 | 174.94 | 13.85 | 178.40 | 2 |
| DeepSeek-R1-Distill-Qwen-7B-Q4_K_M | 12.86 | 120.51 | 12.86 | 120.51 | 1 |
| Ministral-3-8B-Instruct-2512-IQ4_XS | 12.86 | 108.79 | 12.86 | 108.79 | 1 |
| Qwen_Qwen3.5-4B-Q4_K_M | 11.60 | 160.43 | 11.60 | 160.43 | 1 |
| gemma-3-4b-it-Q4_K_M | 10.94 | 19.11 | 10.94 | 19.11 | 1 |
| qwen2.5-7b-instruct-q3_k_m | 10.58 | 95.72 | 10.58 | 95.72 | 1 |
| dolphin3.0-llama3.1-8b-q4_k_m | 10.45 | 71.55 | 10.45 | 71.55 | 1 |
| Llama3.3-8B-Instruct-Thinking-Heretic-Uncensored-Claude-4.5-Opus-High-Reasoning.i1-IQ4_XS | 5.89 | 8.94 | 5.89 | 8.94 | 1 |
| Meta-Llama-3.1-8B-Instruct-Q5_K_M | 4.80 | 51.41 | 4.80 | 51.41 | 1 |
| DeepSeek-R1-0528-Qwen3-8B-IQ4_NL | 3.02 | 6.91 | 3.02 | 6.91 | 1 |
Head-to-Head Record
1–50 of 271 rows
1 / 6
Performance by App Version
ImprovedRegressed