Leaderboard
On-device LLM performance rankings powered by Glicko-2
iPhone 13 Pro
iOSRank
#28
Rating
1,895
±16 RD
Win Rate
88.1%
Conservative Rating
1,864
TG Rating
1,801
PP Rating
1,915
Matches
1,096
Record
966W – 130L
Models Tested
| Model | TG Median (tok/s) | PP Median (tok/s) | TG Best | PP Best | Runs |
|---|---|---|---|---|---|
| tinygemma3-Q8_0 | 524.16 | 2225.33 | 542.09 | 13831.62 | 3 |
| SmolLM2-135M-Instruct-Q4_0 | 179.29 | 437.71 | 191.37 | 477.63 | 10 |
| SmolLM2-135M-Instruct-IQ3_M | 123.91 | 239.51 | 123.91 | 239.51 | 1 |
| SmolLM2-135M-Instruct-Q8_0 | 92.07 | 3049.85 | 92.31 | 3055.35 | 2 |
| SmolLM2-360M-Instruct.Q8_0 | 62.04 | 134.92 | 68.65 | 171.10 | 2 |
| google_functiongemma-270m-it-Q8_0 | 57.96 | 4190.18 | 57.96 | 4190.18 | 1 |
| qwen-memento-ckpt-2500-q8_0 | 57.11 | 4187.68 | 57.11 | 4187.68 | 1 |
| gemma-3-270m-it-F16 | 54.53 | 250.62 | 54.53 | 250.62 | 1 |
| SmolLM2-360M-Instruct-Q8_0 | 52.26 | 1317.94 | 52.26 | 1317.94 | 1 |
| qwen-memento-ckpt-7000-q8_0 | 32.44 | 1023.93 | 32.44 | 1023.93 | 1 |
| Qwen_Qwen3-0.6B-Q8_0 | 32.11 | 1056.49 | 32.27 | 1105.21 | 2 |
| DeepSeek-R1-Distill-Qwen-1.5B-Q2_K | 31.24 | 382.84 | 31.24 | 382.84 | 1 |
| LFM2-1.2B-Q4_K_M | 30.05 | 488.74 | 30.05 | 488.74 | 1 |
| LFM2-700M-Q8_0 | 29.95 | 844.16 | 29.95 | 844.16 | 1 |
| google_gemma-3-1b-it-qat-Q4_K_M | 28.94 | 627.58 | 28.94 | 627.58 | 1 |
| Qwen_Qwen3-1.7B-IQ3_XXS | 28.08 | 356.44 | 28.57 | 372.25 | 2 |
| DeepSeek-R1-Distill-Qwen-1.5B-Q4_K_S | 26.86 | 379.56 | 26.86 | 379.56 | 1 |
| Qwen_Qwen3-1.7B-IQ2_M | 26.71 | 377.86 | 26.86 | 377.91 | 2 |
| Qwen_Qwen3-1.7B-IQ3_XS | 26.46 | 364.30 | 26.46 | 364.30 | 1 |
| LFM2-1.2B-Q5_K_M | 26.22 | 439.43 | 26.22 | 439.43 | 1 |
| agentica-org_DeepScaleR-1.5B-Preview-Q4_K_M | 25.18 | 387.56 | 25.18 | 387.56 | 1 |
| gemma-3-1b-it-Q8_0 | 24.28 | 716.16 | 24.41 | 726.14 | 2 |
| Qwen_Qwen3-1.7B-IQ4_XS | 24.15 | 325.27 | 24.15 | 325.27 | 1 |
| qwen-memento-ckpt-3000-q4_k_s | 24.12 | 398.83 | 24.33 | 399.27 | 3 |
| qwen-memento-ckpt-2000-q4_k_s | 23.83 | 382.84 | 23.83 | 382.84 | 1 |
| qwen-memento-ckpt-3000-q4_k_s | 23.26 | 381.04 | 23.26 | 381.04 | 1 |
| Qwen_Qwen3-1.7B-Q3_K_S | 23.24 | 335.56 | 23.24 | 335.56 | 1 |
| Qwen_Qwen3-1.7B-Q4_K_S | 22.92 | 370.94 | 23.36 | 383.34 | 2 |
| Qwen_Qwen3-1.7B-IQ4_NL | 22.67 | 390.09 | 22.67 | 390.09 | 1 |
| Qwen_Qwen3-1.7B-Q4_K_M | 22.34 | 350.50 | 22.34 | 350.50 | 1 |
| Qwen_Qwen3-1.7B-Q4_0 | 22.09 | 382.30 | 22.09 | 382.30 | 1 |
| qwen-memento-ckpt-5000-q4_k_s | 22.00 | 300.19 | 22.00 | 300.19 | 1 |
| llama-3.2-1b-instruct-q8_0 | 20.89 | 543.69 | 23.35 | 596.67 | 17 |
| LFM2-1.2B-Q8_0 | 20.17 | 552.08 | 20.17 | 552.08 | 1 |
| qwen2.5-1.5b-instruct-q8_0 | 18.39 | 36.73 | 18.39 | 36.73 | 1 |
| DeepSeek-R1-Distill-Qwen-1.5B-Q8_0 | 17.51 | 240.95 | 17.84 | 441.30 | 2 |
| DeepSeek-R1-Distill-Qwen-1.5B-Q4_0 | 14.80 | 22.33 | 14.80 | 22.33 | 1 |
| Llama-3.2-3B-Instruct-Q4_K_S | 14.08 | 184.08 | 14.08 | 184.08 | 1 |
| SmolLM2-1.7B-Instruct-Q8_0 | 13.89 | 252.39 | 14.53 | 321.20 | 2 |
| Gemmasutra-Mini-2B-v1-Q6_K | 13.59 | 246.11 | 13.59 | 246.11 | 1 |
| positivedetox-qwen3-4b-iq4_xs | 10.00 | 37.32 | 10.00 | 37.32 | 1 |
| gemma-2-2b-it-Q6_K | 9.32 | 15.02 | 12.25 | 235.35 | 6 |
| qwen2.5-3b-instruct-q5_k_m | 7.97 | 81.49 | 12.83 | 172.58 | 5 |
| gemma-3-4b-it.Q4_K_M | 7.90 | 97.66 | 7.90 | 97.66 | 1 |
| Llama-3.2-3B-Instruct-Q6_K | 6.77 | 9.76 | 9.89 | 162.88 | 3 |
| Hermes-3-Llama-3.2-3B.Q5_K_M | 6.42 | 9.68 | 6.42 | 9.68 | 1 |
| Llama-3.2-3B-Instruct.Q6_K | 6.24 | 9.34 | 6.24 | 9.34 | 1 |
| Hermes-3-Llama-3.2-3B.Q6_K | 5.84 | 8.40 | 5.84 | 8.40 | 1 |
| Phi-3.5-mini-instruct.Q4_K_M | 5.28 | 9.03 | 6.03 | 9.72 | 2 |
| Qwen3-4B-abliterated-iq2_xs | 4.76 | 4.87 | 4.76 | 4.87 | 1 |
1–50 of 52 rows
1 / 2
Head-to-Head Record
1–50 of 337 rows
1 / 7
Performance by App Version
ImprovedRegressed