Leaderboard
On-device LLM performance rankings powered by Glicko-2
Pixel 7
AndroidRank
#186
Rating
1,326
±16 RD
Win Rate
33.2%
Conservative Rating
1,293
TG Rating
1,342
PP Rating
1,266
Matches
1,031
Record
342W – 689L
Models Tested
| Model | TG Median (tok/s) | PP Median (tok/s) | TG Best | PP Best | Runs |
|---|---|---|---|---|---|
| gemma-3-270m-it-IQ4_NL | 31.37 | 230.82 | 34.97 | 243.02 | 2 |
| SmolLM2-135M-Instruct-Q4_0 | 31.06 | 295.80 | 32.17 | 309.17 | 2 |
| SmolLM2-135M-Instruct-Q8_0 | 29.12 | 339.55 | 30.80 | 351.87 | 2 |
| LFM2.5-1.2B-Thinking-Q4_K_M | 14.65 | 79.92 | 14.65 | 79.92 | 1 |
| llama-3.2-1b-instruct-q8_0 | 9.34 | 44.45 | 12.15 | 73.29 | 5 |
| DeepSeek-R1-Distill-Qwen-1.5B-Q8_0 | 6.32 | 31.13 | 6.32 | 31.13 | 1 |
| qwen2.5-1.5b-instruct-q8_0 | 5.42 | 27.27 | 5.95 | 27.79 | 2 |
| Phi-3.5-mini-instruct.Q4_K_M | 5.35 | 12.87 | 5.35 | 12.87 | 1 |
| Qwen3.5-2B-Q4_K_M | 4.66 | 31.81 | 4.66 | 31.81 | 1 |
| DeepSeek-R1-Distill-Qwen-1.5B-Q4_K_M | 3.94 | 15.95 | 3.94 | 15.95 | 1 |
| gemma-2-2b-it-Q6_K | 3.74 | 12.19 | 4.10 | 14.44 | 2 |
| gemma-3n-E4B-it-Q2_K | 3.63 | 6.55 | 3.63 | 6.55 | 1 |
| llama-3.2-3b-instruct-q8_0 | 3.28 | 10.60 | 3.28 | 10.60 | 1 |
| qwen2.5-3b-instruct-q5_k_m | 3.27 | 10.14 | 4.33 | 10.76 | 2 |
| Llama-3.2-3B-Instruct-Q6_K | 2.95 | 8.59 | 3.24 | 10.34 | 2 |
| Meta-Llama-3-8B-Instruct.IQ2_XS | 2.39 | 3.33 | 2.39 | 3.33 | 1 |
| Qwen3.5-4B-Q3_K_M | 1.06 | 6.81 | 1.06 | 6.81 | 1 |
Head-to-Head Record
1–50 of 295 rows
1 / 6
Performance by App Version
ImprovedRegressed