Zabio has not run its own tests on this model. Everything below is aggregated from public benchmarks, with the sources named.
Price against score in each kind of job (14) Price per 1,000 tasks, USD. Score, %. Greyed out: a cheaper model already scores as well. Ringed: the best, balanced and cheap choice.
Writing, Medium 50 60 70 80 90 100 $0.1 $1 $10
deepseek-v4-flash-0731@deepinfra: 70.4 +/- 3.8, $0.12
deepseek-v4-flash@deepinfra: 66.3 +/- 3.9, $0.14
deepseek-v4-flash@parasail: 66.3 +/- 3.9, $0.22
deepseek-v4-flash-0731: 70.4 +/- 3.8, $0.22
deepseek-v4-flash-0731@parasail: 70.4 +/- 3.8, $0.22
deepseek-flash@deepinfra: 76.0 +/- 3.6, $0.28
gpt-6-luna: 61.8 +/- 4.0, $0.28
claude-haiku-5-5: 63.0 +/- 4.0, $0.28
glm-5.3-flash: 62.7 +/- 3.9, $0.32
glm-5.3-flash@deepinfra: 62.7 +/- 3.9, $0.32
deepseek-v4-flash: 66.3 +/- 3.9, $0.32
deepseek-v4-flash-vision-exp: 76.6 +/- 3.5, $0.43
gpt-5.6-luna: 60.2 +/- 4.0, $0.64
gpt-5.4-nano: 64.6 +/- 3.9, $0.66
minimax-m3@deepinfra: 66.4 +/- 3.9, $0.66
deepseek-flash: 76.0 +/- 3.6, $0.72
deepseek-flash@alibaba-intl: 76.0 +/- 3.6, $0.72
deepseek-flash@parasail: 76.0 +/- 3.6, $0.72
deepseek-flash@together: 76.0 +/- 3.6, $0.72
minimax-m3: 66.4 +/- 3.9, $0.72
minimax-m3@parasail: 66.4 +/- 3.9, $0.72
qwen3.8-27b@deepinfra: 74.6 +/- 3.6, $0.87
deepseek-v4-flash-0731@alibaba-intl: 70.4 +/- 3.8, $0.88
qwen3.8-27b@parasail: 74.6 +/- 3.6, $1.07
deepseek-flash-fast@parasail: 76.0 +/- 3.6, $1.08
minimax-m3-fast@parasail: 66.4 +/- 3.9, $1.08
glm-5.2@deepinfra: 69.3 +/- 3.9, $1.17
gemini-3.5-flash-lite: 68.4 +/- 3.8, $1.24
glm-5.3@deepinfra: 72.3 +/- 3.8, $1.45
qwen3.6-27b@deepinfra: 56.4 +/- 4.1, $1.54
qwen3.8-27b: 74.6 +/- 3.6, $1.60
qwen3.6-plus: 67.8 +/- 3.9, $1.60
qwen3.6-27b: 56.4 +/- 4.1, $1.92
kimi-k2.6: 68.4 +/- 3.9, $2.00
deepseek-v4-pro-0813@deepinfra: 75.4 +/- 3.6, $2.08
deepseek-v4-pro@deepinfra: 67.2 +/- 3.9, $2.08
gemini-3.8-flash: 83.9 +/- 3.1, $2.10
gemini-3.7-flash: 81.8 +/- 3.3, $2.10
gemini-3.6-flash: 79.1 +/- 3.4, $2.10
gemini-3.5-flash: 80.1 +/- 3.4, $2.10
kimi-k2.7-code: 63.2 +/- 3.9, $2.36
gpt-5.4-mini: 64.9 +/- 3.9, $2.40
inkling: 72.2 +/- 3.7, $2.42
deepseek-v4-pro-0813: 75.4 +/- 3.6, $2.64
deepseek-v4-pro: 67.2 +/- 3.9, $2.64
deepseek-v4-pro@parasail: 67.2 +/- 3.9, $2.64
deepseek-v4-pro@together: 67.2 +/- 3.9, $2.64
glm-5.3: 72.3 +/- 3.8, $2.88
glm-5.3@alibaba-intl: 72.3 +/- 3.8, $2.88
glm-5.3@parasail: 72.3 +/- 3.8, $2.88
glm-5.2: 69.3 +/- 3.9, $2.88
glm-5.2@alibaba-intl: 69.3 +/- 3.9, $2.88
glm-5.2@alibaba-us: 69.3 +/- 3.9, $2.88
deepseek-v4-pro-fast@parasail: 67.2 +/- 3.9, $3.96
qwen3.8-max-0902: 76.5 +/- 3.5, $4.00
glm-5.3-fast@parasail: 72.3 +/- 3.8, $4.32
grok-4.6: 77.7 +/- 3.5, $4.40
grok-4.7: 79.1 +/- 3.4, $4.40
qwen3.7-max: 75.9 +/- 3.6, $5.00
qwen3.7-max@alibaba-us: 75.9 +/- 3.6, $5.00
gpt-6.1-sol: 81.4 +/- 3.4, $5.60
gpt-6-sol: 77.7 +/- 3.6, $5.60
claude-sonnet-5-5: 63.1 +/- 3.9, $5.60
claude-sonnet-5: 68.1 +/- 3.9, $5.60
gemini-3.1-pro-preview: 82.6 +/- 3.2, $6.40
gpt-5.6-terra: 70.0 +/- 3.8, $6.40
gpt-5.2: 72.0 +/- 3.8, $7.00
kimi-k3@deepinfra: 80.0 +/- 3.4, $7.98
gpt-5.4: 73.0 +/- 3.7, $8.00
kimi-k3: 80.0 +/- 3.4, $8.40
kimi-k3@moonshot: 80.0 +/- 3.4, $8.40
claude-sonnet-4-6: 67.8 +/- 3.9, $8.40
gpt-5.6-sol: 79.6 +/- 3.5, $11.20
claude-opus-5-5: 75.3 +/- 3.7, $11.20
claude-opus-5: 77.2 +/- 3.6, $14.00
claude-opus-4-6: 73.1 +/- 3.7, $14.00
claude-opus-4-7: 71.8 +/- 3.8, $14.00
claude-opus-4-8: 73.8 +/- 3.7, $14.00
claude-opus-4-5: 73.5 +/- 3.7, $14.00
gpt-5.5: 79.6 +/- 3.5, $16.00
claude-fable-5: 84.2 +/- 3.2, $28.00
claude-fable-5-1: 80.5 +/- 3.4, $28.00
gpt-6-astra: 83.8 +/- 3.2, $28.00
deepseek-v4-pro@deepinfra gemini-3.8-flash Cheap claude-fable-5 Best, Balanced Creative writing, Medium 0 25 50 75 100 $0.1 $1 $10
deepseek-v4-flash-0731@deepinfra: 70.5 +/- 6.5, $0.12
deepseek-v4-flash@deepinfra: 67.3 +/- 6.6, $0.14
deepseek-v4-flash@parasail: 67.3 +/- 6.6, $0.22
deepseek-v4-flash-0731: 70.5 +/- 6.5, $0.22
deepseek-v4-flash-0731@parasail: 70.5 +/- 6.5, $0.22
deepseek-flash@deepinfra: 72.7 +/- 6.3, $0.28
gpt-6-luna: 63.3 +/- 6.8, $0.28
claude-haiku-5-5: 73.6 +/- 6.2, $0.28
glm-5.3-flash: 59.9 +/- 6.9, $0.32
glm-5.3-flash@deepinfra: 59.9 +/- 6.9, $0.32
deepseek-v4-flash: 67.3 +/- 6.6, $0.32
deepseek-v4-flash-vision-exp: 73.8 +/- 6.2, $0.43
gpt-5.6-luna: 69.1 +/- 6.5, $0.64
gpt-5.4-nano: 67.7 +/- 6.6, $0.66
minimax-m3@deepinfra: 63.6 +/- 6.8, $0.66
deepseek-flash: 72.7 +/- 6.3, $0.72
deepseek-flash@alibaba-intl: 72.7 +/- 6.3, $0.72
deepseek-flash@parasail: 72.7 +/- 6.3, $0.72
deepseek-flash@together: 72.7 +/- 6.3, $0.72
minimax-m3: 63.6 +/- 6.8, $0.72
minimax-m3@parasail: 63.6 +/- 6.8, $0.72
qwen3.8-27b@deepinfra: 73.1 +/- 6.3, $0.87
deepseek-v4-flash-0731@alibaba-intl: 70.5 +/- 6.5, $0.88
qwen3.8-27b@parasail: 73.1 +/- 6.3, $1.07
deepseek-flash-fast@parasail: 72.7 +/- 6.3, $1.08
minimax-m3-fast@parasail: 63.6 +/- 6.8, $1.08
glm-5.2@deepinfra: 62.9 +/- 6.8, $1.17
gemini-3.5-flash-lite: 67.5 +/- 6.6, $1.24
glm-5.3@deepinfra: 72.1 +/- 6.3, $1.45
qwen3.6-27b@deepinfra: 60.1 +/- 6.9, $1.54
qwen3.6-plus: 56.9 +/- 7.0, $1.60
qwen3.8-27b: 73.1 +/- 6.3, $1.60
qwen3.6-27b: 60.1 +/- 6.9, $1.92
kimi-k2.6: 66.9 +/- 6.7, $2.00
deepseek-v4-pro-0813@deepinfra: 74.9 +/- 6.1, $2.08
deepseek-v4-pro@deepinfra: 67.8 +/- 6.6, $2.08
gemini-3.8-flash: 82.8 +/- 5.3, $2.10
gemini-3.7-flash: 82.3 +/- 5.4, $2.10
gemini-3.6-flash: 75.6 +/- 6.1, $2.10
gemini-3.5-flash: 75.2 +/- 6.1, $2.10
kimi-k2.7-code: 68.6 +/- 6.6, $2.36
gpt-5.4-mini: 66.6 +/- 6.7, $2.40
inkling: 74.2 +/- 6.2, $2.42
deepseek-v4-pro-0813: 74.9 +/- 6.1, $2.64
deepseek-v4-pro: 67.8 +/- 6.6, $2.64
deepseek-v4-pro@parasail: 67.8 +/- 6.6, $2.64
deepseek-v4-pro@together: 67.8 +/- 6.6, $2.64
glm-5.3: 72.1 +/- 6.3, $2.88
glm-5.3@alibaba-intl: 72.1 +/- 6.3, $2.88
glm-5.3@parasail: 72.1 +/- 6.3, $2.88
glm-5.2: 62.9 +/- 6.8, $2.88
glm-5.2@alibaba-intl: 62.9 +/- 6.8, $2.88
glm-5.2@alibaba-us: 62.9 +/- 6.8, $2.88
deepseek-v4-pro-fast@parasail: 67.8 +/- 6.6, $3.96
qwen3.8-max-0902: 76.5 +/- 6.0, $4.00
glm-5.3-fast@parasail: 72.1 +/- 6.3, $4.32
grok-4.6: 78.3 +/- 5.8, $4.40
grok-4.7: 74.9 +/- 6.1, $4.40
qwen3.7-max: 76.3 +/- 6.0, $5.00
qwen3.7-max@alibaba-us: 76.3 +/- 6.0, $5.00
claude-sonnet-5-5: 69.9 +/- 6.5, $5.60
gpt-6.1-sol: 70.8 +/- 6.4, $5.60
gpt-6-sol: 66.9 +/- 6.7, $5.60
claude-sonnet-5: 62.3 +/- 6.9, $5.60
gemini-3.1-pro-preview: 77.3 +/- 5.9, $6.40
gpt-5.6-terra: 66.8 +/- 6.7, $6.40
gpt-5.2: 65.7 +/- 6.7, $7.00
kimi-k3@deepinfra: 75.4 +/- 6.1, $7.98
gpt-5.4: 79.0 +/- 5.8, $8.00
kimi-k3: 75.4 +/- 6.1, $8.40
kimi-k3@moonshot: 75.4 +/- 6.1, $8.40
claude-sonnet-4-6: 68.9 +/- 6.6, $8.40
claude-opus-5-5: 70.9 +/- 6.4, $11.20
gpt-5.6-sol: 75.9 +/- 6.1, $11.20
claude-opus-4-8: 80.6 +/- 5.6, $14.00
claude-opus-4-7: 69.4 +/- 6.5, $14.00
claude-opus-4-6: 66.1 +/- 6.7, $14.00
claude-opus-4-5: 65.9 +/- 6.7, $14.00
claude-opus-5: 61.3 +/- 6.9, $14.00
gpt-5.5: 72.6 +/- 6.3, $16.00
claude-fable-5-1: 78.1 +/- 5.9, $28.00
claude-fable-5: 71.7 +/- 6.4, $28.00
gpt-6-astra: 74.0 +/- 6.2, $28.00
deepseek-v4-flash-0731@deepinfra Cheap deepseek-v4-pro@deepinfra gemini-3.8-flash Best, Balanced Summarizing, Medium 0 25 50 75 100 $0.1 $1 $10
deepseek-v4-flash-0731@deepinfra: 71.5 +/- 6.4, $0.18
deepseek-v4-flash@deepinfra: 68.9 +/- 6.6, $0.25
claude-haiku-5-5: 67.0 +/- 6.7, $0.34
gpt-6-luna: 55.3 +/- 7.0, $0.34
deepseek-v4-flash-0731: 71.5 +/- 6.4, $0.39
deepseek-v4-flash-0731@parasail: 71.5 +/- 6.4, $0.39
deepseek-v4-flash@parasail: 68.9 +/- 6.6, $0.39
deepseek-flash@deepinfra: 69.7 +/- 6.5, $0.42
glm-5.3-flash: 56.5 +/- 7.0, $0.46
glm-5.3-flash@deepinfra: 56.5 +/- 7.0, $0.46
deepseek-v4-flash: 68.9 +/- 6.6, $0.56
deepseek-v4-flash-vision-exp: 72.1 +/- 6.3, $0.65
gpt-5.6-luna: 64.9 +/- 6.8, $0.72
gpt-5.4-nano: 71.8 +/- 6.4, $0.73
qwen3.8-27b@deepinfra: 75.9 +/- 6.1, $0.74
minimax-m3@deepinfra: 53.9 +/- 7.1, $0.89
deepseek-flash: 69.7 +/- 6.5, $0.96
deepseek-flash@alibaba-intl: 69.7 +/- 6.5, $0.96
deepseek-flash@parasail: 69.7 +/- 6.5, $0.96
deepseek-flash@together: 69.7 +/- 6.5, $0.96
minimax-m3: 53.9 +/- 7.1, $0.96
minimax-m3@parasail: 53.9 +/- 7.1, $0.96
qwen3.8-27b@parasail: 75.9 +/- 6.1, $1.02
gemini-3.5-flash-lite: 68.5 +/- 6.6, $1.22
deepseek-v4-flash-0731@alibaba-intl: 71.5 +/- 6.4, $1.32
qwen3.6-27b@deepinfra: 52.6 +/- 7.1, $1.41
deepseek-flash-fast@parasail: 69.7 +/- 6.5, $1.44
minimax-m3-fast@parasail: 53.9 +/- 7.1, $1.44
glm-5.2@deepinfra: 68.4 +/- 6.6, $1.71
qwen3.8-27b: 75.9 +/- 6.1, $1.80
qwen3.6-plus: 64.3 +/- 6.8, $1.80
glm-5.3@deepinfra: 74.6 +/- 6.2, $1.85
qwen3.6-27b: 52.6 +/- 7.1, $2.16
kimi-k2.6: 67.5 +/- 6.6, $2.50
gemini-3.8-flash: 87.5 +/- 4.7, $2.55
gemini-3.7-flash: 84.8 +/- 5.1, $2.55
gemini-3.5-flash: 82.6 +/- 5.4, $2.55
gemini-3.6-flash: 75.9 +/- 6.1, $2.55
gpt-5.4-mini: 59.6 +/- 6.9, $2.70
kimi-k2.7-code: 58.3 +/- 7.0, $3.08
inkling: 74.9 +/- 6.1, $3.21
deepseek-v4-pro@deepinfra: 69.5 +/- 6.5, $3.64
deepseek-v4-pro-0813@deepinfra: 65.7 +/- 6.7, $3.64
deepseek-v4-pro: 69.5 +/- 6.5, $3.96
deepseek-v4-pro@parasail: 69.5 +/- 6.5, $3.96
deepseek-v4-pro@together: 69.5 +/- 6.5, $3.96
deepseek-v4-pro-0813: 65.7 +/- 6.7, $3.96
glm-5.3: 74.6 +/- 6.2, $4.24
glm-5.3@alibaba-intl: 74.6 +/- 6.2, $4.24
glm-5.3@parasail: 74.6 +/- 6.2, $4.24
glm-5.2: 68.4 +/- 6.6, $4.24
glm-5.2@alibaba-intl: 68.4 +/- 6.6, $4.24
glm-5.2@alibaba-us: 68.4 +/- 6.6, $4.24
deepseek-v4-pro-fast@parasail: 69.5 +/- 6.5, $5.94
qwen3.8-max-0902: 79.2 +/- 5.8, $6.00
glm-5.3-fast@parasail: 74.6 +/- 6.2, $6.36
grok-4.7: 78.2 +/- 5.8, $6.60
grok-4.6: 68.1 +/- 6.6, $6.60
gpt-6.1-sol: 81.3 +/- 5.5, $6.80
claude-sonnet-5: 71.2 +/- 6.4, $6.80
gpt-6-sol: 68.0 +/- 6.6, $6.80
claude-sonnet-5-5: 58.2 +/- 7.0, $6.80
gpt-5.2: 63.2 +/- 6.8, $7.00
gemini-3.1-pro-preview: 88.2 +/- 4.6, $7.20
gpt-5.6-terra: 73.2 +/- 6.3, $7.20
qwen3.7-max: 83.8 +/- 5.2, $7.50
qwen3.7-max@alibaba-us: 83.8 +/- 5.2, $7.50
gpt-5.4: 68.4 +/- 6.6, $9.00
kimi-k3@deepinfra: 69.4 +/- 6.5, $9.69
kimi-k3: 69.4 +/- 6.5, $10.20
kimi-k3@moonshot: 69.4 +/- 6.5, $10.20
claude-sonnet-4-6: 61.7 +/- 6.9, $10.20
gpt-5.6-sol: 69.4 +/- 6.5, $13.60
claude-opus-5-5: 69.1 +/- 6.5, $13.60
claude-opus-4-8: 77.7 +/- 5.9, $17.00
claude-opus-5: 66.4 +/- 6.7, $17.00
claude-opus-4-7: 66.3 +/- 6.7, $17.00
claude-opus-4-6: 65.2 +/- 6.7, $17.00
claude-opus-4-5: 63.7 +/- 6.8, $17.00
gpt-5.5: 72.3 +/- 6.3, $18.00
claude-fable-5: 81.7 +/- 5.5, $34.00
gpt-6-astra: 79.6 +/- 5.7, $34.00
claude-fable-5-1: 72.2 +/- 6.3, $34.00
deepseek-v4-flash-vision-exp Cheap qwen3.8-27b@deepinfra Balanced gemini-3.8-flash Best deepseek-v4-pro@deepinfra Pulling out data, Medium 50 60 70 80 90 100 $0.1 $1 $10
deepseek-v4.1-flash-1017@deepinfra: 89.8 +/- 1.7, $0.14
gpt-6-luna: 86.7 +/- 1.9, $0.18
claude-haiku-5-5: 75.6 +/- 2.3, $0.18
glm-5.3-flash: 86.3 +/- 2.2, $0.22
glm-5.3-flash@deepinfra: 86.3 +/- 2.2, $0.22
gemma-4-31b-it: 76.7 +/- 2.1, $0.24
minimax-m3.1-flash: 84.7 +/- 2.1, $0.24
minimax-m3.1-flash@parasail: 84.7 +/- 2.1, $0.24
mimo-v2.6-pro: 85.2 +/- 2.1, $0.47
deepseek-v4.1-flash-1017: 89.8 +/- 1.7, $0.48
deepseek-v4.1-flash-1017@parasail: 89.8 +/- 1.7, $0.48
qwen3.8-27b@deepinfra: 86.4 +/- 2.2, $0.50
qwen3.8-27b@parasail: 86.4 +/- 2.2, $0.63
mistral-large-4-0: 88.3 +/- 1.8, $0.98
qwen3.8-27b: 86.4 +/- 2.2, $1.00
gemini-3.8-flash: 75.1 +/- 2.4, $1.35
inkling: 86.4 +/- 1.9, $1.61
gpt-6.1-terra: 91.2 +/- 1.6, $1.80
gemini-4-hydrogen: 90.6 +/- 1.8, $2.70
qwen3.8-max-0902: 88.2 +/- 1.9, $2.80
grok-4.6: 86.7 +/- 2.0, $3.08
grok-4.7: 88.5 +/- 1.8, $3.08
kimi-k3.1-pro@deepinfra: 88.6 +/- 1.9, $3.42
gemini-4-carbon: 90.3 +/- 2.2, $3.60
gemini-4-argon: 89.7 +/- 2.2, $3.60
gpt-6.1-sol: 91.4 +/- 1.6, $3.60
kimi-k3.1-pro: 88.6 +/- 1.9, $3.60
kimi-k3.1-pro@together: 88.6 +/- 1.9, $3.60
claude-sonnet-5-5: 78.8 +/- 2.2, $3.60
gpt-5.6-terra: 89.7 +/- 1.7, $4.00
kimi-k3@deepinfra: 88.4 +/- 1.9, $5.13
kimi-k3: 88.4 +/- 1.9, $5.40
kimi-k3@moonshot: 88.4 +/- 1.9, $5.40
claude-opus-5-5: 89.2 +/- 1.9, $7.20
claude-fable-5-2: 89.8 +/- 1.9, $14.40
claude-fable-5-1: 87.2 +/- 2.3, $18.00
gpt-6-astra: 91.5 +/- 1.6, $18.00
deepseek-v4.1-flash-1017@deepinfra Cheap glm-5.3-flash Balanced gemini-4-hydrogen Best Sorting into categories, Medium 0 25 50 75 100 $0.1 $1 $10
deepseek-v4-flash-0731@deepinfra: 65.5 +/- 3.4, $0.08
deepseek-v4-flash@deepinfra: 63.1 +/- 3.4, $0.11
deepseek-v4-flash-0731: 65.5 +/- 3.4, $0.17
deepseek-v4-flash-0731@parasail: 65.5 +/- 3.4, $0.17
deepseek-v4-flash@parasail: 63.1 +/- 3.4, $0.17
claude-haiku-5-5: 66.5 +/- 3.3, $0.18
gpt-6-luna: 55.9 +/- 3.5, $0.18
deepseek-flash@deepinfra: 70.0 +/- 3.2, $0.20
glm-5.3-flash: 52.8 +/- 3.5, $0.22
glm-5.3-flash@deepinfra: 52.8 +/- 3.5, $0.22
deepseek-v4-flash: 63.1 +/- 3.4, $0.24
deepseek-v4-flash-vision-exp: 71.0 +/- 3.2, $0.30
gpt-5.6-luna: 60.1 +/- 3.5, $0.40
gpt-5.4-nano: 67.2 +/- 3.3, $0.41
minimax-m3@deepinfra: 57.5 +/- 3.5, $0.44
deepseek-flash: 70.0 +/- 3.2, $0.48
deepseek-flash@alibaba-intl: 70.0 +/- 3.2, $0.48
deepseek-flash@parasail: 70.0 +/- 3.2, $0.48
deepseek-flash@together: 70.0 +/- 3.2, $0.48
minimax-m3: 57.5 +/- 3.5, $0.48
minimax-m3@parasail: 57.5 +/- 3.5, $0.48
qwen3.8-27b@deepinfra: 72.7 +/- 3.2, $0.50
deepseek-v4-flash-0731@alibaba-intl: 65.5 +/- 3.4, $0.62
qwen3.8-27b@parasail: 72.7 +/- 3.2, $0.63
deepseek-flash-fast@parasail: 70.0 +/- 3.2, $0.72
minimax-m3-fast@parasail: 57.5 +/- 3.5, $0.72
gemini-3.5-flash-lite: 67.2 +/- 3.3, $0.74
glm-5.2@deepinfra: 62.3 +/- 3.4, $0.81
qwen3.6-27b@deepinfra: 53.2 +/- 3.5, $0.90
glm-5.3@deepinfra: 69.3 +/- 3.3, $0.95
qwen3.8-27b: 72.7 +/- 3.2, $1.00
qwen3.6-plus: 58.3 +/- 3.5, $1.00
qwen3.6-27b: 53.2 +/- 3.5, $1.20
kimi-k2.6: 64.4 +/- 3.4, $1.30
gemini-3.8-flash: 81.4 +/- 2.8, $1.35
gemini-3.7-flash: 79.9 +/- 2.8, $1.35
gemini-3.5-flash: 75.6 +/- 3.0, $1.35
gemini-3.6-flash: 75.4 +/- 3.1, $1.35
gpt-5.4-mini: 59.8 +/- 3.5, $1.50
deepseek-v4-pro-0813@deepinfra: 67.7 +/- 3.3, $1.56
deepseek-v4-pro@deepinfra: 62.4 +/- 3.4, $1.56
kimi-k2.7-code: 56.3 +/- 3.5, $1.56
inkling: 70.1 +/- 3.2, $1.61
deepseek-v4-pro-0813: 67.7 +/- 3.3, $1.85
deepseek-v4-pro: 62.4 +/- 3.4, $1.85
deepseek-v4-pro@parasail: 62.4 +/- 3.4, $1.85
deepseek-v4-pro@together: 62.4 +/- 3.4, $1.85
glm-5.3: 69.3 +/- 3.3, $2.00
glm-5.3@alibaba-intl: 69.3 +/- 3.3, $2.00
glm-5.3@parasail: 69.3 +/- 3.3, $2.00
glm-5.2: 62.3 +/- 3.4, $2.00
glm-5.2@alibaba-intl: 62.3 +/- 3.4, $2.00
glm-5.2@alibaba-us: 62.3 +/- 3.4, $2.00
deepseek-v4-pro-fast@parasail: 62.4 +/- 3.4, $2.77
qwen3.8-max-0902: 74.1 +/- 3.1, $2.80
glm-5.3-fast@parasail: 69.3 +/- 3.3, $3.00
grok-4.7: 75.3 +/- 3.1, $3.08
grok-4.6: 71.9 +/- 3.2, $3.08
qwen3.7-max: 74.0 +/- 3.1, $3.50
qwen3.7-max@alibaba-us: 74.0 +/- 3.1, $3.50
gpt-6.1-sol: 74.2 +/- 3.1, $3.60
gpt-6-sol: 68.6 +/- 3.3, $3.60
claude-sonnet-5: 63.9 +/- 3.4, $3.60
claude-sonnet-5-5: 56.8 +/- 3.5, $3.60
gemini-3.1-pro-preview: 79.1 +/- 2.9, $4.00
gpt-5.6-terra: 64.6 +/- 3.4, $4.00
gpt-5.2: 61.8 +/- 3.4, $4.20
gpt-5.4: 70.2 +/- 3.2, $5.00
kimi-k3@deepinfra: 71.4 +/- 3.2, $5.13
kimi-k3: 71.4 +/- 3.2, $5.40
kimi-k3@moonshot: 71.4 +/- 3.2, $5.40
claude-sonnet-4-6: 63.2 +/- 3.4, $5.40
gpt-5.6-sol: 71.9 +/- 3.2, $7.20
claude-opus-5-5: 65.7 +/- 3.4, $7.20
claude-opus-4-8: 72.0 +/- 3.2, $9.00
claude-opus-4-7: 66.7 +/- 3.3, $9.00
claude-opus-5: 63.8 +/- 3.4, $9.00
claude-opus-4-6: 63.3 +/- 3.4, $9.00
claude-opus-4-5: 62.6 +/- 3.4, $9.00
gpt-5.5: 70.7 +/- 3.2, $10.00
claude-fable-5: 75.8 +/- 3.0, $18.00
gpt-6-astra: 75.6 +/- 3.0, $18.00
claude-fable-5-1: 73.0 +/- 3.1, $18.00
gemini-3.8-flash Best, Balanced, Cheap deepseek-v4-pro@deepinfra Conversation, Medium 0 25 50 75 100 $0.1 $1 $10
deepseek-v4-flash-0731@deepinfra: 33.6 +/- 1.5, $0.17
gpt-5-nano: 11.7 +/- 1.0, $0.24
deepseek-v4.1-flash-1017@deepinfra: 42.5 +/- 1.6, $0.27
qwen3.5-flash-2026-02-23: 20.3 +/- 1.3, $0.32
gpt-4.1-nano: 6.0 +/- 0.8, $0.32
deepseek-v4-flash-0731: 33.6 +/- 1.5, $0.34
deepseek-v4-flash-0731@parasail: 33.6 +/- 1.5, $0.34
gpt-6-luna: 41.4 +/- 1.6, $0.36
claude-haiku-5-5: 23.8 +/- 1.4, $0.36
deepseek-flash@deepinfra: 42.9 +/- 1.6, $0.39
gemma-4-31b-it: 10.4 +/- 1.0, $0.48
gpt-4o-mini: 8.3 +/- 0.9, $0.48
gpt-5.6-luna: 41.0 +/- 1.6, $0.80
gpt-5.4-nano: 11.7 +/- 1.0, $0.82
deepseek-flash: 42.9 +/- 1.6, $0.96
deepseek-flash@alibaba-intl: 42.9 +/- 1.6, $0.96
deepseek-flash@parasail: 42.9 +/- 1.6, $0.96
deepseek-flash@together: 42.9 +/- 1.6, $0.96
deepseek-v4.1-flash-1017: 42.5 +/- 1.6, $0.96
deepseek-v4.1-flash-1017@parasail: 42.5 +/- 1.6, $0.96
qwen3.6-flash: 15.9 +/- 1.2, $1.00
qwen3.6-flash@alibaba-us: 15.9 +/- 1.2, $1.00
inkling-small: 19.1 +/- 1.2, $1.20
gpt-5-mini: 21.6 +/- 1.3, $1.20
deepseek-v4-flash-0731@alibaba-intl: 33.6 +/- 1.5, $1.23
gpt-4.1-mini: 12.7 +/- 1.1, $1.28
qwen3-235b-a22b-thinking-2507: 40.4 +/- 1.6, $1.29
glm-4.7: 32.2 +/- 1.5, $1.34
deepseek-flash-fast@parasail: 42.9 +/- 1.6, $1.44
glm-5.2@deepinfra: 34.2 +/- 1.5, $1.62
glm-5.3@deepinfra: 41.0 +/- 1.6, $1.90
mistral-large-4-0: 20.0 +/- 1.3, $1.96
gemini-3-flash-preview: 66.8 +/- 1.5, $2.00
qwen3.6-plus: 44.1 +/- 1.6, $2.00
kimi-k2.6: 34.9 +/- 1.5, $2.60
gemini-3.8-flash: 69.7 +/- 1.5, $2.70
gemini-3.7-flash: 69.2 +/- 1.5, $2.70
gemini-3.6-flash: 66.2 +/- 1.5, $2.70
gemini-3.5-flash: 66.2 +/- 1.5, $2.70
gpt-5.4-mini: 29.4 +/- 1.4, $3.00
deepseek-v4-pro-0813@deepinfra: 52.9 +/- 1.6, $3.12
deepseek-v4-pro@deepinfra: 47.0 +/- 1.6, $3.12
kimi-k2.7-code: 36.5 +/- 1.5, $3.12
inkling: 40.3 +/- 1.6, $3.22
o4-mini: 19.6 +/- 1.3, $3.52
o3-mini: 15.3 +/- 1.1, $3.52
gpt-6.1-terra: 57.3 +/- 1.6, $3.60
claude-haiku-4-5: 13.2 +/- 1.1, $3.60
deepseek-v4-pro-0813: 52.9 +/- 1.6, $3.70
deepseek-v4-pro: 47.0 +/- 1.6, $3.70
deepseek-v4-pro@parasail: 47.0 +/- 1.6, $3.70
deepseek-v4-pro@together: 47.0 +/- 1.6, $3.70
glm-5.3: 41.0 +/- 1.6, $4.00
glm-5.3@alibaba-intl: 41.0 +/- 1.6, $4.00
glm-5.3@parasail: 41.0 +/- 1.6, $4.00
glm-5.1: 34.0 +/- 1.5, $4.00
glm-5.2: 34.2 +/- 1.5, $4.00
glm-5.2@alibaba-intl: 34.2 +/- 1.5, $4.00
glm-5.2@alibaba-us: 34.2 +/- 1.5, $4.00
qwen3.6-max-preview: 52.0 +/- 1.6, $5.20
deepseek-v4-pro-fast@parasail: 47.0 +/- 1.6, $5.54
qwen3.8-max-0902: 47.3 +/- 1.6, $5.60
glm-5.3-fast@parasail: 41.0 +/- 1.6, $6.00
gpt-5.1: 48.0 +/- 1.6, $6.00
gpt-5: 50.1 +/- 1.6, $6.00
grok-4.6: 49.3 +/- 1.6, $6.16
grok-4.7: 56.0 +/- 1.6, $6.16
o3: 49.4 +/- 1.6, $6.40
gpt-4.1: 31.1 +/- 1.5, $6.40
qwen3.7-max: 55.8 +/- 1.6, $7.00
qwen3.7-max@alibaba-us: 55.8 +/- 1.6, $7.00
gemini-4-carbon: 77.7 +/- 1.3, $7.20
gemini-4-argon: 76.0 +/- 1.4, $7.20
gpt-6.1-sol: 73.9 +/- 1.4, $7.20
gpt-6-sol: 60.7 +/- 1.6, $7.20
claude-sonnet-5-5: 46.5 +/- 1.6, $7.20
claude-sonnet-5: 33.7 +/- 1.5, $7.20
gemini-3.1-pro-preview: 73.5 +/- 1.4, $8.00
gpt-5.6-terra: 43.2 +/- 1.6, $8.00
gpt-4o: 26.0 +/- 1.4, $8.00
gpt-5.2: 32.8 +/- 1.5, $8.40
gpt-5.4: 45.1 +/- 1.6, $10.00
kimi-k3@deepinfra: 50.6 +/- 1.6, $10.26
kimi-k3: 50.6 +/- 1.6, $10.80
kimi-k3@moonshot: 50.6 +/- 1.6, $10.80
claude-sonnet-4-6: 32.8 +/- 1.5, $10.80
claude-sonnet-4-5: 23.7 +/- 1.4, $10.80
claude-opus-5-5: 72.2 +/- 1.4, $14.40
gpt-5.6-sol: 69.7 +/- 1.5, $14.40
claude-opus-4-7: 51.7 +/- 1.6, $18.00
claude-opus-4-8: 53.0 +/- 1.6, $18.00
claude-opus-5: 59.9 +/- 1.6, $18.00
claude-opus-4-6: 47.0 +/- 1.6, $18.00
claude-opus-4-5: 45.7 +/- 1.6, $18.00
gpt-5.5: 63.0 +/- 1.5, $20.00
claude-fable-5-2: 94.0 +/- 0.8, $28.80
claude-fable-5: 70.7 +/- 1.4, $36.00
gpt-6-astra: 75.6 +/- 1.4, $36.00
claude-fable-5-1: 70.8 +/- 1.4, $36.00
o1: 41.1 +/- 1.6, $48.00
deepseek-v4-pro@deepinfra gemini-4-carbon Best, Balanced, Cheap Analysis and reasoning, Hard 0 25 50 75 100 $0.1 $1 $10 $100
gpt-oss-20b: 53.2 +/- 2.5, $0.19
gemma-3-4b-it: 23.2 +/- 1.9, $0.19
gpt-oss-120b@deepinfra: 75.8 +/- 2.7, $0.23
gemma-3-12b-it: 39.5 +/- 2.5, $0.24
phi-4: 56.1 +/- 2.6, $0.27
deepseek-v4-flash-0731@deepinfra: 91.0 +/- 1.8, $0.28
gemma-3-27b-it: 47.7 +/- 2.7, $0.30
qwen3.5-9b@deepinfra: 79.0 +/- 2.4, $0.34
qwen3-32b: 65.7 +/- 2.8, $0.41
qwen3.5-9b@parasail: 79.0 +/- 2.4, $0.43
gemma-4-26b-a4b-it: 73.2 +/- 3.2, $0.45
qwen3-14b: 63.8 +/- 2.9, $0.46
gpt-5-nano: 57.6 +/- 3.5, $0.46
deepseek-v4.1-flash-1017@deepinfra: 87.0 +/- 3.1, $0.46
llama-4-scout-17b-16e-instruct: 51.8 +/- 3.2, $0.47
llama-3.3-70b-instruct-turbo@deepinfra: 47.4 +/- 2.9, $0.49
deepseek-v4-flash-0731: 91.0 +/- 1.8, $0.53
deepseek-v4-flash-0731@parasail: 91.0 +/- 1.8, $0.53
qwen3.5-flash-2026-02-23: 82.3 +/- 2.7, $0.56
gpt-4.1-nano: 48.9 +/- 2.5, $0.56
qwen3.5-9b: 79.0 +/- 2.4, $0.57
gpt-6-luna: 73.6 +/- 3.7, $0.65
claude-haiku-5-5: 85.5 +/- 2.8, $0.65
deepseek-flash@deepinfra: 89.8 +/- 1.7, $0.66
qwen3-30b-a3b: 61.7 +/- 3.0, $0.69
glm-5.3-flash: 80.6 +/- 3.3, $0.75
glm-5.3-flash@deepinfra: 80.6 +/- 3.3, $0.75
gemma-4-31b-it: 61.7 +/- 3.6, $0.76
minimax-m3.1-flash: 79.8 +/- 4.7, $0.84
minimax-m3.1-flash@parasail: 79.8 +/- 4.7, $0.84
gpt-oss-120b: 75.8 +/- 2.7, $0.84
gpt-4o-mini: 37.7 +/- 2.4, $0.84
llama-3.3-70b-instruct-turbo@parasail: 47.4 +/- 2.9, $0.89
qwen3.6-35b-a3b@parasail: 83.8 +/- 2.6, $1.01
qwen3.6-35b-a3b@deepinfra: 83.8 +/- 2.6, $1.06
qwen3-30b-a3b-instruct-2507: 55.6 +/- 2.9, $1.12
qwen3.5-35b-a3b@deepinfra: 83.5 +/- 2.2, $1.18
qwen3.5-35b-a3b@parasail: 83.5 +/- 2.2, $1.20
mimo-v2.6-pro: 82.2 +/- 4.1, $1.48
gpt-5.6-luna: 63.6 +/- 3.4, $1.48
gpt-5.4-nano: 55.6 +/- 3.5, $1.53
minimax-m3@deepinfra: 90.9 +/- 2.1, $1.55
deepseek-v4.1-flash-1017: 87.0 +/- 3.1, $1.68
deepseek-v4.1-flash-1017@parasail: 87.0 +/- 3.1, $1.68
deepseek-flash: 89.8 +/- 1.7, $1.68
deepseek-flash@alibaba-intl: 89.8 +/- 1.7, $1.68
deepseek-flash@parasail: 89.8 +/- 1.7, $1.68
deepseek-flash@together: 89.8 +/- 1.7, $1.68
minimax-m3: 90.9 +/- 2.1, $1.68
minimax-m3@parasail: 90.9 +/- 2.1, $1.68
qwen3.7-plus: 87.9 +/- 2.3, $1.79
qwen3.7-plus@alibaba-us: 87.9 +/- 2.3, $1.79
qwen3.6-flash: 83.3 +/- 2.7, $1.85
qwen3.6-flash@alibaba-us: 83.3 +/- 2.7, $1.85
gemini-3.1-flash-lite: 74.2 +/- 3.1, $1.85
qwen-plus-2025-01-25: 48.1 +/- 2.7, $1.88
inkling-small: 88.5 +/- 1.9, $1.98
qwen3.8-27b@deepinfra: 70.7 +/- 5.4, $1.99
deepseek-v4-flash-0731@alibaba-intl: 91.0 +/- 1.8, $2.07
gpt-4.1-mini: 65.9 +/- 2.7, $2.24
qwen3.5-35b-a3b: 83.5 +/- 2.2, $2.30
gpt-5-mini: 71.7 +/- 3.2, $2.30
gpt-3.5-turbo: 27.2 +/- 2.0, $2.35
glm-4.7: 83.3 +/- 2.4, $2.38
qwen3.8-27b@parasail: 70.7 +/- 5.4, $2.46
deepseek-flash-fast@parasail: 89.8 +/- 1.7, $2.52
minimax-m3-fast@parasail: 90.9 +/- 2.1, $2.52
qwen3-235b-a22b-thinking-2507: 80.1 +/- 2.6, $2.53
qwen3-30b-a3b-thinking-2507: 70.1 +/- 2.8, $2.56
glm-5.2@deepinfra: 71.2 +/- 3.2, $2.75
qwen3.6-35b-a3b: 83.8 +/- 2.6, $2.78
gemini-3.5-flash-lite: 75.8 +/- 3.1, $2.85
deepseek-r1-0528: 76.3 +/- 2.4, $2.94
llama-3.3-70b-instruct-turbo: 47.4 +/- 2.9, $3.02
mistral-large-4-0: 84.5 +/- 3.0, $3.29
glm-5.3@deepinfra: 90.9 +/- 1.6, $3.38
qwen3.6-27b@deepinfra: 85.9 +/- 2.5, $3.52
qwen3.5-397b-a17b@deepinfra: 85.9 +/- 2.5, $3.60
qwen3.6-plus: 88.4 +/- 2.3, $3.70
gemini-3-flash-preview: 83.2 +/- 1.9, $3.70
qwen3.8-27b: 70.7 +/- 5.4, $3.70
qwen3.5-397b-a17b@parasail: 85.9 +/- 2.5, $4.24
qwen3.6-27b: 85.9 +/- 2.5, $4.44
qwen3.5-397b-a17b: 85.9 +/- 2.5, $4.44
kimi-k2.6: 90.8 +/- 1.7, $4.65
gemini-3.8-flash: 88.0 +/- 3.0, $4.88
gemini-3.7-flash: 94.8 +/- 1.4, $4.88
gemini-3.5-flash: 88.9 +/- 2.2, $4.88
gemini-3.6-flash: 85.9 +/- 2.5, $4.88
deepseek-v4-pro-0813@deepinfra: 91.7 +/- 1.5, $4.94
deepseek-v4-pro@deepinfra: 73.2 +/- 3.2, $4.94
kimi-k2.7-code: 87.9 +/- 2.3, $5.50
gpt-5.4-mini: 64.1 +/- 3.4, $5.55
inkling: 75.6 +/- 3.7, $5.65
o4-mini: 75.3 +/- 3.1, $6.16
o3-mini: 68.2 +/- 3.3, $6.16
deepseek-v4-pro-0813: 91.7 +/- 1.5, $6.20
deepseek-v4-pro: 73.2 +/- 3.2, $6.20
deepseek-v4-pro@parasail: 73.2 +/- 3.2, $6.20
deepseek-v4-pro@together: 73.2 +/- 3.2, $6.20
gpt-6.1-terra: 87.4 +/- 3.0, $6.50
claude-haiku-4-5: 60.5 +/- 2.8, $6.50
glm-5.3: 90.9 +/- 1.6, $6.76
glm-5.3@alibaba-intl: 90.9 +/- 1.6, $6.76
glm-5.3@parasail: 90.9 +/- 1.6, $6.76
glm-5.1: 89.9 +/- 2.2, $6.76
glm-5.2: 71.2 +/- 3.2, $6.76
glm-5.2@alibaba-intl: 71.2 +/- 3.2, $6.76
glm-5.2@alibaba-us: 71.2 +/- 3.2, $6.76
deepseek-v4-pro-fast@parasail: 73.2 +/- 3.2, $9.31
qwen3.8-max-0902: 84.4 +/- 3.3, $9.40
qwen3.6-max-preview: 87.4 +/- 2.4, $9.62
gemini-4-hydrogen: 87.7 +/- 3.8, $9.75
glm-5.3-fast@parasail: 90.9 +/- 1.6, $10.14
grok-4.6: 91.1 +/- 2.4, $10.34
grok-4.7: 88.3 +/- 2.4, $10.34
o3: 79.8 +/- 2.9, $11.20
gpt-4.1: 66.9 +/- 2.8, $11.20
gpt-5: 71.7 +/- 3.2, $11.50
gpt-5.1: 66.7 +/- 3.4, $11.50
qwen3.7-max: 90.9 +/- 2.1, $11.75
qwen3.7-max@alibaba-us: 90.9 +/- 2.1, $11.75
kimi-k3.1-pro@deepinfra: 86.6 +/- 4.2, $12.35
gemini-4-argon: 92.8 +/- 2.5, $13.00
gemini-4-carbon: 93.2 +/- 2.0, $13.00
gpt-6.1-sol: 92.7 +/- 2.3, $13.00
claude-sonnet-5-5: 91.7 +/- 2.4, $13.00
gpt-6-sol: 94.3 +/- 1.5, $13.00
kimi-k3.1-pro: 86.6 +/- 4.2, $13.00
kimi-k3.1-pro@together: 86.6 +/- 4.2, $13.00
claude-sonnet-5: 80.3 +/- 2.8, $13.00
gpt-4o: 47.9 +/- 2.6, $14.00
gemini-3.1-pro-preview: 94.1 +/- 1.7, $14.80
gpt-5.6-terra: 81.9 +/- 2.9, $14.80
gpt-5.2: 73.2 +/- 3.2, $16.10
gpt-5.4: 74.8 +/- 3.1, $18.50
kimi-k3@deepinfra: 85.1 +/- 2.9, $18.53
kimi-k3: 85.1 +/- 2.9, $19.50
kimi-k3@moonshot: 85.1 +/- 2.9, $19.50
claude-sonnet-4-6: 78.8 +/- 2.9, $19.50
claude-sonnet-4-5: 73.7 +/- 3.1, $19.50
claude-opus-5-5: 90.6 +/- 2.4, $26.00
gpt-5.6-sol: 82.8 +/- 2.7, $26.00
claude-opus-5: 92.9 +/- 1.8, $32.50
claude-opus-4-6: 88.4 +/- 2.3, $32.50
claude-opus-4-7: 86.4 +/- 2.5, $32.50
claude-opus-4-8: 85.4 +/- 2.5, $32.50
claude-opus-4-5: 80.7 +/- 2.4, $32.50
gpt-5.5: 77.3 +/- 3.0, $37.00
gpt-4-turbo: 46.6 +/- 2.7, $47.00
claude-fable-5-2: 92.4 +/- 1.9, $52.00
gpt-6-astra: 93.0 +/- 2.2, $65.00
claude-fable-5-1: 90.1 +/- 3.5, $65.00
claude-fable-5: 78.8 +/- 2.9, $65.00
o1: 50.3 +/- 2.8, $84.00
gpt-4: 35.7 +/- 2.4, $114
gemini-3.8-flash Cheap deepseek-v4-pro@deepinfra gemini-4-argon Best, Balanced Analysis and reasoning, Hardest 0 25 50 75 100 $1 $10 $100
deepseek-v4-flash-0731@deepinfra: 71.5 +/- 2.4, $0.45
deepseek-v4.1-flash-1017@deepinfra: 81.6 +/- 2.2, $0.74
gpt-5-nano: 9.6 +/- 1.2, $0.75
llama-4-scout-17b-16e-instruct: 0.3 +/- 0.3, $0.75
deepseek-v4-flash-0731: 71.5 +/- 2.4, $0.84
deepseek-v4-flash-0731@parasail: 71.5 +/- 2.4, $0.84
gpt-4.1-nano: 0.0 +/- 0.2, $0.90
deepseek-flash@deepinfra: 78.0 +/- 2.3, $1.05
gpt-6-luna: 50.9 +/- 2.4, $1.05
claude-haiku-5-5: 45.3 +/- 1.6, $1.05
qwen3-235b-a22b-instruct-2507@deepinfra: 6.1 +/- 0.9, $1.10
glm-5.3-flash: 61.0 +/- 2.5, $1.20
glm-5.3-flash@deepinfra: 61.0 +/- 2.5, $1.20
deepseek-v3.2: 30.5 +/- 1.5, $1.35
gpt-4o-mini: 0.0 +/- 0.4, $1.35
qwen3-235b-a22b-instruct-2507@parasail: 6.1 +/- 0.9, $1.62
qwen3-235b-a22b-instruct-2507: 6.1 +/- 0.9, $2.07
inkling-2-small@deepinfra: 89.1 +/- 1.8, $2.28
inkling-2-small: 89.1 +/- 1.8, $2.40
gpt-5.6-luna: 52.9 +/- 2.3, $2.40
gpt-5.4-nano: 20.9 +/- 1.5, $2.48
deepseek-v4.1-flash-1017: 81.6 +/- 2.2, $2.70
deepseek-v4.1-flash-1017@parasail: 81.6 +/- 2.2, $2.70
deepseek-flash: 78.0 +/- 2.3, $2.70
deepseek-flash@alibaba-intl: 78.0 +/- 2.3, $2.70
deepseek-flash@parasail: 78.0 +/- 2.3, $2.70
deepseek-flash@together: 78.0 +/- 2.3, $2.70
gemini-3.1-flash-lite: 8.6 +/- 1.1, $3.00
inkling-small: 55.5 +/- 2.4, $3.15
qwen3.8-27b@deepinfra: 17.8 +/- 1.3, $3.26
deepseek-v4-flash-0731@alibaba-intl: 71.5 +/- 2.4, $3.30
gpt-4.1-mini: 1.8 +/- 0.5, $3.60
gpt-5-mini: 25.2 +/- 1.1, $3.75
qwen3.8-27b@parasail: 17.8 +/- 1.3, $4.02
deepseek-flash-fast@parasail: 78.0 +/- 2.3, $4.05
glm-5.2@deepinfra: 49.9 +/- 2.2, $4.39
gemini-3.5-flash-lite: 31.9 +/- 1.9, $4.65
deepseek-r1-0528: 11.2 +/- 1.1, $4.73
gemini-3-flash-preview: 11.4 +/- 1.1, $6.00
qwen3.8-27b: 17.8 +/- 1.3, $6.00
deepseek-v4-pro-0813@deepinfra: 73.4 +/- 2.4, $7.80
gemini-3.8-flash: 73.3 +/- 1.2, $7.88
gemini-3.7-flash: 90.0 +/- 1.7, $7.88
gemini-3.5-flash: 82.3 +/- 2.2, $7.88
gemini-3.6-flash: 75.8 +/- 2.3, $7.88
gpt-5.4-mini: 35.6 +/- 2.0, $9.00
inkling: 58.0 +/- 2.4, $9.08
deepseek-v4-pro-0813: 73.4 +/- 2.4, $9.90
o4-mini: 27.6 +/- 1.2, $9.90
o3-mini: 18.8 +/- 1.4, $9.90
claude-haiku-4-5: 7.8 +/- 1.0, $10.50
glm-5.2: 49.9 +/- 2.2, $10.80
glm-5.2@alibaba-intl: 49.9 +/- 2.2, $10.80
glm-5.2@alibaba-us: 49.9 +/- 2.2, $10.80
grok-4.6: 76.1 +/- 2.3, $16.50
grok-4.7: 74.3 +/- 2.4, $16.50
o3: 29.2 +/- 1.2, $18.00
gpt-4.1: 3.8 +/- 0.5, $18.00
gpt-5: 32.6 +/- 1.3, $18.75
gpt-5.1: 36.3 +/- 1.4, $18.75
gemini-4-argon: 94.9 +/- 1.3, $21.00
gemini-4-carbon: 94.8 +/- 1.3, $21.00
gpt-6.1-sol: 95.1 +/- 1.3, $21.00
claude-sonnet-5-5: 68.5 +/- 1.6, $21.00
gpt-6-sol: 80.0 +/- 2.2, $21.00
gpt-4o: 2.4 +/- 0.4, $22.50
gemini-3.1-pro-preview: 73.8 +/- 1.5, $24.00
gpt-5.6-terra: 79.5 +/- 2.3, $24.00
gpt-5.2: 47.2 +/- 1.6, $26.25
kimi-k3@deepinfra: 70.8 +/- 2.4, $29.93
gpt-5.4: 65.5 +/- 1.6, $30.00
kimi-k3: 70.8 +/- 2.4, $31.50
kimi-k3@moonshot: 70.8 +/- 2.4, $31.50
claude-sonnet-4-6: 73.5 +/- 2.4, $31.50
claude-sonnet-4-5: 12.3 +/- 1.0, $31.50
claude-opus-5-5: 95.9 +/- 1.2, $42.00
gpt-5.6-sol: 91.2 +/- 1.7, $42.00
claude-opus-5: 92.9 +/- 1.5, $52.50
claude-opus-4-7: 62.3 +/- 1.5, $52.50
claude-opus-4-8: 82.0 +/- 2.2, $52.50
claude-opus-4-6: 51.8 +/- 1.5, $52.50
claude-opus-4-5: 20.7 +/- 1.2, $52.50
gpt-5.5: 88.9 +/- 1.8, $60.00
claude-fable-5-2: 94.0 +/- 1.4, $84.00
claude-fable-5: 91.5 +/- 1.6, $105
claude-fable-5-1: 77.1 +/- 1.2, $105
gpt-6-astra: 78.4 +/- 1.1, $105
o1: 13.0 +/- 1.1, $135
gemini-4-argon Best, Balanced Math, Hard 0 25 50 75 100 $1 $10 $100
phi-4: 64.9 +/- 1.1, $0.27
deepseek-v4-flash-0731@deepinfra: 86.8 +/- 1.7, $0.28
gemma-3-27b-it: 74.0 +/- 1.1, $0.30
deepseek-v4-flash@deepinfra: 79.7 +/- 2.1, $0.34
gpt-5-nano: 94.9 +/- 0.5, $0.46
llama-4-scout-17b-16e-instruct: 62.3 +/- 1.2, $0.47
llama-3.3-70b-instruct-turbo@deepinfra: 41.6 +/- 1.2, $0.49
deepseek-v4-flash-0731: 86.8 +/- 1.7, $0.53
deepseek-v4-flash-0731@parasail: 86.8 +/- 1.7, $0.53
deepseek-v4-flash@parasail: 79.7 +/- 2.1, $0.53
gpt-4.1-nano: 70.0 +/- 1.0, $0.56
claude-haiku-5-5: 92.8 +/- 1.3, $0.65
gpt-6-luna: 89.1 +/- 1.6, $0.65
deepseek-flash@deepinfra: 93.3 +/- 1.3, $0.66
glm-5.3-flash: 81.2 +/- 2.0, $0.75
glm-5.3-flash@deepinfra: 81.2 +/- 2.0, $0.75
deepseek-v4-flash: 79.7 +/- 2.1, $0.76
gpt-4o-mini: 52.6 +/- 1.1, $0.84
llama-3.3-70b-instruct-turbo@parasail: 41.6 +/- 1.2, $0.89
deepseek-v4-flash-vision-exp: 87.8 +/- 1.7, $1.01
gpt-5.6-luna: 87.2 +/- 1.7, $1.48
gpt-5.4-nano: 91.0 +/- 1.5, $1.53
minimax-m3@deepinfra: 77.0 +/- 2.2, $1.55
deepseek-flash: 93.3 +/- 1.3, $1.68
deepseek-flash@alibaba-intl: 93.3 +/- 1.3, $1.68
deepseek-flash@parasail: 93.3 +/- 1.3, $1.68
deepseek-flash@together: 93.3 +/- 1.3, $1.68
minimax-m3: 77.0 +/- 2.2, $1.68
minimax-m3@parasail: 77.0 +/- 2.2, $1.68
qwen-plus-2025-01-25: 65.3 +/- 1.1, $1.88
qwen3.8-27b@deepinfra: 86.2 +/- 1.8, $1.99
deepseek-v4-flash-0731@alibaba-intl: 86.8 +/- 1.7, $2.07
gpt-4.1-mini: 87.3 +/- 0.7, $2.24
gpt-5-mini: 97.9 +/- 0.3, $2.30
gpt-3.5-turbo: 11.6 +/- 0.6, $2.35
qwen3.8-27b@parasail: 86.2 +/- 1.8, $2.46
deepseek-flash-fast@parasail: 93.3 +/- 1.3, $2.52
minimax-m3-fast@parasail: 77.0 +/- 2.2, $2.52
glm-5.2@deepinfra: 89.8 +/- 1.6, $2.75
gemini-3.5-flash-lite: 73.7 +/- 2.3, $2.85
deepseek-r1-0528: 96.6 +/- 0.4, $2.94
llama-3.3-70b-instruct-turbo: 41.6 +/- 1.2, $3.02
glm-5.3@deepinfra: 87.9 +/- 1.7, $3.38
qwen3.6-27b@deepinfra: 79.9 +/- 2.1, $3.52
qwen3.8-27b: 86.2 +/- 1.8, $3.70
qwen3.6-plus: 83.7 +/- 1.9, $3.70
qwen3.6-27b: 79.9 +/- 2.1, $4.44
kimi-k2.6: 84.3 +/- 1.9, $4.65
gemini-3.7-flash: 93.5 +/- 1.3, $4.88
gemini-3.8-flash: 91.6 +/- 1.4, $4.88
gemini-3.5-flash: 88.2 +/- 1.7, $4.88
gemini-3.6-flash: 86.4 +/- 1.8, $4.88
deepseek-v4-pro-0813@deepinfra: 95.1 +/- 1.1, $4.94
deepseek-v4-pro@deepinfra: 90.7 +/- 1.5, $4.94
kimi-k2.7-code: 79.6 +/- 2.1, $5.50
gpt-5.4-mini: 78.5 +/- 2.1, $5.55
inkling: 88.4 +/- 1.7, $5.65
o4-mini: 97.8 +/- 0.3, $6.16
o3-mini: 96.5 +/- 0.4, $6.16
deepseek-v4-pro-0813: 95.1 +/- 1.1, $6.20
deepseek-v4-pro: 90.7 +/- 1.5, $6.20
deepseek-v4-pro@parasail: 90.7 +/- 1.5, $6.20
deepseek-v4-pro@together: 90.7 +/- 1.5, $6.20
claude-haiku-4-5: 86.9 +/- 0.8, $6.50
glm-5.2: 89.8 +/- 1.6, $6.76
glm-5.2@alibaba-intl: 89.8 +/- 1.6, $6.76
glm-5.2@alibaba-us: 89.8 +/- 1.6, $6.76
glm-5.3: 87.9 +/- 1.7, $6.76
glm-5.3@alibaba-intl: 87.9 +/- 1.7, $6.76
glm-5.3@parasail: 87.9 +/- 1.7, $6.76
deepseek-v4-pro-fast@parasail: 90.7 +/- 1.5, $9.31
qwen3.8-max-0902: 91.3 +/- 1.5, $9.40
glm-5.3-fast@parasail: 87.9 +/- 1.7, $10.14
grok-4.7: 95.7 +/- 1.0, $10.34
grok-4.6: 92.6 +/- 1.4, $10.34
o3: 97.8 +/- 0.3, $11.20
gpt-4.1: 83.0 +/- 0.9, $11.20
gpt-5: 98.1 +/- 0.3, $11.50
qwen3.7-max: 85.3 +/- 1.8, $11.75
qwen3.7-max@alibaba-us: 85.3 +/- 1.8, $11.75
gpt-6.1-sol: 96.8 +/- 0.9, $13.00
gpt-6-sol: 96.4 +/- 1.0, $13.00
claude-sonnet-5-5: 96.1 +/- 1.0, $13.00
claude-sonnet-5: 92.9 +/- 1.3, $13.00
gpt-4o: 49.8 +/- 1.1, $14.00
gpt-5.6-terra: 94.9 +/- 1.1, $14.80
gemini-3.1-pro-preview: 91.0 +/- 1.5, $14.80
gpt-5.2: 93.2 +/- 1.3, $16.10
gpt-5.4: 94.2 +/- 1.2, $18.50
kimi-k3@deepinfra: 84.4 +/- 1.9, $18.53
claude-sonnet-4-5: 97.7 +/- 0.4, $19.50
claude-sonnet-4-6: 87.0 +/- 1.7, $19.50
kimi-k3: 84.4 +/- 1.9, $19.50
kimi-k3@moonshot: 84.4 +/- 1.9, $19.50
claude-opus-5-5: 97.1 +/- 0.9, $26.00
gpt-5.6-sol: 96.2 +/- 1.0, $26.00
claude-opus-5: 95.7 +/- 1.0, $32.50
claude-opus-4-8: 94.3 +/- 1.2, $32.50
claude-opus-4-7: 92.9 +/- 1.3, $32.50
claude-opus-4-5: 90.4 +/- 1.5, $32.50
claude-opus-4-6: 89.3 +/- 1.6, $32.50
gpt-5.5: 95.9 +/- 1.0, $37.00
gpt-4-turbo: 46.7 +/- 1.1, $47.00
claude-fable-5-1: 97.0 +/- 0.9, $65.00
gpt-6-astra: 96.8 +/- 0.9, $65.00
claude-fable-5: 96.0 +/- 1.0, $65.00
o1: 81.7 +/- 1.1, $84.00
gpt-4: 23.0 +/- 0.9, $114
gpt-5-mini Best, Balanced, Cheap deepseek-v4-pro@deepinfra Math, Hardest 0 25 50 75 100 $1 $10 $100
gpt-oss-20b: 40.3 +/- 6.3, $0.30
gemma-3-4b-it: 7.5 +/- 2.8, $0.30
gpt-oss-120b@deepinfra: 88.9 +/- 4.5, $0.37
gemma-3-12b-it: 16.7 +/- 4.2, $0.38
phi-4: 13.8 +/- 3.7, $0.42
deepseek-v4-flash-0731@deepinfra: 76.0 +/- 1.8, $0.45
gemma-3-27b-it: 22.5 +/- 5.3, $0.48
qwen3.5-9b@deepinfra: 42.8 +/- 6.5, $0.53
qwen3-32b: 66.9 +/- 6.0, $0.66
qwen3.5-9b@parasail: 42.8 +/- 6.5, $0.68
gemma-4-26b-a4b-it: 82.2 +/- 5.8, $0.72
qwen3-14b: 66.4 +/- 6.2, $0.72
deepseek-v4.1-flash-1017@deepinfra: 85.3 +/- 1.4, $0.74
gpt-5-nano: 21.1 +/- 2.8, $0.75
llama-4-scout-17b-16e-instruct: 3.9 +/- 2.9, $0.75
llama-3.3-70b-instruct-turbo@deepinfra: 5.1 +/- 2.4, $0.78
deepseek-v4-flash-0731: 76.0 +/- 1.8, $0.84
deepseek-v4-flash-0731@parasail: 76.0 +/- 1.8, $0.84
qwen3.5-9b: 42.8 +/- 6.5, $0.89
qwen3.5-flash-2026-02-23: 33.4 +/- 2.0, $0.90
gpt-4.1-nano: 15.0 +/- 3.0, $0.90
gpt-6-luna: 88.9 +/- 1.3, $1.05
claude-haiku-5-5: 86.2 +/- 1.4, $1.05
deepseek-flash@deepinfra: 82.9 +/- 1.4, $1.05
qwen3-30b-a3b: 62.8 +/- 6.2, $1.11
glm-5.3-flash: 74.8 +/- 1.8, $1.20
glm-5.3-flash@deepinfra: 74.8 +/- 1.8, $1.20
gemma-4-31b-it: 73.3 +/- 6.7, $1.20
gpt-oss-120b: 88.9 +/- 4.5, $1.35
gpt-4o-mini: 3.8 +/- 1.4, $1.35
llama-3.3-70b-instruct-turbo@parasail: 5.1 +/- 2.4, $1.41
qwen3.6-35b-a3b@parasail: 53.5 +/- 2.8, $1.65
qwen3.6-35b-a3b@deepinfra: 53.5 +/- 2.8, $1.73
qwen3-30b-a3b-instruct-2507: 62.2 +/- 6.5, $1.80
qwen3.5-35b-a3b@deepinfra: 54.4 +/- 6.2, $1.92
qwen3.5-35b-a3b@parasail: 54.4 +/- 6.2, $1.95
gpt-5.6-luna: 57.3 +/- 4.4, $2.40
gpt-5.4-nano: 25.7 +/- 2.9, $2.48
minimax-m3@deepinfra: 71.1 +/- 6.8, $2.49
deepseek-v4.1-flash-1017: 85.3 +/- 1.4, $2.70
deepseek-v4.1-flash-1017@parasail: 85.3 +/- 1.4, $2.70
deepseek-flash: 82.9 +/- 1.4, $2.70
deepseek-flash@alibaba-intl: 82.9 +/- 1.4, $2.70
deepseek-flash@parasail: 82.9 +/- 1.4, $2.70
deepseek-flash@together: 82.9 +/- 1.4, $2.70
minimax-m3: 71.1 +/- 6.8, $2.70
minimax-m3@parasail: 71.1 +/- 6.8, $2.70
qwen3.7-plus: 63.9 +/- 2.4, $2.88
qwen3.7-plus@alibaba-us: 63.9 +/- 2.4, $2.88
qwen3.6-flash: 37.3 +/- 2.1, $3.00
qwen3.6-flash@alibaba-us: 37.3 +/- 2.1, $3.00
gemini-3.1-flash-lite: 33.5 +/- 3.9, $3.00
qwen-plus-2025-01-25: 17.8 +/- 4.4, $3.00
qwen-plus: 1.7 +/- 0.8, $3.00
inkling-small: 68.2 +/- 2.5, $3.15
deepseek-v4-flash-0731@alibaba-intl: 76.0 +/- 1.8, $3.30
gpt-4.1-mini: 18.6 +/- 2.2, $3.60
qwen3.5-35b-a3b: 54.4 +/- 6.2, $3.75
gpt-5-mini: 34.1 +/- 2.9, $3.75
gpt-3.5-turbo: 1.1 +/- 2.7, $3.75
glm-4.7: 42.9 +/- 2.4, $3.83
deepseek-flash-fast@parasail: 82.9 +/- 1.4, $4.05
minimax-m3-fast@parasail: 71.1 +/- 6.8, $4.05
qwen3-235b-a22b-thinking-2507: 47.6 +/- 2.7, $4.14
qwen3-30b-a3b-thinking-2507: 70.3 +/- 6.0, $4.20
glm-5.2@deepinfra: 41.4 +/- 4.2, $4.39
qwen3.6-35b-a3b: 53.5 +/- 2.8, $4.50
glm-4.6: 3.8 +/- 1.1, $4.50
gemini-3.5-flash-lite: 44.6 +/- 4.2, $4.65
llama-3.3-70b-instruct-turbo: 5.1 +/- 2.4, $4.68
deepseek-r1-0528: 66.4 +/- 4.9, $4.73
mistral-large-4-0: 78.5 +/- 1.5, $5.25
glm-5.3@deepinfra: 79.9 +/- 2.2, $5.44
qwen3.6-27b@deepinfra: 62.6 +/- 2.6, $5.76
qwen3.5-397b-a17b@deepinfra: 60.1 +/- 2.7, $5.85
gemini-3-flash-preview: 59.9 +/- 1.8, $6.00
qwen3.6-plus: 52.6 +/- 1.8, $6.00
qwen3.5-397b-a17b@parasail: 60.1 +/- 2.7, $6.90
qwen3.6-27b: 62.6 +/- 2.6, $7.20
qwen3.5-397b-a17b: 60.1 +/- 2.7, $7.20
kimi-k2.6: 64.1 +/- 1.6, $7.50
deepseek-v4-pro-0813@deepinfra: 81.6 +/- 1.5, $7.80
deepseek-v4-pro@deepinfra: 46.1 +/- 4.6, $7.80
gemini-3.8-flash: 83.7 +/- 1.5, $7.88
gemini-3.7-flash: 84.4 +/- 1.6, $7.88
gemini-3.5-flash: 65.3 +/- 2.2, $7.88
gemini-3.6-flash: 71.7 +/- 3.4, $7.88
kimi-k2.7-code: 74.8 +/- 2.1, $8.85
gpt-5.4-mini: 23.6 +/- 2.7, $9.00
inkling: 61.1 +/- 2.4, $9.08
deepseek-v4-pro-0813: 81.6 +/- 1.5, $9.90
deepseek-v4-pro: 46.1 +/- 4.6, $9.90
deepseek-v4-pro@parasail: 46.1 +/- 4.6, $9.90
deepseek-v4-pro@together: 46.1 +/- 4.6, $9.90
o4-mini: 28.2 +/- 2.7, $9.90
o3-mini: 20.7 +/- 2.7, $9.90
gpt-6.1-terra: 95.4 +/- 2.6, $10.50
claude-haiku-4-5: 20.0 +/- 3.2, $10.50
glm-5.3: 79.9 +/- 2.2, $10.80
glm-5.3@alibaba-intl: 79.9 +/- 2.2, $10.80
glm-5.3@parasail: 79.9 +/- 2.2, $10.80
glm-5.1: 54.5 +/- 1.8, $10.80
glm-5.2: 41.4 +/- 4.2, $10.80
glm-5.2@alibaba-intl: 41.4 +/- 4.2, $10.80
glm-5.2@alibaba-us: 41.4 +/- 4.2, $10.80
deepseek-v4-pro-fast@parasail: 46.1 +/- 4.6, $14.85
qwen3.8-max-0902: 82.8 +/- 2.9, $15.00
qwen3.6-max-preview: 57.1 +/- 2.5, $15.60
glm-5.3-fast@parasail: 79.9 +/- 2.2, $16.20
grok-4.6: 82.6 +/- 1.4, $16.50
grok-4.7: 75.5 +/- 1.6, $16.50
o3: 29.7 +/- 2.7, $18.00
gpt-4.1: 16.6 +/- 2.2, $18.00
qwen3.7-max: 80.1 +/- 2.1, $18.75
qwen3.7-max@alibaba-us: 80.1 +/- 2.1, $18.75
gpt-5: 39.2 +/- 3.0, $18.75
gpt-5.1: 19.9 +/- 3.7, $18.75
gpt-6.1-sol: 96.8 +/- 2.6, $21.00
gemini-4-carbon: 95.3 +/- 2.7, $21.00
gpt-6-sol: 94.9 +/- 2.7, $21.00
claude-sonnet-5-5: 94.4 +/- 2.7, $21.00
gemini-4-argon: 93.0 +/- 2.7, $21.00
claude-sonnet-5: 72.8 +/- 3.3, $21.00
gpt-4o: 2.3 +/- 0.8, $22.50
gemini-3.1-pro-preview: 64.1 +/- 1.7, $24.00
gpt-5.6-terra: 66.8 +/- 4.5, $24.00
gpt-5.2: 51.8 +/- 3.1, $26.25
kimi-k3@deepinfra: 70.4 +/- 4.0, $29.93
gpt-5.4: 60.9 +/- 3.3, $30.00
kimi-k3: 70.4 +/- 4.0, $31.50
kimi-k3@moonshot: 70.4 +/- 4.0, $31.50
claude-sonnet-4-6: 51.8 +/- 3.7, $31.50
claude-sonnet-4-5: 22.8 +/- 2.8, $31.50
claude-opus-5-5: 95.6 +/- 2.6, $42.00
gpt-5.6-sol: 77.2 +/- 4.2, $42.00
claude-opus-5: 92.8 +/- 1.6, $52.50
claude-opus-4-8: 72.3 +/- 2.5, $52.50
claude-opus-4-7: 66.9 +/- 2.2, $52.50
claude-opus-4-6: 61.8 +/- 1.9, $52.50
claude-opus-4-5: 34.4 +/- 2.6, $52.50
gpt-5.5: 69.1 +/- 4.5, $60.00
gpt-4-turbo: 3.7 +/- 1.2, $75.00
claude-fable-5-2: 97.4 +/- 0.7, $84.00
gpt-6-astra: 96.8 +/- 2.6, $105
claude-fable-5-1: 95.1 +/- 2.7, $105
claude-fable-5: 92.9 +/- 1.5, $105
o1: 17.6 +/- 2.8, $135
gpt-4: 1.1 +/- 0.5, $180
gpt-oss-120b@deepinfra Cheap deepseek-v4.1-flash-1017@deepinfra Balanced deepseek-v4-pro@deepinfra gpt-6.1-sol Best Writing code, Hard 0 25 50 75 100 $1 $10 $100
deepseek-v4.1-flash-1017@deepinfra: 38.8 +/- 11.5, $0.59
deepseek-flash@deepinfra: 64.4 +/- 11.4, $0.85
claude-haiku-5-5: 63.4 +/- 7.1, $0.88
gpt-6-luna: 25.7 +/- 6.8, $0.88
gemma-4-31b-it: 8.5 +/- 6.6, $0.94
glm-5.3-flash: 20.7 +/- 9.6, $0.98
glm-5.3-flash@deepinfra: 20.7 +/- 9.6, $0.98
minimax-m3.1-flash: 25.7 +/- 10.3, $1.11
minimax-m3.1-flash@parasail: 25.7 +/- 10.3, $1.11
mimo-v2.6-pro: 36.4 +/- 11.3, $1.83
gpt-5.6-luna: 21.1 +/- 9.8, $2.02
deepseek-flash: 64.4 +/- 11.4, $2.22
deepseek-flash@alibaba-intl: 64.4 +/- 11.4, $2.22
deepseek-flash@parasail: 64.4 +/- 11.4, $2.22
deepseek-flash@together: 64.4 +/- 11.4, $2.22
deepseek-v4.1-flash-1017: 38.8 +/- 11.5, $2.22
deepseek-v4.1-flash-1017@parasail: 38.8 +/- 11.5, $2.22
qwen3.8-27b@deepinfra: 15.9 +/- 8.6, $2.83
deepseek-flash-fast@parasail: 64.4 +/- 11.4, $3.33
qwen3.8-27b@parasail: 15.9 +/- 8.6, $3.45
glm-5.2@deepinfra: 36.7 +/- 11.5, $3.56
mistral-large-4-0: 32.4 +/- 7.4, $4.24
qwen3.8-27b: 15.9 +/- 8.6, $5.05
deepseek-v4-pro-0813@deepinfra: 33.3 +/- 10.0, $6.11
gemini-3.8-flash: 27.7 +/- 7.4, $6.56
gemini-3.7-flash: 41.1 +/- 12.2, $6.56
gemini-3.6-flash: 23.3 +/- 9.3, $6.56
inkling: 9.8 +/- 7.0, $7.47
deepseek-v4-pro-0813: 33.3 +/- 10.0, $7.99
glm-5.2: 36.7 +/- 11.5, $8.74
glm-5.2@alibaba-intl: 36.7 +/- 11.5, $8.74
glm-5.2@alibaba-us: 36.7 +/- 11.5, $8.74
gpt-6.1-terra: 66.4 +/- 7.5, $8.75
qwen3.8-max-0902: 37.2 +/- 7.3, $12.10
gemini-4-hydrogen: 61.8 +/- 11.5, $13.13
grok-4.7: 49.9 +/- 8.2, $13.31
grok-4.6: 29.9 +/- 7.2, $13.31
gpt-5: 4.4 +/- 3.4, $16.00
kimi-k3.1-pro@deepinfra: 38.7 +/- 11.5, $16.63
gemini-4-carbon: 72.7 +/- 10.5, $17.50
gpt-6.1-sol: 63.4 +/- 8.0, $17.50
gemini-4-argon: 58.7 +/- 8.3, $17.50
claude-sonnet-5-5: 80.6 +/- 5.7, $17.50
kimi-k3.1-pro: 38.7 +/- 11.5, $17.50
kimi-k3.1-pro@together: 38.7 +/- 11.5, $17.50
gpt-6-sol: 48.9 +/- 11.2, $17.50
claude-sonnet-5: 63.3 +/- 10.7, $17.50
gpt-5.6-terra: 33.5 +/- 8.1, $20.20
gemini-3.1-pro-preview: 8.9 +/- 4.6, $20.20
gpt-5.2: 32.2 +/- 10.0, $22.40
kimi-k3@deepinfra: 45.2 +/- 8.0, $24.94
gpt-5.4: 23.3 +/- 8.1, $25.25
kimi-k3: 45.2 +/- 8.0, $26.25
kimi-k3@moonshot: 45.2 +/- 8.0, $26.25
claude-opus-5-5: 84.2 +/- 5.4, $35.00
gpt-5.6-sol: 20.0 +/- 9.2, $35.00
claude-opus-5: 87.8 +/- 7.2, $43.75
claude-opus-4-7: 43.3 +/- 9.6, $43.75
claude-opus-4-8: 65.6 +/- 10.9, $43.75
claude-opus-4-6: 33.3 +/- 10.0, $43.75
gpt-5.5: 23.3 +/- 10.1, $50.50
claude-fable-5-2: 82.9 +/- 8.9, $70.00
claude-fable-5-1: 83.8 +/- 5.5, $87.50
gpt-6-astra: 67.8 +/- 7.8, $87.50
claude-fable-5: 70.0 +/- 10.4, $87.50
deepseek-flash@deepinfra Cheap gemini-4-carbon Best, Balanced Fixing code, Hard 0 25 50 75 100 $1 $10 $100
gpt-oss-20b: 3.1 +/- 1.5, $0.25
gpt-oss-120b@deepinfra: 25.6 +/- 1.9, $0.30
gemma-3-27b-it: 4.9 +/- 1.4, $0.38
qwen3.5-9b@deepinfra: 9.2 +/- 2.4, $0.40
qwen3.5-9b@parasail: 9.2 +/- 2.4, $0.54
qwen3-32b: 40.0 +/- 3.3, $0.54
deepseek-v4.1-flash-1017@deepinfra: 80.4 +/- 8.1, $0.59
gpt-5-nano: 7.0 +/- 1.9, $0.64
qwen3.5-9b: 9.2 +/- 2.4, $0.68
gpt-4.1-nano: 8.9 +/- 1.9, $0.74
claude-haiku-5-5: 77.6 +/- 8.5, $0.88
gpt-6-luna: 70.8 +/- 9.3, $0.88
qwen3-235b-a22b-instruct-2507@deepinfra: 59.6 +/- 3.3, $0.92
gemma-4-31b-it: 20.3 +/- 8.2, $0.94
glm-5.3-flash: 70.9 +/- 9.3, $0.98
glm-5.3-flash@deepinfra: 70.9 +/- 9.3, $0.98
deepseek-v3.2: 39.6 +/- 2.8, $1.03
minimax-m3.1-flash: 68.8 +/- 9.5, $1.11
minimax-m3.1-flash@parasail: 68.8 +/- 9.5, $1.11
gpt-oss-120b: 25.6 +/- 1.9, $1.11
gpt-4o-mini: 3.6 +/- 1.2, $1.11
qwen3-235b-a22b-instruct-2507@parasail: 59.6 +/- 3.3, $1.36
qwen3.6-35b-a3b@parasail: 24.6 +/- 3.2, $1.42
qwen3.6-35b-a3b@deepinfra: 24.6 +/- 3.2, $1.48
qwen3-235b-a22b-instruct-2507: 59.6 +/- 3.3, $1.70
mimo-v2.6-pro: 64.5 +/- 9.8, $1.83
deepseek-v4.1-flash-1017: 80.4 +/- 8.1, $2.22
deepseek-v4.1-flash-1017@parasail: 80.4 +/- 8.1, $2.22
qwen3.8-27b@deepinfra: 69.3 +/- 9.4, $2.83
gpt-4.1-mini: 32.4 +/- 3.1, $2.96
glm-4.7: 33.3 +/- 2.5, $3.16
gpt-5-mini: 47.0 +/- 1.7, $3.20
qwen3.8-27b@parasail: 69.3 +/- 9.4, $3.45
glm-5.2@deepinfra: 78.7 +/- 1.9, $3.56
glm-4.6: 24.5 +/- 2.4, $3.70
qwen3.6-35b-a3b: 24.6 +/- 3.2, $3.79
deepseek-r1-0528: 71.4 +/- 3.0, $3.90
mistral-large-4-0: 79.4 +/- 8.3, $4.24
gemini-3-flash-preview: 66.1 +/- 1.7, $5.05
qwen3.8-27b: 69.3 +/- 9.4, $5.05
qwen3.6-plus: 57.9 +/- 2.3, $5.05
deepseek-v4-pro@deepinfra: 77.6 +/- 1.9, $6.11
kimi-k2.6: 76.7 +/- 1.9, $6.23
gemini-3.5-flash: 79.3 +/- 1.8, $6.56
gemini-3.8-flash: 73.3 +/- 9.0, $6.56
inkling: 48.7 +/- 10.2, $7.47
deepseek-v4-pro: 77.6 +/- 1.9, $7.99
deepseek-v4-pro@parasail: 77.6 +/- 1.9, $7.99
deepseek-v4-pro@together: 77.6 +/- 1.9, $7.99
o4-mini: 72.0 +/- 3.0, $8.14
o3-mini: 60.4 +/- 3.3, $8.14
glm-5.2: 78.7 +/- 1.9, $8.74
glm-5.2@alibaba-intl: 78.7 +/- 1.9, $8.74
glm-5.2@alibaba-us: 78.7 +/- 1.9, $8.74
glm-5.1: 74.2 +/- 2.0, $8.74
gpt-6.1-terra: 89.2 +/- 6.3, $8.75
claude-haiku-4-5: 13.9 +/- 2.7, $8.75
deepseek-v4-pro-fast@parasail: 77.6 +/- 1.9, $11.98
qwen3.8-max-0902: 60.9 +/- 10.0, $12.10
gemini-4-hydrogen: 78.5 +/- 8.4, $13.13
qwen3.6-max-preview: 76.7 +/- 1.9, $13.13
grok-4.6: 76.7 +/- 8.6, $13.31
grok-4.7: 73.1 +/- 9.1, $13.31
o3: 68.4 +/- 1.7, $14.80
gpt-4.1: 49.8 +/- 1.9, $14.80
qwen3.7-max: 77.3 +/- 1.9, $15.13
qwen3.7-max@alibaba-us: 77.3 +/- 1.9, $15.13
gpt-5: 63.4 +/- 1.4, $16.00
gpt-5.1: 59.5 +/- 1.7, $16.00
kimi-k3.1-pro@deepinfra: 67.8 +/- 9.5, $16.63
gemini-4-carbon: 91.0 +/- 5.8, $17.50
gemini-4-argon: 85.7 +/- 7.2, $17.50
gpt-6.1-sol: 82.1 +/- 7.8, $17.50
claude-sonnet-5-5: 77.2 +/- 8.6, $17.50
kimi-k3.1-pro: 67.8 +/- 9.5, $17.50
kimi-k3.1-pro@together: 67.8 +/- 9.5, $17.50
gpt-4o: 28.4 +/- 1.7, $18.50
gpt-5.6-terra: 84.2 +/- 7.5, $20.20
gemini-3.1-pro-preview: 61.4 +/- 4.1, $20.20
gpt-5.2: 65.5 +/- 1.7, $22.40
kimi-k3@deepinfra: 53.1 +/- 10.2, $24.94
gpt-5.4: 78.9 +/- 1.4, $25.25
claude-sonnet-4-6: 67.9 +/- 1.6, $26.25
claude-sonnet-4-5: 60.9 +/- 1.7, $26.25
kimi-k3: 53.1 +/- 10.2, $26.25
kimi-k3@moonshot: 53.1 +/- 10.2, $26.25
claude-opus-5-5: 87.1 +/- 6.8, $35.00
claude-opus-4-7: 82.1 +/- 1.3, $43.75
claude-opus-4-6: 75.8 +/- 1.5, $43.75
claude-opus-4-5: 72.1 +/- 1.6, $43.75
gpt-5.5: 66.1 +/- 2.5, $50.50
claude-fable-5-2: 79.2 +/- 8.3, $70.00
claude-fable-5-1: 89.7 +/- 6.2, $87.50
gpt-6-astra: 87.5 +/- 6.8, $87.50
o1: 61.7 +/- 3.2, $111
deepseek-v4.1-flash-1017@deepinfra Cheap deepseek-v4-pro@deepinfra gpt-6.1-terra Best, Balanced