Zabio has not run its own tests on this model. Everything below is aggregated from public benchmarks, with the sources named.
Price against score in each kind of job (14) Price per 1,000 tasks, USD. Score, %. Greyed out: a cheaper model already scores as well. Ringed: the best, balanced and cheap choice.
Writing, Medium 50 60 70 80 90 100 $1 $10
gpt-5.4-nano: 64.6 +/- 3.9, $0.66
deepseek-flash: 66.3 +/- 3.9, $0.72
deepseek-flash@together: 66.3 +/- 3.9, $0.72
gemini-3.5-flash-lite: 68.4 +/- 3.8, $1.24
gemini-3.8-flash: 83.9 +/- 3.1, $2.10
gpt-5.4-mini: 64.9 +/- 3.9, $2.40
deepseek-v4-pro: 67.2 +/- 3.9, $2.64
deepseek-v4-pro@together: 67.2 +/- 3.9, $2.64
grok-4.7: 79.1 +/- 3.4, $4.40
gemini-3.5-flash: 80.1 +/- 3.4, $4.80
claude-sonnet-5: 68.1 +/- 3.9, $5.60
gemini-3.1-pro-preview: 82.6 +/- 3.2, $6.40
gpt-5.4: 73.0 +/- 3.7, $8.00
claude-sonnet-4-6: 67.8 +/- 3.9, $8.40
claude-opus-5-5: 75.3 +/- 3.7, $11.20
claude-opus-5: 77.2 +/- 3.6, $14.00
claude-opus-4-8: 73.8 +/- 3.7, $14.00
gpt-5.5: 79.6 +/- 3.5, $16.00
claude-fable-5-1: 80.5 +/- 3.4, $28.00
gpt-6-astra: 83.8 +/- 3.2, $28.00
gemini-3.8-flash Best, Balanced, Cheap Creative writing, Medium 50 60 70 80 90 100 $1 $10
gpt-5.4-nano: 67.7 +/- 6.6, $0.66
deepseek-flash: 67.3 +/- 6.6, $0.72
deepseek-flash@together: 67.3 +/- 6.6, $0.72
gemini-3.5-flash-lite: 67.5 +/- 6.6, $1.24
gemini-3.8-flash: 82.8 +/- 5.3, $2.10
gpt-5.4-mini: 66.6 +/- 6.7, $2.40
deepseek-v4-pro: 67.8 +/- 6.6, $2.64
deepseek-v4-pro@together: 67.8 +/- 6.6, $2.64
grok-4.7: 74.9 +/- 6.1, $4.40
gemini-3.5-flash: 75.2 +/- 6.1, $4.80
claude-sonnet-5: 62.3 +/- 6.9, $5.60
gemini-3.1-pro-preview: 77.3 +/- 5.9, $6.40
gpt-5.4: 79.0 +/- 5.8, $8.00
claude-sonnet-4-6: 68.9 +/- 6.6, $8.40
claude-opus-5-5: 70.9 +/- 6.4, $11.20
claude-opus-4-8: 80.6 +/- 5.6, $14.00
claude-opus-5: 61.3 +/- 6.9, $14.00
gpt-5.5: 72.6 +/- 6.3, $16.00
claude-fable-5-1: 78.1 +/- 5.9, $28.00
gpt-6-astra: 74.0 +/- 6.2, $28.00
gemini-3.8-flash Best, Balanced, Cheap Pulling out data, Medium 50 60 70 80 90 100 $1 $10
gpt-5.4-nano: 82.8 +/- 2.1, $0.41
deepseek-flash: 83.0 +/- 2.1, $0.48
deepseek-flash@together: 83.0 +/- 2.1, $0.48
gemini-3.5-flash-lite: 75.7 +/- 2.3, $0.74
gemini-3.8-flash: 75.1 +/- 2.5, $1.35
gpt-5.4-mini: 85.4 +/- 1.9, $1.50
deepseek-v4-pro: 87.3 +/- 1.9, $1.85
deepseek-v4-pro@together: 87.3 +/- 1.9, $1.85
gemini-3.5-flash: 80.5 +/- 2.4, $3.00
grok-4.7: 88.4 +/- 1.8, $3.08
claude-sonnet-5: 84.9 +/- 2.1, $3.60
gemini-3.1-pro-preview: 88.3 +/- 1.9, $4.00
gpt-5.4: 89.7 +/- 1.7, $5.00
claude-sonnet-4-6: 87.0 +/- 2.2, $5.40
claude-opus-5-5: 89.2 +/- 1.9, $7.20
claude-opus-5: 84.3 +/- 2.4, $9.00
claude-opus-4-8: 80.1 +/- 2.5, $9.00
gpt-5.5: 90.8 +/- 1.7, $10.00
gpt-6-astra: 91.5 +/- 1.6, $18.00
claude-fable-5-1: 87.2 +/- 2.3, $18.00
gpt-5.4-mini Cheap deepseek-v4-pro Balanced claude-opus-5-5 Best Conversation, Medium 0 25 50 75 100 $1 $10
gpt-4o-mini: 8.3 +/- 0.9, $0.48
gpt-5.4-nano: 11.7 +/- 1.0, $0.82
deepseek-flash: 33.6 +/- 1.5, $0.96
deepseek-flash@together: 33.6 +/- 1.5, $0.96
gpt-5-mini: 21.6 +/- 1.3, $1.20
gpt-4.1-mini: 12.7 +/- 1.1, $1.28
gemini-3.8-flash: 69.7 +/- 1.5, $2.70
gpt-5.4-mini: 29.4 +/- 1.4, $3.00
claude-haiku-4-5: 13.2 +/- 1.1, $3.60
deepseek-v4-pro: 52.9 +/- 1.6, $3.70
deepseek-v4-pro@together: 52.9 +/- 1.6, $3.70
gemini-3.5-flash: 66.2 +/- 1.5, $6.00
gpt-4.1: 31.1 +/- 1.5, $6.40
claude-sonnet-5: 33.7 +/- 1.5, $7.20
gemini-3.1-pro-preview: 73.5 +/- 1.4, $8.00
gpt-5.4: 45.1 +/- 1.6, $10.00
claude-sonnet-4-6: 32.8 +/- 1.5, $10.80
claude-opus-5-5: 72.2 +/- 1.4, $14.40
claude-opus-4-8: 53.0 +/- 1.6, $18.00
claude-opus-5: 59.9 +/- 1.6, $18.00
gpt-5.5: 63.0 +/- 1.5, $20.00
gpt-6-astra: 75.6 +/- 1.4, $36.00
claude-fable-5-1: 70.8 +/- 1.4, $36.00
gemini-3.8-flash Cheap gemini-3.1-pro-preview Best, Balanced Analysis and reasoning, Medium 0 25 50 75 100 $1 $10
gpt-oss-120b: 22.1 +/- 2.9, $0.36
gpt-4o-mini: 10.7 +/- 2.2, $0.36
deepseek-flash: 61.1 +/- 3.5, $0.72
deepseek-flash@together: 61.1 +/- 3.5, $0.72
mistral-large-2512: 22.5 +/- 3.0, $1.00
llama-3.3-70b-instruct-turbo: 19.9 +/- 2.8, $1.25
gemini-3.5-flash: 76.7 +/- 3.0, $4.80
gpt-4.1: 27.0 +/- 3.1, $4.80
claude-sonnet-5: 60.6 +/- 3.5, $5.60
gemini-3.1-pro-preview: 79.6 +/- 2.9, $6.40
claude-opus-5: 80.6 +/- 2.8, $14.00
claude-opus-4-8: 64.8 +/- 3.4, $14.00
gpt-5.5: 69.0 +/- 3.3, $16.00
gpt-oss-120b gemini-3.5-flash Best, Balanced, Cheap Analysis and reasoning, Hard 0 25 50 75 100 $1 $10 $100
qwen3.5-9b: 79.0 +/- 2.4, $0.57
gpt-oss-120b: 75.8 +/- 2.7, $0.84
mistral-small-2603: 47.5 +/- 2.8, $0.84
gpt-4o-mini: 37.7 +/- 2.4, $0.84
gpt-5.4-nano: 66.1 +/- 2.4, $1.53
deepseek-flash: 79.0 +/- 2.0, $1.68
deepseek-flash@together: 79.0 +/- 2.0, $1.68
gpt-4.1-mini: 65.9 +/- 2.7, $2.24
gpt-5-mini: 71.7 +/- 3.2, $2.30
mistral-large-2512: 51.3 +/- 2.7, $2.35
gemini-3.5-flash-lite: 68.7 +/- 2.3, $2.85
llama-3.3-70b-instruct-turbo: 47.4 +/- 2.9, $3.02
gemini-3.8-flash: 92.3 +/- 1.3, $4.88
gpt-5.4-mini: 67.1 +/- 2.4, $5.55
deepseek-v4-pro: 78.0 +/- 2.1, $6.20
deepseek-v4-pro@together: 78.0 +/- 2.1, $6.20
claude-haiku-4-5: 60.5 +/- 2.8, $6.50
mistral-medium-3-5: 59.5 +/- 2.8, $9.75
grok-4.7: 82.7 +/- 2.7, $10.34
gemini-3.5-flash: 85.8 +/- 1.7, $11.10
gpt-4.1: 66.9 +/- 2.8, $11.20
claude-sonnet-5: 84.5 +/- 1.8, $13.00
gemini-3.1-pro-preview: 89.5 +/- 1.5, $14.80
gpt-5.4: 80.3 +/- 2.1, $18.50
claude-sonnet-4-6: 81.8 +/- 1.9, $19.50
claude-opus-5-5: 91.4 +/- 1.4, $26.00
claude-opus-5: 92.2 +/- 1.4, $32.50
claude-opus-4-8: 86.9 +/- 1.7, $32.50
gpt-5.5: 82.4 +/- 2.0, $37.00
gpt-6-astra: 94.2 +/- 1.2, $65.00
claude-fable-5-1: 91.7 +/- 1.9, $65.00
gpt-oss-120b gemini-3.8-flash Cheap claude-opus-5-5 Best, Balanced Analysis and reasoning, Hardest 0 25 50 75 100 $1 $10 $100
gpt-4o-mini: 0.0 +/- 0.4, $1.35
gpt-5.4-nano: 9.9 +/- 1.1, $2.48
deepseek-flash: 7.0 +/- 1.0, $2.70
deepseek-flash@together: 7.0 +/- 1.0, $2.70
gpt-4.1-mini: 1.8 +/- 0.5, $3.60
gpt-5-mini: 15.8 +/- 0.9, $3.75
gemini-3.5-flash-lite: 9.3 +/- 1.1, $4.65
gemini-3.8-flash: 44.5 +/- 2.0, $7.88
gpt-5.4-mini: 7.1 +/- 1.0, $9.00
deepseek-v4-pro: 6.9 +/- 0.9, $9.90
deepseek-v4-pro@together: 6.9 +/- 0.9, $9.90
claude-haiku-4-5: 7.8 +/- 1.0, $10.50
mistral-medium-3-5: 4.5 +/- 0.8, $15.75
gemini-3.5-flash: 28.9 +/- 1.8, $18.00
gpt-4.1: 3.8 +/- 0.5, $18.00
gemini-3.1-pro-preview: 73.8 +/- 1.5, $24.00
gpt-5.4: 45.0 +/- 1.8, $30.00
claude-sonnet-4-6: 73.5 +/- 2.4, $31.50
claude-opus-5-5: 94.6 +/- 1.3, $42.00
claude-opus-5: 92.9 +/- 1.5, $52.50
claude-opus-4-8: 75.1 +/- 2.4, $52.50
gpt-5.5: 54.8 +/- 2.4, $60.00
claude-fable-5-1: 73.2 +/- 1.6, $105
gpt-6-astra: 66.8 +/- 1.7, $105
gemini-3.8-flash Cheap claude-opus-5-5 Best, Balanced Math, Hardest 0 25 50 75 100 $1 $10 $100
qwen3.5-9b: 42.8 +/- 6.5, $0.89
gpt-oss-120b: 88.9 +/- 4.5, $1.35
mistral-small-2603: 5.3 +/- 2.6, $1.35
gpt-4o-mini: 3.8 +/- 1.4, $1.35
gpt-5.4-nano: 25.7 +/- 2.9, $2.48
deepseek-flash: 76.0 +/- 1.8, $2.70
deepseek-flash@together: 76.0 +/- 1.8, $2.70
gpt-4.1-mini: 18.6 +/- 2.2, $3.60
gpt-5-mini: 34.1 +/- 2.9, $3.75
mistral-large-2512: 1.1 +/- 0.5, $3.75
gemini-3.5-flash-lite: 44.6 +/- 4.2, $4.65
llama-3.3-70b-instruct-turbo: 5.1 +/- 2.4, $4.68
gemini-3.8-flash: 83.7 +/- 1.5, $7.88
gpt-5.4-mini: 23.6 +/- 2.7, $9.00
deepseek-v4-pro: 54.0 +/- 4.6, $9.90
deepseek-v4-pro@together: 54.0 +/- 4.6, $9.90
claude-haiku-4-5: 20.0 +/- 3.2, $10.50
mistral-medium-3-5: 16.3 +/- 3.0, $15.75
gemini-3.5-flash: 65.3 +/- 2.2, $18.00
gpt-4.1: 16.6 +/- 2.2, $18.00
claude-sonnet-5: 72.8 +/- 3.3, $21.00
gemini-3.1-pro-preview: 64.1 +/- 1.7, $24.00
gpt-5.4: 60.9 +/- 3.3, $30.00
claude-sonnet-4-6: 51.8 +/- 3.7, $31.50
claude-opus-5-5: 95.6 +/- 2.6, $42.00
claude-opus-5: 92.8 +/- 1.6, $52.50
claude-opus-4-8: 72.3 +/- 2.5, $52.50
gpt-5.5: 69.1 +/- 4.5, $60.00
gpt-6-astra: 96.8 +/- 2.6, $105
claude-fable-5-1: 95.1 +/- 2.7, $105
gpt-oss-120b Cheap deepseek-flash Balanced claude-opus-5-5 Best Writing code, Hard 0 25 50 75 100 $2 $5 $10 $20 $50 $100
gpt-5.4-nano: 58.8 +/- 3.6, $2.09
deepseek-flash: 53.4 +/- 3.6, $2.22
deepseek-flash@together: 53.4 +/- 3.6, $2.22
gemini-3.5-flash-lite: 60.7 +/- 3.5, $3.98
gemini-3.8-flash: 63.4 +/- 3.6, $6.56
gpt-5.4-mini: 56.7 +/- 3.6, $7.58
deepseek-v4-pro: 56.3 +/- 3.6, $7.99
deepseek-v4-pro@together: 56.3 +/- 3.6, $7.99
grok-4.7: 65.6 +/- 3.5, $13.31
gemini-3.5-flash: 63.6 +/- 3.5, $15.15
claude-sonnet-5: 70.0 +/- 3.4, $17.50
gemini-3.1-pro-preview: 43.2 +/- 2.8, $20.20
gpt-5.4: 49.0 +/- 3.5, $25.25
claude-sonnet-4-6: 61.0 +/- 3.5, $26.25
claude-opus-5-5: 79.5 +/- 3.4, $35.00
claude-opus-5: 73.3 +/- 3.3, $43.75
claude-opus-4-8: 66.2 +/- 3.4, $43.75
gpt-5.5: 48.7 +/- 3.1, $50.50
claude-fable-5-1: 75.3 +/- 3.7, $87.50
gpt-6-astra: 61.5 +/- 4.6, $87.50
claude-opus-5-5 Best, Balanced, Cheap Fixing code, Hard 0 25 50 75 100 $1 $10 $100
qwen3.5-9b: 9.2 +/- 2.4, $0.68
gpt-oss-120b: 25.6 +/- 1.9, $1.11
gpt-4o-mini: 3.6 +/- 1.2, $1.11
codestral-2508: 11.1 +/- 2.1, $1.82
gpt-5.4-nano: 46.8 +/- 5.9, $2.09
deepseek-flash: 37.6 +/- 5.7, $2.22
deepseek-flash@together: 37.6 +/- 5.7, $2.22
gpt-4.1-mini: 32.4 +/- 3.1, $2.96
gpt-5-mini: 47.0 +/- 1.7, $3.20
gemini-3.5-flash-lite: 45.3 +/- 5.9, $3.98
gemini-3.8-flash: 54.2 +/- 5.9, $6.56
gpt-5.4-mini: 41.7 +/- 5.8, $7.58
deepseek-v4-pro: 57.1 +/- 3.5, $7.99
deepseek-v4-pro@together: 57.1 +/- 3.5, $7.99
claude-haiku-4-5: 13.9 +/- 2.7, $8.75
grok-4.7: 54.0 +/- 5.9, $13.31
gpt-4.1: 49.8 +/- 1.9, $14.80
gemini-3.5-flash: 64.2 +/- 3.1, $15.15
claude-sonnet-5: 59.4 +/- 5.8, $17.50
gemini-3.1-pro-preview: 49.8 +/- 4.0, $20.20
gpt-5.4: 69.7 +/- 2.3, $25.25
claude-sonnet-4-6: 59.9 +/- 2.2, $26.25
claude-opus-5-5: 71.7 +/- 5.3, $35.00
claude-opus-5: 65.2 +/- 5.6, $43.75
claude-opus-4-8: 50.5 +/- 5.9, $43.75
gpt-5.5: 59.0 +/- 3.6, $50.50
claude-fable-5-1: 66.1 +/- 5.6, $87.50
gpt-6-astra: 57.3 +/- 5.8, $87.50
gpt-oss-120b claude-sonnet-5 Cheap claude-opus-5-5 Best, Balanced Using tools, Hard 0 25 50 75 100 $1 $10 $100
qwen3.5-9b: 9.2 +/- 2.4, $1.13
gpt-oss-120b: 14.2 +/- 2.3, $1.68
gpt-5-mini: 22.3 +/- 2.6, $4.60
claude-haiku-4-5: 13.9 +/- 2.7, $13.00
gemini-3.1-pro-preview: 59.4 +/- 4.2, $29.60
gpt-5.4: 81.8 +/- 2.0, $37.00
claude-sonnet-4-6: 62.8 +/- 1.8, $39.00
gpt-5.5: 66.1 +/- 2.5, $74.00
gpt-oss-120b gpt-5.4 Best, Balanced, Cheap Questions about a document, Hard 0 25 50 75 100 $1 $2 $5 $10 $20 $50
gpt-oss-120b: 36.1 +/- 5.0, $1.74
gpt-5-mini: 69.4 +/- 5.0, $3.80
gpt-4.1-mini: 44.4 +/- 5.0, $4.64
llama-3.3-70b-instruct-turbo: 33.3 +/- 5.0, $9.26
gpt-4.1: 63.9 +/- 5.0, $23.20
gpt-oss-120b