Join us for

Wagtail Space 2026

18-20 November

Find out more
Get started

Agents on Wagtail

Comparing coding performance, cost and energy

We evaluate the performance of AI agents on Wagtail tasks, so we better understand how to improve DX with AI with our agent skills for developers.

Scatter plot of AI models, with the drawn pareto frontier in the top right. Task accuracy as Y axis, cost per task as X axis. GPT 6 Luna and DeepSeek V4.1 Flash are at the top.

Benchmark scores

Accuracy out of 20 benchmark tasks, with median execution measures per task: cost, energy, carbon footprint, speed.

Model Accuracy Cost Energy (Wh) CO₂ (gCO2e) Speed (m:s)

95%

$0.09

14.9

5.58

11:27

95%

$0.15

–

–

3:32

95%

$0.55

–

–

5:10

95%

$0.14

–

–

1:56

85%

$1.00

34.8

12.31

8:18

85%

$0.01

–

–

3:24

75%

$0.15

–

–

19:27

70%

$0.87

56.9

19.72

18:50

70%

$1.02

–

–

13:12

55%

$0.16

26.2

1.33

13:51

40%

$0.42

8.6

0.53

19:28

35%

$0.01

2.6

0.16

5:47

15%

$0.03

–

–

19:32

Cost: API rates across providers, based on tokens usage. Energy and CO2 emissions: as reported by the Neuralwatt inference provider per the Neuralwatt energy methodology.