Claude Code cost benchmarks

Through Context Mode Gateway, Claude Code on Claude Opus 5.5 cost 65.5% less on a long agentic coding session, which went from $7.20 to $2.49, 4 of 4 pairs cheaper. On the 10 shorter tasks shown here, 155 of 168 paired runs cost less, and 168 of 168 tasks succeeded in both arms. Each pair ran the same task with Claude Code direct to Anthropic and through the gateway, back to back, with the plan committed before the first pair.

Same task success, lower cost. Task checks passed equally in both arms on every row below. The long-session rows also ask 10 memory questions; each row gives both arms' scores.

Claude Opus 5.5

Cost per task is the mean cost at list price. "Likely" gives the 95% range. Saved per 1,000 tasks is the mean saving per task, times 1,000.

Three parallel subagents: the 15% and 20 of 20 come from the gateway's record, and Claude Code's own cost figures for both arms also give 15% and 20 of 20. The gateway did not save how long each cache write lived, so from the saved tokens alone a reader can confirm 5 to 23% lower, with 15 to 20 of 20 pairs lower.

Long agentic coding session: what else we measured

4 pairs on 2026-10-03, Claude Code 2.1.283 on Claude Opus 5.5 with its real 1M window, Recall on in the Context Mode arm. Each pair ran the same 30 steps in both arms. Every number links to the file it comes from.

• Task checks. 8 of 8 in both arms, in every pair. No compaction and no upstream 4xx error in any of the 8 runs.
• Memory questions. In this run the agent could not search its archive. The run below lets it, the way the product works.
• Tokens sent per call. 330,785 direct, 75,591 through Context Mode (mean of the 4 pairs).
• The same session, archive search allowed. Context Mode cost $2.97 against $7.24 direct, 59.0% lower. Memory answers through Context Mode rose to 10, 10, 10 and 8 of 10, against 10 of 10 direct. What that means.

Long session on a 200K window: savings and quality

3 pairs on 2026-10-04: the same 30 steps as above, with Claude Code 2.1.283 on Claude Opus 5.5 and its real 200K window, and the gateway account's Context limit at 100K. Recall on in the Context Mode arm. The rules were written before the first pair, and all 3 of 3 pairs are valid. What that means.

• Savings. $3.04 per session through Context Mode against $4.86 direct: $1.83 saved, 37.6% lower (30.7 to 44.3%). Cheaper in 3 of 3 pairs.
• Quality. Exact answers to the memory questions: 9 of 10 through Context Mode (the value was in the answer in 10 of 10), 4 of 10 direct. Direct compacted 3 times; through Context Mode the session never compacted. Task checks 8 of 8 in both arms.

All published Opus 5.5 short-task pairs: 155 of 168 cost less

Every Opus 5.5 pair in the three published evidence files, added up: 168 paired runs over 10 tasks, $28.28 direct and $23.16 through Context Mode at list price. 155 of 168 pairs cost less, and 168 of 168 tasks succeeded in both arms. It is a sum over pairs, not an average of rows. These are the 10 tasks where cost was lower: workloads without a lower-cost result are not in the published files. On API billing it is cash; on a Claude plan it is list-price value, not cash.

Claude Haiku 4.5

The same tasks on a smaller, cheaper model.

Task success was equal in both arms on every Haiku 4.5 row. On data analysis, 11 of 12 answers were correct in each arm.

How we measured

• Paired A/B. Each pair runs the same task twice, back to back: once direct to Anthropic, once through Context Mode. The order changes every pair.
• The real client. Claude Code 2.1.283 (claude -p) in both arms, with a fresh config and no plugin for every run.
• The cost. List price applied to the token counts Anthropic returned for each call. The runs used a claude.ai login, which has no per-token invoice. Direct: Claude Code's own total. Context Mode: the gateway's per-call record, hidden calls included. See Cost basis.
• Written down first. For every row, the tasks, the number of pairs and the analysis were committed before the first pair ran. Within each row shown, no pair was dropped.
• Which rows. This page shows the rows where cost was lower. It is not every workload we ran: a pre-registration marked "edited" shows where workloads were taken off.
• Gateway settings. Each workload ran on its own new account, at the default settings, and no run changed them: memory on, skills on with none imported, and Cage off.
• Pairs. 8 to 20 per row, shown in the head-to-head column. One warm-up pair per task ran first and does not count.
• What counts as a win. A row counts only when the whole 95% range is below zero. Head-to-head counts the pairs where Context Mode cost less.
• When. 2026-09-29, on Claude Opus 5.5 and Claude Haiku 4.5.

We also ran the test-fix loop in a second language (Ruby) to check that the result is not tied to one toolchain: Opus 5.5 61% lower cost (likely 52 to 70%), Haiku 4.5 46% lower (likely 34 to 57%), 8 pairs each, 8 of 8 won on each model. We quote no figure for other models or for Codex. Your own saving depends on your mix of work; the console measures it from your traffic.

Verify it yourself

Every evidence file holds the token counts of each call and the list price we used, so you can recompute every dollar yourself. A checking script is available on request.

Sources

Every number on this page comes from an evidence file published on this site: each pair, its cost per arm, and the analysis. The files are copies of the committed files in the gateway repository, with private fields removed.

Full table of the rows where cost was lower, with each row's pairs, range and verdict: launch benchmark, full table.