"Worker exceeded memory limit": 18 days of fixing it on Cloudflare Workers

October 23, 2026Engineering8 min read

Your coding agent shows "API Error", and you did nothing wrong. For 18 days, that is what some Context Mode users saw. Our gateway runs on Cloudflare Workers, and long agent sessions send 20 to 36 MB on every turn. When one of those requests pushed an isolate past 128 MB, Cloudflare killed the isolate, and every request on it failed with it. Here is what caused it, what did not work, what we moved to Rust, and the checklist we would give anyone with heavy request bodies on Workers. Killed requests went from 35.2 per hour to 0.

The short version

• An isolate does not die of what it holds. It dies of what it allocates faster than the garbage collector can free it.
• A 64-character slice of a 25 MB string keeps all 25 MB alive. We kept 128 of them per isolate.
• The hot path that reads request bodies now runs in Rust and Wasm: 55 to 58% less CPU, 52 to 77% less memory in our test.
• Twice we called it fixed after one quiet hour. It was not. Decide on 3 hours or more of daytime traffic.

The problem

A coding agent sends the whole conversation on every turn. A long session reaches 2,000 to 7,000 messages and 20 to 36 MB per request. Every request on one isolate shares the same 128 MB. So one big request can kill every small request next to it. On 22 October, one isolate death took 64 requests with it.

We count two numbers. A death is one isolate killed. A killed request is every request that was on it. One number hides the other.

See it in one isolate

// a view: keeps the whole body alive
tail = raw.slice(N - 64, N)

// a new string: the body can go
tail = copy(raw, N - 64, N)

CPU per request  471.0 → 211.0 ms −55.2%
Wasm memory      2.9 MB, 10 bodies

Six platform facts that decided everything

1. 128 MB per isolate, shared by every request on it.
2. Garbage counts until GC runs. On a 36 MB request, the live heap after a forced GC was about 38 MB. But the turn allocated about 144 MB to decode the body to text, 101 MB to count bytes and 51 MB in one toLowerCase.
3. A V8 slice keeps its parent alive. raw.slice(N-64, N) is a 64-character view of the whole string.
4. Wasm memory never shrinks. Once an instance grows, it keeps that memory for the life of the isolate. Copy a 30 MB body into it once, and you lose 30 MB for good.
5. Isolate memory is not shared. A map or cache in one isolate is empty in the next. The Cache API is per data center and outlives isolates.
6. waitUntil keeps its inputs alive. After the answer, background writes still held 26.7 MB on a 36 MB turn. It was gone 300 to 500 ms later: queued work, not a leak.

Two more limits came from outside. The model provider takes at most 32 MB per request, so the gateway must read the body to cut it. And Workers run on V8, so Bun is not an option. The only native-speed path inside a Worker is Wasm.

What did not work

The causes we found and fixed

Local numbers come from a heap probe with bodies shaped like real traffic. Every fix went straight to production.

1. The archive wrote the whole conversation again on every turn

"This message is already stored" lived only in isolate memory. A new isolate archived all 7,700 messages again. Now a record per conversation sits in the Cache API, written only after the store confirms. On a 24 MB conversation in a new isolate: CPU 2.7 s to 0.74 s, peak 92 to 51 MB, lookups 7,746 to 16.

2. A tokenizer built every word one character at a time

cur += fold(c) for each character. One 36 MB turn allocated 1,473 MB in the archive write. Slicing runs of characters that need no change cut it to 141 MB.

3. A timer let bodies in over the budget

Big bodies wait for a memory budget. But after a timeout (2 s to 240 s), a waiting body got in anyway. One isolate died holding two 24.9 MB bodies and two more of 9.0 and 7.2 MB. Now no timer admits over the budget.

4. The queue had no bound

After fix 3, waiting bodies stayed on one isolate for hours, and one death took them all. Now an isolate holds at most 4 waiting large bodies. The rest get a 408 with retry-after and x-should-retry: true. We first used 503, but agent clients treat some overload answers as a reason to switch models. We checked the real client code and tested that it retries the 408.

5. A 64-character tail kept the whole body alive

We stored the last 64 characters of each body in a map, with slice. V8 kept each full body alive, 128 per isolate. Normal 1 MB requests then killed isolates. The fix is to copy the tail into a new string. Over 80 conversations in a row: retained memory 57 to 2.4 MB, peak 98 to 36 MB.

6. Large bodies were parsed whole

Bodies over 12 MiB are now read item by item from the Rust scanner, never as one JSON.parse. Peak for a 30 MiB body: 95 to 32 MB.

The move to Rust and Wasm

The rule: everything that does memory or CPU work on a request body moves to Rust. JavaScript keeps only I/O: the Worker entry, bindings, fetch, storage calls and waitUntil. We built a prototype first, on workerd with a 35.5 MiB body, 3 runs:

Forwarded bytes and archive ids matched in 51 of 51 requests. Wasm memory stayed at 2.9 MB over 10 bodies in a row. Rust now does the JSON scan, the stream cut, hashes, archive records, length counts and the secret check, with one shared Wasm instance per isolate. Workers Builds compiles it from source; we commit no binary.

• Never copy the body into Wasm. Feed it through one fixed 1 MiB buffer, in slices.
• Keep SHA-256 native in JavaScript.
• Test every Rust part against the JavaScript output: forwarded bytes, archive ids and stored rows, on CJK text, images, thinking blocks and bodies from 1 to 36 MB.

Some JavaScript stays, and we say where: JSON.parse on bodies of 1 to 12 MiB, and one parse per message in Recall. Moving them means the passes must edit bytes, not objects.

Live results

Night has less traffic than day, so compare windows at the same time of day. One death at 04:32 (one request) we could not reproduce yet.

What we got wrong

• We judged fixes by the first hour. Twice we saw zero deaths for 20 to 47 minutes and called it fixed. One of those versions then died 40 times in 3.6 hours.
• Our heap tests never ran the Rust code. The test bundle swapped the Wasm module for a stub, so "heap test passed" measured the JavaScript fallback. Now the tests load the module production loads.
• Our probe hid the shared cache. It gave each isolate a new Cache API. In production the cache is shared per data center.
• We blamed the newest commits. Numbers per tenant showed one client's requests caused most deaths on both versions.
• A push did not always start a build. Now we check that a build exists for the exact commit, and start one if it does not.

A checklist for heavy request bodies on Workers

1. Count isolate deaths and killed requests, per version.
2. Watch allocation, not only retained heap. Profile a body shaped like your real traffic.
3. Never keep a slice of a big string in a long-lived map. Copy it.
4. Keep "already done" records in the Cache API, written after the store confirms. Isolate memory is gone on the next isolate.
5. Admit big bodies against a byte budget. Never let a timer admit over it. Bound the queue, and answer the rest with a status your clients really retry.
6. Do not parse a 20 MB JSON body into objects if you can scan bytes.
7. In Wasm: one reusable buffer, never the whole body, hashes native in JavaScript, and a test against the old output for every part.
8. Make your heap tests load the same module production loads.
9. Decide on 3 hours or more of daytime traffic, not the first quiet hour.