Eighteen months ago a cold start on our edge runtime took 41 milliseconds at the median and well over 200 at the tail. Today the median is 1.7 milliseconds in every one of our 384 edge regions. This is the story of the three changes that got us there, and the two that did not.
Where the time actually went
Before optimising anything we traced 2.3 million cold starts end to end. The surprise was that compiling user code was not the bottleneck. Almost 60 per cent of the time was spent creating a fresh isolate, allocating its heap and running the bootstrap script that wires up the standard library.
Fetching the bundle from regional storage accounted for another 25 per cent, and the remainder was split between module instantiation and the first request handler call. That breakdown told us exactly where to dig.
Isolate pooling with warm heaps
Each edge node now keeps a small pool of pre-forked isolates whose heaps already contain the bootstrapped runtime. When a request arrives for code that is not resident, we take an isolate from the pool, attach the tenant's module and hand it the request. Pool size adapts to traffic every ten seconds, so quiet regions hold only a handful while busy ones keep hundreds ready.
Isolation is not weakened. A pooled isolate has never run tenant code, and once used it is destroyed rather than returned. The pool simply moves the expensive part of creation off the request path.
export default {
runtime: "edge",
regions: "global",
warmPool: { min: 4, max: 512, scaleEvery: "10s" },
snapshot: { enabled: true, restore: "deterministic" },
};The fastest cold start is the one that does nothing new. Every gain came from moving work off the request path, never from making the work itself cleverer.
Deterministic snapshot restoration
The second change was to snapshot each tenant module after its top-level code has run, then restore that snapshot instead of re-executing initialisation. Restoring a snapshot is a memory copy; re-running initialisation can mean parsing configuration, building routing tables and opening connections.
Snapshots must be deterministic to be safe, so we reject any module whose top level reads the clock, generates random numbers or performs input and output. In practice 94 per cent of deployed modules qualify without changes, and the build output tells developers how to fix the rest.
Moving bundles closer to the CPU
Finally, we stopped treating bundle storage as a separate tier. Every deployment is now pushed to all regions at publish time and pinned in local memory on the nodes most likely to serve it, based on the previous hour of traffic. The fetch step has gone from milliseconds to microseconds.
- Bundles are content addressed, so identical code is stored once per node.
- Eviction follows recent traffic rather than age, which keeps popular routes resident.
- A deploy is acknowledged only after 99 per cent of regions confirm the bundle.
What did not work
We tried ahead-of-time compiling every module to native code. Start-up improved slightly, but binaries were four times larger and pushed other tenants out of memory, which made the tail worse. We also tried predicting which modules to warm from request headers alone; the hit rate never passed 70 per cent and the extra work cost more than it saved.
- Measure the whole cold start before optimising any single step.
- Pool the expensive, tenant-free work and destroy isolates after use.
- Snapshots are only safe when module start-up is deterministic.
- Data locality matters as much as compute speed at the edge.