Stop paying for cold caches: GitHub Actions runner cache benchmarks

The step nobody profiles

Everyone stares at the test job. It's red, it's slow, it's obviously the problem. Meanwhile the actions/cache restore step at the top of your build quietly eats 40 seconds per job, times 12 jobs, times 300 builds a day. Nobody profiles it because the log line just says Cache restored from key... in cheerful green.

I pulled the numbers on a mid-size monorepo last quarter and the cache restore step alone accounted for 9% of total billed CI minutes. Not the build. Not the tests. The act of fetching a tarball over the network.

This post is about where those minutes go on GitHub Actions runners, what actually moves the needle, and the specific cases where self-hosted runners turn a 40-second restore into a 2-second one. Numbers throughout — measured, not vibes.

Where the time actually goes

GitHub-hosted runners give you a clean VM every run. That's the whole value proposition and also the whole problem. Clean VM means empty disk means every dependency, every Docker layer, every compiled artifact starts from nothing.

The actions/cache action papers over this by shipping a tarball to GitHub's cache backend (Azure Blob Storage under the hood) and pulling it back next run. Here's what that costs on a ubuntu-latest runner for a Node monorepo with a ~1.2 GB node_modules:

Step                         Cold      Warm cache
-------------------------------------------------
Cache restore (node_modules)  --        38s
npm ci                        94s       6s (integrity check only)
Build (tsc + esbuild)         71s       71s
Test                          210s      210s
-------------------------------------------------
Total                         375s      325s

The warm cache saves you 88 seconds on install but hands 38 of them right back on restore. Net win is real but smaller than the marketing implies, and it gets worse as your cache grows. A 3 GB cache pushes restore past 90 seconds, and GitHub caps total cache storage at 10 GB per repo — hit that and older entries get evicted, silently turning warm builds cold again.

Optimization 1: cache less, not more

The instinct when a cache is slow is to cache more aggressively. Wrong direction. The tarball has to be compressed, uploaded, downloaded, and decompressed. Every megabyte is round-trip latency.

For Node, cache the package manager's store, not node_modules:

- uses: actions/setup-node@v4
  with:
    node-version: 20
    cache: 'npm'   # caches ~/.npm, not node_modules

- run: npm ci

The ~/.npm cache is smaller and more compressible than a fully-linked node_modules, and npm ci still runs but pulls from the local store instead of the network. In the same monorepo this dropped restore from 38s to 14s, and npm ci from 6s to 19s — net faster and more correct, because you're not shipping symlinked, platform-specific binaries across runner images.

pnpm is even better here because its content-addressable store dedupes across the whole cache. Same principle for Go (~/go/pkg/mod), Rust (~/.cargo plus target selectively), and Python (~/.cache/pip). Cache the source of truth, let the tool rebuild the linked tree locally.

Optimization 2: scope your cache keys or pay for garbage

A cache key that's too broad restores stale junk. Too narrow and you never hit. The sweet spot is a lockfile hash with a sane restore fallback:

- uses: actions/cache@v4
  with:
    path: ~/.npm
    key: npm-${{ runner.os }}-${{ hashFiles('package-lock.json') }}
    restore-keys: |
      npm-${{ runner.os }}-

The restore-keys fallback matters more than people think. On a lockfile change, an exact-key miss with a prefix hit still restores 95% of your dependencies — you only download the delta. Without it, one dependency bump means a full cold install. I've seen teams eat 90-second cold installs on every Dependabot PR because they had no restore-keys line. That's a one-line fix worth thousands of minutes a month.

Optimization 3: Docker layer caching that doesn't lie to you

Docker builds on hosted runners are where minutes go to die. The default has no layer cache at all — every docker build starts from your base image and rebuilds every layer.

docker/build-push-action with a registry cache backend helps:

- uses: docker/build-push-action@v6
  with:
    context: .
    push: true
    tags: myreg/app:${{ github.sha }}
    cache-from: type=registry,ref=myreg/app:buildcache
    cache-to: type=registry,ref=myreg/app:buildcache,mode=max

But mode=max caches every intermediate layer, and pulling those layers back over the network on a hosted runner can cost more than rebuilding cheap layers. Benchmark it. For our image, mode=max restore was 62s vs a 71s cold build — a 9-second win that evaporated the moment the base image changed. type=gha (the GitHub Actions cache backend) was worse: it shares the same 10 GB repo cap and evicted our npm cache to make room.

The honest answer for heavy Docker workloads is that the cache-over-network model has a ceiling, and you hit it fast.

Where self-hosted runners change the math

This is the part hosted runners can't fix by tuning YAML. On a self-hosted runner with a persistent disk, the cache is the disk. No tarball, no upload, no download. node_modules is already there. The Docker layer cache is already warm in the local BuildKit store.

Same monorepo, same jobs, on a persistent self-hosted runner:

Step              Hosted (warm)   Self-hosted (persistent)
---------------------------------------------------------
Cache restore      14s             0s (nothing to restore)
npm ci             19s             3s
Docker build       62s (cache)     8s (local layer reuse)
Test               210s            188s (warmer CPU cache)
---------------------------------------------------------
Total              305s            199s

A 35% wall-clock reduction, and most of it comes from deleting the cache dance entirely. The catch: persistent disk means persistent state, which means test pollution and dependency drift are now your problem. A runner that never resets will eventually accumulate a rogue global package or a leftover port binding that makes a test pass locally and fail in CI — or worse, the reverse.

That's the trade. Hosted runners give you cleanliness at the cost of cold starts. Persistent self-hosted runners give you speed at the cost of a mutable environment you have to actively manage.

BuildPulse's managed runners split the difference: warm caches and local layer reuse for the speed, ephemeral job isolation so state doesn't leak between runs. The point isn't that self-hosting is always right — it's that the cache-over-network overhead is a real, measurable tax on hosted runners, and past a certain build volume it's cheaper to keep the cache on disk.

Don't let a fast build hide a flaky one

Here's the trap. You shave 100 seconds off every build, everyone's thrilled, and then a persistent runner starts leaking state between jobs. A test that depended on a fresh environment now passes or fails depending on what ran before it. Congratulations — you've traded slow-and-reliable for fast-and-flaky.

That 22-second test speedup on the persistent runner above? Some of it is warm CPU cache. Some of it might be a test quietly reusing a fixture the previous job left behind. You can't tell the difference from a green checkmark. This is exactly why flaky-test detection has to sit alongside any runner change — a faster pipeline that you can't trust the signal from isn't faster, it's just wrong sooner.

When you migrate to self-hosted or persistent runners, watch your flake rate before and after. If new intermittent failures show up right after the migration, they're almost always environment leakage, not real bugs. Fix the isolation, not the tests.

The short version

• Profile the cache restore step. It's not free and it grows with your dependency tree.
• Cache the package manager store, not the linked dependency tree. Smaller, more correct.
• Always set restore-keys — a prefix fallback turns cold installs into deltas.
• Benchmark Docker mode=max before trusting it; the network round-trip has a ceiling.
• Persistent self-hosted runners delete the cache overhead entirely, but hand you state management in return.
• Whatever you change, track flake rate across the migration. A faster pipeline with a poisoned signal is a downgrade.

Every second in your build config is measurable. Measure it. The runners you're optimizing for are the ones whose numbers you actually pulled, not the ones the docs promised.