Time-dependent flaky tests: why your CI only fails at midnight

The build that only fails after dinner

A release candidate fails on main at 00:03 UTC. The on-call engineer, who is asleep because they are a functioning human, wakes up to a red pipeline, hits rerun at 07:40, and watches it go green. Ticket closed: "flaky, passed on retry."

Three weeks later it happens again. At 00:01 UTC. Nobody connects the two events, because who looks at the wall-clock timestamp of a test failure?

You should. Time-dependent flaky tests are one of the most common root-cause classes I see in real suites, and they are almost perfectly camouflaged. They don't fail randomly. They fail on a schedule. And the schedule is exactly when nobody is watching: midnight boundaries, the last day of the month, the two Sundays a year when daylight saving time shifts, and February 29th, which arrives just often enough to break code written by people who no longer work at your company.

Why clock flakes hide from naive detection

Most teams reason about flaky tests statistically: this test failed on 3% of runs, so it's flaky. That model works for race conditions and resource contention. It quietly fails for clock bugs, for two reasons.

First, the failure rate is misleadingly low. A test that breaks in the last 90 seconds before midnight fails on roughly 0.1% of uniformly distributed runs. That's below the noise floor of most eyeball-based test detection. It looks like cosmic rays.

Second, the retry always passes. By the time a human or an auto-retry mechanism reruns the job, the clock has moved past the boundary. The rerun isn't evidence that the failure was spurious; it's evidence that time is monotonic. Rerun-to-green is precisely the wrong instinct here, because it destroys the one signal you had.

The tell is temporal clustering. If you record every failure occurrence with a timestamp (not just a count), clock bugs light up immediately: failures pile up at 23:5x and 00:0x, or on the 30th and 31st of the month, or on March 9th and November 2nd. This is one reason flaky test detection needs occurrence history, not just a pass/fail ratio. When BuildPulse shows you every disruption a test has caused with when it happened, a midnight-clustered test stops looking like noise and starts looking like a bug with an alibi.

The five clock bugs I keep finding

After enough of these postmortems, the same patterns recur:

• Boundary crossings. The test computes "now" twice: once in setup, once in an assertion. If midnight (or the month or year boundary) falls between those two reads, isSameDay(createdAt, now) is suddenly false. Any test slower than zero milliseconds is exposed.
• Duration math near boundaries. "Created within the last 24 hours" implemented as a date comparison instead of a timestamp delta. Works all day, lies at 00:01.
• Timezone mismatches. The dev laptop is in America/New_York, the CI runner is in UTC, and the code under test formats dates in the process-local zone. The test passes locally and fails in CI between 19:00 and 00:00 Eastern, which the team reads as "CI is haunted in the evenings."
• DST transitions. Adding "one day" as 86,400 seconds lands you an hour off twice a year. Tests that assert on formatted local times fail exactly two mornings a year, then pass for six months. Nobody debugs a bug with a six-month reproduction cycle.
• Real sleeps in TTL and expiry tests. The test creates a token with a 2-second TTL, does some work, and asserts the token is still valid. On a loaded runner the "some work" takes 2.4 seconds. This one isn't calendar-shaped, but it's the same disease: the test depends on the actual passage of time.

None of these are exotic. All of them will pass a code review where nobody asks "who owns the clock in this test?"

A concrete failure: the midnight boundary

Here's a distilled version of a real one, in Jest:

test("new signups appear in today's report", async () => {
  const user = await createUser(); // sets createdAt = new Date()

  // ...a few hundred ms of setup, seeding, HTTP round trips...

  const report = await getDailyReport(new Date());
  expect(report.signups).toContainEqual(
    expect.objectContaining({ id: user.id })
  );
});

If the suite starts at 23:59:58 UTC, createUser runs before midnight and getDailyReport runs after it. The user was created "yesterday." The test fails, the rerun passes, and the failure rate is low enough that it survives for a year.

There's a bonus failure mode hiding in here too: if createdAt goes through a database column with second-level precision, or an ORM that truncates milliseconds, equality assertions against the in-memory Date will flake independently of midnight. Timestamp precision mismatches between your language runtime and your database are their own reliable little flake factory.

Fix it by owning the clock

The fix is never "widen the assertion window." Tolerating a bigger delta is the temporal equivalent of adding a retry: you've made the test agree to be lied to slightly more. The fix is making the test the sole owner of time.

In Jest, freeze it:

beforeEach(() => {
  jest.useFakeTimers({ now: new Date("2025-06-15T10:00:00Z") });
});

afterEach(() => {
  jest.useRealTimers();
});

test("new signups appear in today's report", async () => {
  const user = await createUser();
  const report = await getDailyReport(new Date()); // deterministic "now"
  expect(report.signups).toContainEqual(
    expect.objectContaining({ id: user.id })
  );
});

In Python, freezegun does the same job:

from freezegun import freeze_time

@freeze_time("2025-06-15 10:00:00")
def test_new_signups_appear_in_todays_report():
    user = create_user()
    report = get_daily_report(datetime.utcnow())
    assert user.id in [s.id for s in report.signups]

And critically: pick a boring frozen time on purpose, then write the interesting cases explicitly. 23:59:59 on December 31st. The spring-forward instant in your gnarliest supported timezone. February 29th. These become fast, deterministic unit tests instead of annual production incidents.

The deeper fix is architectural. If your domain code calls Date.now() or datetime.now() directly, every test of that code is hostage to the wall clock. Inject a clock interface instead, the same way you'd inject a database connection. It's a one-day refactor in most codebases and it converts an entire class of CI flakiness into compile-time-visible dependencies. An ESLint rule banning bare Date.now() outside the clock module makes the refactor stick.

Pin the timezone in CI, then break it on purpose

Two pipeline-level moves are worth making regardless of framework.

First, pin the runner timezone explicitly so "works on my machine" and "works in CI" at least disagree deterministically:

jobs:
  test:
    runs-on: ubuntu-latest
    env:
      TZ: UTC
    steps:
      - uses: actions/checkout@v4
      - run: npm ci && npm test

Second, ambush your own suite. Clock bugs cluster at boundaries, so schedule a run that straddles one:

on:
  schedule:
    # Start just before midnight UTC so the suite runs across the boundary
    - cron: "55 23 * * *"
    # And on the last days of the month, when date math gets creative
    - cron: "55 23 28-31 * *"

A nightly job that deliberately runs across midnight will surface in weeks what organic traffic would take a year to find. Route its failures into the same test detection pipeline as your PR builds so the occurrence history accumulates in one place. If you're already tracking flaky tests in BuildPulse, these scheduled runs are cheap fuel for detection, and quarantining anything they catch keeps the blast radius off your developers' PRs while the fix is in flight (more on why quarantine beats retry in our quarantining guide).

The audit dimension: a rerun is not evidence

If you're a SOC2 or ISO 27001 shop, your CI gate is probably named in a change-management control: changes to production require passing automated tests. That sentence in your control description has an uncomfortable implication for the midnight flake.

When a protected-branch build fails and someone reruns it to green with no root cause recorded, you've created a small gap between what the control says and what actually happened. The test signaled a failure. The organization's documented response was, in effect, "we asked again until we got the answer we wanted." One instance is nothing. A culture of it is the kind of pattern that makes an auditor start pulling threads, and makes you unable to answer the more important internal question: was that failure a flake, or the first sighting of a real defect?

Clock flakes are actually the easy case to handle well, because the evidence trail is so clean. A failure timestamped 00:02 UTC, a detection record showing three prior occurrences all within minutes of midnight, a quarantine event linked to a ticket, a fix that freezes the clock, and a re-enable once the occurrence history goes quiet. That's a defensible, documented exception-handling story. "Passed on retry" is not a story; it's an absence of one. The difference matters to your auditors, and it matters more to your engineers' trust in the CI signal, which is the asset all of this is actually protecting.

A short policy that actually sticks

You don't need a task force. You need four defaults:

• Freeze time in the test harness by default. Real clocks are opt-in, and opting in requires a comment explaining why.
• Ban direct clock reads in domain code via lint rule; inject a clock instead.
• Run a scheduled suite across the midnight UTC boundary and on days 28–31 of the month, feeding the same detection pipeline as everything else.
• Treat any failure clustered near a time boundary as a real bug with an owner and a ticket, never as a rerun candidate.

The payoff is disproportionate. Clock bugs are a small fraction of your flaky tests by count, but they're overrepresented in the failures that page someone at midnight, stall a Friday-evening release, or show up in the one build an auditor happens to sample. Fix the clock once, and that entire genre of 00:03 UTC mystery goes away. Your on-call rotation will not send a thank-you card, but only because they'll never know what stopped happening.