BuildPulse

Stop losing time to
flaky tests

Automatically detect flaky tests. Focus on the top offenders to quickly improve your builds. Save developer time and frustration. Regain trust in your test suite.

Try BuildPulse for free

Install via the GitHub Marketplace

Trusted by teams that care about
developer flow reliable tests productivity delivering quality

  • Replit
  • Homebrew
  • RE/MAX
  • InVision
  • Codecademy
  • Culture Biosciences

Automatically track flaky tests

Invest time eliminating flaky tests, not manually filing issues for them. BuildPulse gives you an up-to-date list of flaky tests for each repository.

List of repository's flaky tests

Get a head start on investigation

Get the details you need for each flaky test. Quickly assess its impact. Click directly through to the source code. Go deeper with one-click access to the build logs.

All the essentials. All in one place. All automatically assembled.

Details for an individual flaky test, including flakiness percentage, number of branches disrupted, and direct links to the build and the underlying code
List of recently-built repositories with bar charts showing rate of flaky test over time

Keep an eye on health across all your projects

Identify trends with just a glance.

Spot the projects that need attention. Drill down to see what’s failing and get things back on track.

Quantify impact with metrics

Don’t just wonder whether things are getting better: stay up to date with the latest statistics delivered directly to your inbox each week.

See the percentage of commits being disrupted by flaky tests, the week-over-week change, and a summary of the biggest problem areas.

Details for an individual flaky test, including flakiness percentage, number of branches disrupted, and direct links to the build and the underlying code

Testimonials

  • Michael Menne, CTO, Human Agency

    Flaky tests were a major roadblock delaying our deploys, but BuildPulse helped us get things back on track.

    I especially appreciate getting prompt support directly from the founder.

    Michael Menne

    CTO, Human Agency

  • Josh Goldberg, Lead Engineer at Codecademy

    Within a few days of installing BuildPulse, it had identified our flakiest tests and helped us quantify their impact.

    We quickly prioritized dealing with those top offenders and our build stability shot up from 78% to 93%.

    Josh Goldberg

    Lead Engineer, Codecademy

  • Rob Sanhiem, VP of Technology, first.io

    If the problem is rare enough (i.e., flakes every week or so), fix the low hanging fruit and/or retry with the tougher ones. Once it gets worse (and it always does), it's time for real tooling around it, which means BuildPulse.

    Rob Sanhiem

    VP of Technology, first.io

Works with the
CI you already have

BuildPulse integrates with a variety of CI services and test frameworks to make setup a breeze.

  • CircleCI
  • Travis CI
  • GitHub
  • Semaphore
  • Jenkins

    available in early access

  • Buildkite

    available in early access

Don't see your CI service? Get in touch and we may be able to quickly add support for it.

Plugs into your
existing test framework

Developers are using BuildPulse with JavaScript, Python, Ruby, and just about every other modern programming language.

Our example projects show how to integrate BuildPulse with popular test frameworks/libraries.

Want to see an example for your favorite test framework? Let us know, and we'll get back to you right away.

Frequently asked questions

What is a flaky test?

A test is flaky when it produces different results without any changes to the code. Typically this is seen when a test fails and then a developer re-runs the CI build, and the test then passes. You'll also hear this behavior described as intermittent failures, random failures, or nondeterministic tests.

Why should I care about flaky tests?

To quote Martin Fowler, flaky tests are "a virulent infection that can completely ruin your entire test suite."

Specifically, flaky tests slow everything down: it takes longer to get a passing build, longer to merge a pull request, and longer to get code deployed.

Left to their own devices, flaky tests become a major roadblock to productivity, and they destroy trust in your test suite. And when you can't trust your test results, you don't know whether a failure is due to a misleading flaky test or a legitimate bug.

How do I set up BuildPulse?

After starting your free trial via the GitHub Marketplace, you'll get a few lines of configuration to add to your CI build script. Don't worry: we offer step-by-step guides for each CI service along with example code for popular test frameworks. And if you like, we even provide concierge onboarding to help you get set up.