Stop losing time to
flaky tests

Automatically detect flaky tests. Focus on the top offenders to quickly improve your builds. Save developer time and frustration. Regain trust in your test suite.

Already installed BuildPulse?
Sign in with GitHub.

Automatically track flaky tests

Invest time eliminating flaky tests, not manually filing issues for them. BuildPulse gives you an up-to-date list of flaky tests for each repository.

List of repository's flaky tests

Keep an eye on health across all your projects

Identify trends with just a glance.

Spot the projects that need attention. Drill down to see what’s failing and get things back on track.

List of recently-built repositories with bar charts showing rate of flaky test over time

Get a head start on investigation

Assess the impact of each flaky test. Click directly through to the source code for the test. Go deeper with one-click access to the build logs.

BuildPulse gives you the details you need, right at your fingertips.

Details for an individual flaky test, including flakiness percentage, number of branches disrupted, and direct links to the build and the underlying code

Quantify impact with metrics

Don’t just wonder whether things are getting better: stay up to date with the latest statistics delivered directly to your inbox each week.

See the percentage of commits being disrupted by flaky tests, the week-over-week change, and a summary of the biggest problem areas.

Email message showing stability percentage and flakiness percentage, week-over-week improvement or degradation, and flakiest tests for the past week

Testimonial from Codecademy

Within a few days of installing BuildPulse, it had identified our flakiest tests and helped us quantify their impact. We quickly prioritized dealing with those top offenders and our build stability shot up from 78% to 93%.

If your team is losing time to flaky tests like ours was, BuildPulse can quickly help you get them under control.

Josh Goldberg, Lead Engineer at Codecademy
Josh Goldberg
Lead Engineer, Codecademy

Works with the CI you already have

BuildPulse integrates with a variety of CI services, and we're adding support for others soon. Get in touch to tell us which service you'd like to see us support next! (Azure Pipelines, AppVeyor, etc.)

Jenkins

available in early access

Buildkite

available in early access

Logos of several programming languages supported in BuildPulse: Clojure, Go, Groovy, Java, JavaScript, Kotlin, PHP, Python, Ruby, TypeScript

Plugs into your existing test framework

Developers are using BuildPulse with Cypress, Go, JUnit (Java), minitest (Ruby), PHPUnit, pytest (Python), RSpec, TestCafe, and TestNG (Java).

Whatever test framework you're using, we probably already support it. And if we don't support it yet, we want to! Let us know which test framework you'd like to use with BuildPulse, and we'll get back to you right away.

Frequently asked questions

What is a flaky test?

A test is flaky when it produces different results without any changes to the code. Typically this is seen when a test fails and then a developer re-runs the CI build, and the test then passes. You'll also hear this behavior described as intermittent failures, random failures, or nondeterministic tests.

Why should I care about flaky tests?

To quote Martin Fowler, flaky tests are "a virulent infection that can completely ruin your entire test suite."

Specifically, flaky tests slow everything down: it takes longer to get a passing build, longer to merge a pull request, and longer to get code deployed.

Left to their own devices, flaky tests become a major roadblock to productivity, and they destroy trust in your test suite. And when you can't trust your test results, you don't know whether a failure is due to a misleading flaky test or a legitimate bug.

How do I set up BuildPulse?

After starting your free trial via the GitHub Marketplace, you'll get a few lines of configuration to add to your CI build script. Don't worry: we offer step-by-step guides for each CI service along with example code for popular test frameworks. And if you like, we even provide concierge onboarding to help you get set up.

Ready to get your productivity back?
Start your free trial today.