Automatically detect flaky tests. Focus on the top offenders to quickly improve your builds. Save developer time and frustration. Regain trust in your test suite.
Already installed BuildPulse?
Sign in with GitHub.
Invest time eliminating flaky tests, not manually filing issues for them.
BuildPulse gives you an up-to-date list of flaky tests for each repository.
Identify trends with just a glance.
Spot the projects that need attention. Drill down to see what’s failing and get things back on track.
Assess the impact of each flaky test. Click directly through to the source code for the test. Go deeper with one-click access to the build logs.
BuildPulse gives you the details you need, right at your fingertips.
Don’t just wonder whether things are getting better: stay up to date with the latest statistics delivered directly to your inbox each week.
See the percentage of commits being disrupted by flaky tests, the week-over-week change, and a summary of the biggest problem areas.
Within a few days of installing BuildPulse, it had identified our flakiest tests and helped us quantify their impact. We quickly prioritized dealing with those top offenders and our build stability shot up from 78% to 93%.
If your team is losing time to flaky tests like ours was, BuildPulse can quickly help you get them under control.
BuildPulse integrates with a variety of CI services, and we're adding support for others soon. Get in touch to tell us which service you'd like to see us support next! (Azure Pipelines, AppVeyor, etc.)
available in early access
available in early access
Developers are using BuildPulse with Cypress, Go, JUnit (Java), minitest (Ruby), PHPUnit, pytest (Python), RSpec, TestCafe, and TestNG (Java).
Whatever test framework you're using, we probably already support it. And if we don't support it yet, we want to! Let us know which test framework you'd like to use with BuildPulse, and we'll get back to you right away.
What is a flaky test?
A test is flaky when it produces different results without any changes to the code. Typically this is seen when a test fails and then a developer re-runs the CI build, and the test then passes. You'll also hear this behavior described as intermittent failures, random failures, or nondeterministic tests.
Why should I care about flaky tests?
To quote Martin Fowler, flaky tests are "a virulent infection that can completely ruin your entire test suite."
Specifically, flaky tests slow everything down: it takes longer to get a passing build, longer to merge a pull request, and longer to get code deployed.
Left to their own devices, flaky tests become a major roadblock to productivity, and they destroy trust in your test suite. And when you can't trust your test results, you don't know whether a failure is due to a misleading flaky test or a legitimate bug.
How do I set up BuildPulse?
After starting your free trial via the GitHub Marketplace, you'll get a few lines of configuration to add to your CI build script. Don't worry: we offer step-by-step guides for each CI service along with example code for popular test frameworks. And if you like, we even provide concierge onboarding to help you get set up.