January 12, 2021

Catch Bugs Faster, and Deploy with Confidence — Visual Regression Testing Just Got Better

Dachary Carey

Tugboat’s visual regression testing feature, visual diffs, is an important part of the workflow for many of our clients. Not only is visual regression testing is additional line of defense against bugs making it through to production, but providing this functionality also enables teams to develop and deploy faster, with higher confidence that each new pull request isn’t introducing breaking changes. With that in mind, we’ve super-charged this functionality to make visual diffs even more powerful.

Make your visual regression testing more meaningful

Do you have pages where content changes regularly? Maybe you’ve got an ad that displays different images, or a slide-in feature product or content carousel that may display something different based on personalization. In those cases, you’d expect to see some difference in the page, so a 100% match in visual diff screenshots isn’t a requirement.

With visual diff thresholds, you can tell Tugboat to set a pass/fail status based on how similar your pull request Preview (the After screenshot) is to your Base Preview (the Before screenshot). If you’re A/B testing different headers, you might only expect 90% similarity between pages. If you’ve got a large image or section that might change, you may only need a 70% similarity between images. You can define these thresholds on a per-URL basis, and even a per-breakpoint basis, to make your visual regression testing more meaningful. This way, you’ll know that a failed visual diff is really a fail - not just an expected change to your page.

A screenshot of the visual diffs listing page from the Tugboat dashboard, highlighting two failed regression tests.

Communicate more effectively with developers in GitHub, GitLab, or Bitbucket

When you use visual diff thresholds, you get the added benefit of communicating more effectively with developers in the places they work - by automating visual regression tests against pull requests in git repositories. With thresholds enabled, Tugboat only sends a “Pass” status to GitHub, GitLab, or Bitbucket if your visual diff meets or exceeds your similarity threshold. If it doesn’t, Tugboat sends a “Fail” status to the linked git provider - and the pull request can’t be merged without an override.

A screenshot of a pull request on GitHub, highlighting a failed visual regression test result from Tugboat.

This added step helps communicate to team members to check the visual regression tests and fix issues that are causing the pull request to fail. Once these visual issues are corrected - or someone reviews and approves the visual diff - the status changes to “Pass” and the pull request can be merged. This status reporting makes it easy to add a visual regression review process to your workflow.

A screenshot of a pull request on GitHub, highlighting a passed visual regression test result from Tugboat.

Add a visual regression review process to your workflow

What happens when you see a “Fail” status on a Tugboat visual regression test? Click into the Details to open the Tugboat Dashboard and view all of the visual diffs for the pull request. This at-a-glance-view makes it easy to spot visual diffs that aren’t passing the threshold. Team members can click into Desktop, Tablet, or Mobile breakpoints, review the visual diffs, and manually change the status to “Approve” if the changes are expected and the pull request shouldn’t be blocked.

A screenshot of the visual diffs listing page from the Tugboat dashboard, highlighting the ability to Approve a previously failed visual regression test.

Alternatively, reviewers can manually “Decline” a visual diff that meets the threshold if they spot an issue during the review process.

Make visual regression testing and review an easy part of your workflow with these new features, and enjoy higher confidence in deploying bug-free code to production.

Want to learn more? Get in touch or get started here.

Need More Info on Tugboat?

Tell us about your project, schedule a demo, or consult with a Tugboat Technical Account Executive.

Your request has been sent.
Oops! Something went wrong while submitting the form.