Introduction to Visual Regression Testing

Published on Jul 13, 2016

Ever make a tiny css change, push it to production, and then discover that it completely wrecks another page's style? Good CSS discipline can reduce that problem, but things still slip through... and it's expensive to hire someone to check every page for visual errors.

Percy lets you put that visual check into your test suite. Discover how.

Also, this is an experiment- co-hosting a screencast with the author on an addon. Let me know what you think. And if you want your addon featured on EmberScreencasts- and you have 2-4 hours to spare- email me at jeffrey@emberscreencasts.com.


Links

Code

$ ember install ember-percy
//tests/helpers/module-for-acceptance.js
import '../helpers/percy/register-helpers'
//tests/.jshintrc
{
  "predef": [
    //...
    'percySnapshot'
  ]
  //...
}
//mirage/config.js
this.passthrough('/_percy/**');
//tests/acceptance/monsters-index-test.js
//...

test('visiting /monsters', function(assert){
  server.createList('monster', 5);
  visit('/monsters');
  percySnapshot('monsters-index');
});

Transcript

Jeffrey: Hey and welcome to Ember Screencasts. Today we’re going to be going over Percy, which is a really cool way to keep track of changes in your Ember app. Today I have with me Mike Fotinakis. Is that how you pronounce your name?

Mike: Yeah, that’s right.

Jeffrey: Awesome. So he’s the creator of percy.io, and he’s going to be showing us some of the really cool features of it and how we can set it up with Ember. Could you give a really quick explanation in your words what Percy is?

Mike: Hey, thanks for having me. So Percy is continuous visual integration for web apps. So basically that means that it lets you test apps pixel by pixel. So this kind of gives you, like we’re showing here, it kind of gives you a visual review of your site so you can see for every PR like these exact pixels have changed. So we do things like integrate with GitHub, we have responsive testing. It basically gives you like a visual review process right alongside your code review process. So today we’ll talk about how we actually can integrate that into Ember apps.

Jeffrey: Yeah. So if you have like a css style that changes and it accidentally changes something else, Percy will tell you. So our first step in getting started is to sign in with GitHub. And you probably already have a GitHub, so easy first step. Then, you’re going to have to get it set up with your continuous integration server. Yes, this is going to require you have tests and continuous integration. So if you don’t have those, then get those set up and then go ahead and come back.

So we’re going to be using Travis CI since it’s free for open source projects. And what you have to do is you have to get your Percy token. So it’s going to come in... you hit ‘My Repos’ and then you go to the specific repo, and then you’ll be able to find your Percy token. So you put that in there then you put in your Percy project which is basically your GitHub identifier for the project. Here it’ll be jeffreybiles, that’s my GitHub username, then crud-2016.

So now we’ll actually go ahead and install the Ember Percy addon. Percy isn’t limited to Ember, but since Mike programs in Ember quite a bit, then he’s created this addon so we can get to it really easily.

Mike: And Percy itself is actually an Ember app as well, so this also allows me to test Percy using Percy.

Jeffrey: Nice. And so you also have a Rails plugin. What other plugins do you have?

Mike: We support static websites, Rails sites with Capybara, we recently have a Python client [02:50] and an Ember test as well.

Jeffrey: Awesome. Alright, now that we’ve installed ember-percy, there’s a little bit of setup that we have to do. So let’s go to the ember-percy GitHub page, and in the readme it’ll show us that we have to add this import statement to our module-for-acceptance file. So this will get us the helper that takes a snapshot with Percy.

Next we’ll need to configure JSHint, so we’ll need to add percySnapshot to the predef section. And it says to add it to the one at... in the test folder, but I find it works better if I also add it to the one that is in the main project folder.

So if you’re not using Mirage, you’re completely set up. But if you are using Mirage like us, you’ll need to... let’s scroll down to the bottom to the troubleshooting section, and we’ll need to add this.passthrough. And you’re going to need to add it to... well, let’s go to our files. So in Mirage, in the config, you’ll need to add it, and you’ll need to add it before this.namespace. This is very important, otherwise it will try to interpret it as that when we really want that.

Alright, now we’re completely set up. It’s time to use Percy. So we go to our acceptance test, and after we visited the page, we can call percySnapshot, and let’s go ahead and call it monsters-index. So it takes one argument, and that argument is the name of the snapshot. So this is an async helper, so it can go top level alongside visit and click and stuff like that. So let’s take the snapshot, and then to make the magic happen, we’re going to have to push it up to GitHub, and then Travis will notice it and through that Percy will notice it.

Now let’s go to GitHub, to our repository, and let’s make a PR. So we’ll go ahead and make this PR, put in a comment of ‘Visual diffs!’, and now check out what Percy does for us. We have our Travis CI and then below that we have our Percy checks. And we’ll go ahead and hit the details button. This will take us to our Percy page, and we can check out our snapshot. We’ll see that hey, this is what showed up. That’s pretty good, and nothing has messed up with it, so we’ll go ahead and hit approve. And then back here in our checks, it’ll say that it’s been approved by me. So Percy does a lot of stuff for you automatically, but there’s still a human giving a thumbs up.

So we just showed what happens when things go right. Now let’s show you what happens when things go wrong and Percy saves you. So let’s go ahead and add some exclamation points, and let’s say we had a lot of other changes in here and we forgot to take these exclamation points out. So let’s add those and yes, if you have an exclamation point, it messes up with the quotes. So we’ll go ahead and wait for Travis and Percy to finish. You can see Percy just popped up there, it’s starting, and this takes a couple of minutes in practice. It’s going to be twenty seconds in the screencast. And look, it says ‘Visual diffs are found’. So let’s hit details and see what it has to show us. And look, it’s highlighting all of these exclamation points. So we can click it to see the regular view, then click it again to see highlighted in red all the things that have changed. And on our left we have what it was before. So we can either approve or not approve. In this case we decide, you know what, we like that everyone is so excited about these monsters, so we’ll hit approve. And is there a button to hit in don’t approve?

Mike: Not yet. Feature request.

Jeffrey: Alright. So maybe if you’re watching this in the future, but for now we’ll approve it. And now when we go back to the GitHub PR, it shows all the checks as having passed.

So there is one more thing before we go. Usually in these screencasts I do all free stuff, but Percy, it does cost some money. I think that if you’re working on something that’s... you know, it’s important that it stays looking the same, then it’s worth it. And also if you’re just working on something small, you can get 500 visual diffs a month free. So keep that in mind when you’re making your decision. Alright, anything else you want to say, Mike?

Mike: No. Just thanks for having me.

Jeffrey: And thank you for being our first guest. If you’re the creator of an Ember addon and you have two to four hours to come record with me, go ahead and send me an email and we’ll see if we can feature your addon. So for the addon authors, I’ll see you in my email. That’s jeffrey@emberscreencasts.com, and for everyone else I’ll see you next time.

Subscribe to our mailing list