Recently I set up a Rails app with Cypress end-to-end specs on CircleCI. Capybara's the Rails default for this, but Cypress is better. It's faster, more stable, and its in-browser spec runner is a much better experience than writing specs in Capybara. However, Cypress support is still a bit immature, so you have to do some manual tweaking.
Let Capybara Wander Off To Nibble Grass
In 2016, I argued that Rails projects should abandon the asset pipeline in favor of Webpack. The asset pipeline's still supported for legacy reasons, but Rails did begin integrating Webpack a few months later. Capybara's in the same position; even though Rails is still great overall, a Node.js tool has gotten so much better than its Ruby equivalent that you should just use the Node option at this point. I said "let the asset pipeline die" back in the day, but that feels too strident, so I'm recommending instead that we let Capybara wander off to nibble grass.
Capybaras are pretty chill, after all.
With Cypress, you run your specs in one half of a browser window and Cypress executes each step of your spec in the other half while you watch.
Here's what the Cypress interactive runner looks like:
By contast, writing specs in Capybara is mostly about imagining your web application's behavior in your head and then writing it down. The superiority is so decisive that it feels awkward to even describe it.
Gems
Unfortunately, setting up Cypress to test a Rails app is also a little awkward, both in real life and on Circle CI. Cypress itself is still a relatively new project, with some rough edges that haven't been sanded down yet, and the Rails community is still mostly stuck on Capybara. I've only found two gems for Cypress, and to make matters worse, I basically had to use both.
These are the gems:
Both these gems come from dev shops.
The cypress-rails gem comes from Test Double. I needed it because it launches Cypress from a Rake task while also running a Rails server. Cypress tests against a running server. In your development environment, you can just start the Rails server and then open up a new Terminal or iTerm tab to start up the Cypress process, but you want something more integrated when you're running your specs on CircleCI.
The cypress-on-rails gem comes from Shakacode. I needed this gem as well because it includes an integration with FactoryBot, the very popular — and essentially default — factories gem for specs and tests.
Before we go any further, let me give a big shoutout to both Test Double and Shakacode for making my life easier with their hard work.
I also want to say that you might not need both gems. My team had already been writing specs with the cypress-on-rails FactoryBot integration, and wanted to keep doing it. If you're not into factories and the database dependency issues they sometimes bring, the Test Double gem gives you a system of event hooks for loading and reloading test data.
Anyway, using each of these gems is very easy, and they play well with each other, but I still had to get into some hackery. If I'm wrong, of course, I would love to hear about it. Unnecessary hackery is the best kind of hackery, since it's easy to delete.
But here's why the hackery seemed necessary.
(Fun fact: a hackery means a bull cart in India.)
Config Files
Cypress assumes that its config file cypress.json will sit at the root of your project. But it's cleaner conceptually if you have it sitting inside spec or even spec/cypress. It's also essentially guaranteed that you're going to want different configurations in development and on your continuous integration server.
Compare this idea of one top-level cypress.json file with the Rails approach to environment-specific configuration files. You put them in a directory called config/environments and you give them names like development.rb and production.rb. This is a lot more practical, especially since Cypress will, by default, take screenshots and record videos of its test runner, which just burns resources for no good reason when you're running it on a CI server. You can disable this default, of course, but if your project doesn't have environment-specific configuration files, then disabling this feature in one environment means disabling it globally. You might still want it in other environments.
So it's probably obvious what happened next. I created a config/cypress dir and populated it with two files named development.json and circleci.json. So far, so good. Pretty clean. But next I had to tell Cypress to use these config files, and that got ugly. I kind of made it worse by deciding to support running Cypress through both Rails and Yarn.
First, our front-end developer had already set up cypress.json in spec/, and was used to running Cypress via Yarn, and I wanted to keep that option open for them. In fact, this proved unnecessary; they immediately suggested getting rid of the Yarn runner. But we couldn't really do that.
We had one flaky spec which worked fine if you ran it on the command line through Rake, using bundle exec rails cypress:run. It also worked if you ran it in the interactive runner through Yarn, using yarn cypress:open. And if you ran it headless via Yarn instead of via Rails, it still worked just fine. But it failed for unclear reasons if you ran it in the interactive runner through Rails, using bundle exec rails cypress:open. So we settled for keeping all options open, partly because we needed all options open in order to debug the issue, and mostly because we realized we could be in this situation again.
Really, you should never give up the ability to get closer to the wire, or to run a system in the most native way possible. Cypress is from the JavaScript ecosystem, so keeping a JavaScript runner available is a good idea.
Anyway, in practice this meant that I wanted to run development.json through Yarn and circleci.json through Rails. The good news is that Cypress allows you to specify a config file's location, and the bad news is that the support is a little brittle. Plus, for the CircleCI use case, I needed to have the Rake task pass the config file's location through to Cypress.
Command-Line Arguments
The line that made the magic happen for Yarn looked like this:
This went into package.json as cypress:open inside the "scripts" object.
This command first passes the project location to Cypress, since Cypress wants everything to live in a top-level cypress directory (except for the cypress.json file, which it also wants to have at the project root). Next the --config-file command-line argument identifies the location of the config file, which is pretty obvious, but it took a while to get this part right. Cypress actually wants its config file path option to be a relative path which is not relative to the directory that you run the command from, but instead relative to the directory that the project is in. That's why it's ../config instead of ./config or even config.
This was weird and agonizing. Another nasty pitfall: it will only work if you put --project before --config-file. Putting --config-file before --project may have cost me an hour or more of debugging time. With the options in the wrong order, Cypress doesn't just fail to find its config file; it also then decides to create a new default config file to solve the problem. It's a great tool overall, but this design decision is a little ridiculous.
Anyway, at this point I basically just had to run the same command for a different config file, but this time, I had to pass the command-line arguments to Cypress while going through the cypress-rails Rake task. cypress-rails passes information through to its Rake task through environment variables, so I had to add a new line to .env (and to .env.example, since you don't put .env itself in version control). Here's the new line:
I also had to set this variable's value on CircleCI, using the env vars UI there.
With that out of the way, the rest was easy. I only had to add a couple lines to .circleci/config.yml. First I set it to use the circleci/ruby:2.6.6-node-browsers Docker image. Next I just had it use the Rake task:
Worth It
Now we have end-to-end Cypress specs running on CircleCI. They're faster to write than Capybara specs, and much easier to update, which means we're probably going to use them more. They also seem more stable so far, although I wouldn't say either solution is 100% free of flakiness.
If you're a Capybara diehard, the good news is that Cypress still has some immature design decisions to clean up, but really, this is a pretty cut-and-dry situation. It's obviously early days for Cypress support in Rails, but in the same way that Webpack is gradually replacing the asset pipeline, Cypress is just a better tool than Capybara, so it's reasonable to expect that Rails itself will eventually switch. (The community, certainly, and probably the framework as well.) When this happens, Rails support will get smoother.
In the meantime, setting it up was pretty easy. The framework has to move slower than the community, because the framework has to support legacy applications or at least minimize the pain in the upgrade path, but for greenfield applications, you can start using Cypress in your Rails apps right away. And if you need your specs to run in CircleCI, it's just a few lines of configuration.
