First-class End-to-end Testing on Meteor

I've been a big fan of meteor since its beginnings. Since the graphql trend, it has been kind of forgotten and I wanted to check if it was still awesome. Spoiler: I think it is.

A toy project came to my mind and it was the right fit to give meteor a try again: it needed reactive data. Prototyping is still very fast and the developer experience is awesome, it makes everything a breeze. Except when you want end-to-end (e2e) tests, just like for my toy project.

The last time I've used e2e tests was with selenium, through protractor. And to be honest, it was a real nightmare. Since then, I saw my colleagues use cypress and it seemed a lot better, I got really jealous not to have the occasion to use it.

Here was the challenge: setup first-class e2e testing on the meteor app.

Install Cypress

I followed a simple tutorial to set up cypress on meteor.

Quick version:

• Install cypress with yarn

meteor yarn add -D cypress

• Move cypress tests outside of the folders meteor compiles

mv cypress/ tests/cypress

• Adapt the cypress.json configuration to meteor

{
  "fixturesFolder": "tests/cypress/fixtures",
  "integrationFolder": "tests/cypress/integration",
  "pluginsFile": "tests/cypress/plugins/index.js",
  "screenshotsFolder": "tests/cypress/screenshots",
  "supportFile": "tests/cypress/support/index.js",
  "videosFolder": "tests/cypress/videos",
  "baseUrl": "http://localhost:3000"
}

• Run tests

// package.json
"scripts": {
    "test": "cypress open"
},

meteor yarn test

Create the First Test

Let's create the first test to assert that cypress works. First, remove all the example tests cypress comes with:

rm -Rf tests/cypress/integration/examples

Then, test if users can register:

describe("sign-up", () => {
  beforeEach(() => {
    cy.visit("http://localhost:3000/");
  });

  it("should create and log the new user", () => {
    cy.contains("Register").click();
    cy.get("input#at-field-email").type("jean-peter.mac.calloway@gmail.com");
    cy.get("input#at-field-password").type("awesome-password");
    cy.get("input#at-field-password_again").type("awesome-password");
    // I added a name field on meteor user accounts system
    cy.get("input#at-field-name").type("Jean-Peter");
    cy.get("button#at-btn").click();

    cy.url().should("eq", "http://localhost:3000/board");

    cy.window().then(win => {
      // this allows accessing the window object within the browser
      const user = win.Meteor.user();
      expect(user).to.exist;
      expect(user.profile.name).to.equal("Jean-Peter");
      expect(user.emails[0].address).to.equal(
        "jean-peter.mac.calloway@gmail.com"
      );
    });
  });
});

If everything went fine, the test should pass.

You still have to launch meteor manually (meteor run), and run test on another terminal (meteor yarn test). I would like everything to happen automagically with a single command so that I can integrate it into a continuous integration server without too much hassle.

Run and Stop Meteor Automagically

At first, I thought a script like the following one would be enough:

meteor &
meteor yarn test
killall meteor

But as you can guess, meteor takes some time to boot up and isn't ready when the tests start.

I then stumbled upon a github repository integrating meteor and cypress with circle CI, using start-server-and-test.

As its name suggests, this package starts the server, and launches the tests; except that it waits until a specified address is available before starting the tests.

• Install start-server-and-test

meteor yarn add -D start-server-and-test

• Adapt package.json

"scripts": {
    "start-server": "meteor",
    "cypress:open": "cypress open",
    "test": "start-server-and-test start-server http://localhost:3000 cypress:open",
},

• Run it

meteor yarn test

• It should display this kind of output

yarn run v1.13.0
$ start-server-and-test start-server http://localhost:3000 cypress:open
starting server using command "npm run start-server"
and when url "http://localhost:3000" is responding
running tests using command "cypress:open"

> toy-project@ start-server /toy-project-path
> meteor

[[[[[ ~/Projects/toy-project ]]]]]

=> Started proxy.
=> Started MongoDB
=> Started your app.

=> App running at: http://localhost:3000/

> toy-project@ cypress:open /toy-project-path
> cypress open

Cypress effectively launches only when meteor is ready.

Run Tests on a Production Build

Tests are supposed to be as close to production as possible to detect defects earlier. It would be a lot better to test the meteor production build output instead of just the development mode.

Meteor has us covered with the --production flag.

Add the flag to the package.json script:

"scripts": {
    "start-server": "meteor --production",
    "cypress:open": "cypress open",
    "test": "start-server-and-test start-server http://localhost:3000 cypress:open",
},

Warning: The --production flag should only be used to simulate production bundling for testing
purposes. Use meteor build to create a bundle for production deployment.
See: https://guide.meteor.com/deployment.html

Be Able to Reset Database Between Tests

Everything seems to work fine, but when launching the tests for the second time, they don't pass:

And since the cypress GUI is awesome, I can see the the error I got and inspect the page at this stage:

Indeed, the user has already been created the last time the tests ran, and the database didn't change in-between. Since meteor recreates any missing database or collection, the easiest way is to just drop the whole database every time it's needed.

Cypress has us covered on that part, with custom commands:

• Create a custom command that drops the database

// tests/cypress/support/commands.js
Cypress.Commands.add("resetDatabase", () =>
  cy.exec('mongo mongodb://localhost:3001/meteor --eval "db.dropDatabase()"')
);

• Use it anywhere it's needed

describe('sign-up', () => {
    before(() => {
        cy.resetDatabase();
    });
    //...

// tests/cypress/support/commands.js
Cypress.Commands.add("executeDatabaseScript", filePath =>
  cy.exec(`mongo mongodb://localhost:3001/meteor ${filePath}`)
);

Have a Dedicated Database

Since everything ran perfectly, I went back to writing code. I launched the app but couldn't log in anymore, that's when I realized that the tests had dropped the development database.

Indeed, tests need to run on their own database. But I would like to keep the meteor internal database because it's actually very handy, the CI already has meteor, and I don't want to bother using docker-compose.

I thought that starting tests with MONGO_URL=mongodb://localhost:3001/test would be enough. Actually, meteor doesn't launch its internal mongo instance when you give it a MONGO_URL. Fair enough, but then how to "change" the database name?

We actually need to launch a whole new instance of meteor. To do so, meteor has an undocumented (only appears on 1.3.3 changelog) env var METEOR_LOCAL_DIR which specifies the directory from which meteor launches everything. By default, this value is .meteor/local, but if changed to .meteor/test, the test instance will be completely isolated from the development instance, including the database.

It happens in the package.json:

"scripts": {
    "start-server": "METEOR_LOCAL_DIR=.meteor/test meteor --production",
    "cypress:open": "cypress open",
    "test": "start-server-and-test start-server http://localhost:3000 cypress:open",
},

Do Not Launch Cypress GUI on CI

While Cypress GUI is very handy when writing tests, launching them on a CI requires to run all tests headlessly. Makefile is a very good tool for that:

• Add a new script on package.json

"scripts": {
    "start-server": "METEOR_LOCAL_DIR=.meteor/test meteor --production",
    "cypress:run": "cypress run --browser chrome",
    "cypress:open": "cypress open",
    "test": "start-server-and-test start-server http://localhost:3000 cypress:open",
    "ci": "start-server-and-test start-server http://localhost:3000 cypress:run"
},

• Use a makefile target

test: ## Launch tests
ifeq ($(CI),true)
	meteor yarn ci
else
	meteor yarn test
endif

• Launch tests

# On development machine
make test

# On CI
CI=true make test

Conclusion

Meteor still provides a really awesome experience.

What I liked the most back then is still there: An all-inclusive build tool, free reactive data, free optimistic rendering, code sharing between the client and the server, all-inclusive authentication packages...

What I liked less: Templates rendering with Blaze and meteor package management. They could both be replaced by better known alternatives like React and yarn.

The only thing that's not included is a full-featured e2e tests framework. The documentation is scarce, the meteor build tool is not very flexible, and I really struggled setting up cypress. But it works, and cypress is the ideal fit for meteor, it shares the same values of making the developers' lifes easier.

With this tutorial, you can now set up top-notch e2e testing with cypress without the hassle of setting up anything else than meteor on your CI server. No more excuse for building an app without e2e tests!