Flexible Testing for Ember.js Apps

Testing in Ember.js can be fun, challenging and sometimes ought right confusing. After about 6 months of maintaining and running tests for our Ember-CLI cross platform desktop app, Azure Storage Explorer I learned a thing or two on how to get the most out of Ember.js testing.

First Things First

‘Flexible Testing’ means testing in a way that is reliable enough for a Continuous Integration pipeline but still provides great value for functional validation of features in your Ember application.

Although these tests don’t validate the 100% true user scenario of opening up the application, clicking on things and the application talking to some back-end service, it does bring a level of confidence that the application can do so on every commit.

The Example

I’ve been working on a small project the last few months, which is a tool for browsing contents inside an Azure Storage account, a service similar to Amazon Web Services’ S3 service. Shameless plug: you can find this tool for Linux, Mac and Windows at storageexplorer.com.

The tool allows you to browse containers, which hold blobs, (binary large objects). Each container can be configured with a variety of settings such as access control. There’s a modal dialog for this and it looks like this:

8D1nm8UdqD

The dialog above is showing the Container Properties component which implements this functionality.

It’s corresponding Handlebars template looks like this:

You might think of a few ways you might test component.

Forget About UI Tests

UI Tests are great for validating end-to-end user scenarios. They begin to lose their value in continuous test automation because of their sensitivity to the UI design including the CSS framework you decide on using.

For example, if in the future you decide to re-style this dialog, place it somewhere else in your application or just change the CSS framework you are using, you open yourself up to a ton of potential false failures. Think about it – UI tests validate that things show up in the DOM at a very particular time which may be impacted by the aforementioned factors.

Take for example this ugly piece of code we had to put into our app during a feature contribution we received for allowing the UI testing of a component. Due to the fact that materializecss had a particular time that it would animate a component before rendering it in the DOM, the UI tests will fail. If we changed how we styled this component, this piece of code may or may not work — which is pretty bad.

Unit Tests are Good but Functional Tests Are Better

Unit tests are great, but often they are far from the user experience or test such a minute part of the application, it doesn’t have much value.

In the case of this component, it interacts with another part of the app, an Ember Model called Container which is responsible for modifying properties of the container using an Ember custom adapter called ContainerAdapter. ContainerAdapter uses the Azure SDK for performing requests to the Azure API.

In this example feature there are 3 distinct components being tested:

  • ContainerProperties component
  • Container model
  • ContainerAdapter adapter

We’ll see in a bit how we can tests all these things at once.

Focus on Feature Testing

A feature test is a test which covers the entire scenario of a particular feature of an application. In our project, entire features typically come from a single pull request. So it’s safe to assume, your feature test should test everything in your pull-request.

Because Ember is so modular, you can just take all the pieces necessary for a feature and test that, rather than pulling your entire app code all at once. I’ll talk more about how you can use this.subject() to inject other components into the test subject.

The Observe-Action-Validation Pattern

The Observe-Action-Validate pattern describes a process in which I ‘fake’ the UI layer of our application and only interact with the underlying component that actions are sent to. As you already know, actions are sent from your Handlebars template, and feedback from the component/controller/route is reflected via binding to observable values on that component.

Throughout our project I found that while observers are being phased out of use within ember apps, for testing purposes they are fantastic for detecting when something interesting happens Ember Components and Controllers. We can then validate that our controller or component has reached it’s correct state no matter if the action is asynchronous or not.

Example Test

Let’s take this test example from the storage explorer project:

What’s Going on Here?

This is basically simulating the actions in the dialog shown above. It’s responsible for ensuring that the component changes its selectedAcessType property to the string literal 'CONTAINER'. This however is an asynchronous action and since actions don’t return promises, we’re not sure when this action may complete.

First we do some custom app initialization in which we set up the app for testing and then obtain an Ember Model called Container using our custom getContainer function. We use this container to create the test subject by using this.subject.

Once the subject is obtained, we can start our ‘Action-Observe’ pattern, mimicking actions by the UI to the component.

First, we send the actionAccessControlLevel action to the component, to populate the selectedAccessType property. It is expected that the accessControl level will first be BLOB.

On the first Observer callback we validate the current selectedAccessType property and send another action setAccessControlLevel with the 'CONTAINER' value.

We then assure that on the second run of the Observer handler, we see the selectedAccessType change from 'BLOB' to 'CONTAINER'.

In this test pattern, you have 3 components (besides app initialization), Observe, Action, Validate. This test has 2 validations:

Validation 1

Observe

  • selectedAccessType – Set an observer on this field to detect if and when the component changes it.

Action

  • actionAccessControlLevel – Tells the component to change the view state from it’s default to ‘AccessControlLevel` type.

Validation

  • selectedAccessType – Validates that this field is at it’s expected state BLOB

Validation 2

Observe

  • selectedAccessType – Set an observer on this field to detect if and when the component changes it.

Action

  • setAccessControlLevel – Tells the component to set the containers access control level to CONTAINER

Validate

  • selectedAccessType – Validate that the selectedAccessType field changes to the one specified by the action.

How is this Better than an Integration Test?

In the Ember world, an integration test is essentially a test that inputs actions directly from the DOM (the user interface). This test removes any dependencies of your UI by simulating user events through actions but still tests all the functional components of the feature. This allows for your developers to implement different user experiences, but still maintain adequate code coverage of the functional parts of your code.

What this test doesn’t cover is that you didn’t accidentally hide, or misplace something in the UI. However, you can visually see those things and its much more likely that would be caught earlier by simply looking at that component where as a test like this verifies that the underlying code for the feature is functional.

Everything Else

When it comes to testing the entire user experience you really should rely on your team for doing some manual testing of the scenarios most important for you, as well some automation before you ship a release.

There’s nothing wrong with making automated UI tests or end-to-end tests. However these kinds of tests shouldn’t be used on a per-commit basis since it takes dependencies on little things such as the amount of time an animation takes to happen which can become fragile.

The point is that there are appropriate tests to put into your CI pipeline and tests that you should not. Realizing this will help keep your Devops processes squeaky clean allowing your dev team flexibility to ship product fast and efficiently.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>