Fast and reliable UI tests on Android
How we wrote 250 UI tests that run under 3 minutes with Espresso, Mockito and Dagger.
In this article we’ll explore how we used Mockito and Dagger 2 to achieve faster and more reliable UI tests on Android. If you are a developer looking to start writing UI tests on Android or improve the performance of your existing ones, read on.
The first time I wrote automated UI tests for an Android app was a few years ago using Robotium. I thought the more realistic the test environment is, the better. In the end these kind of tests should behave like a superhuman that can quickly tap everywhere and make sure nothing breaks, right? I thought mocking was a terrible idea. Why would we change the behaviour of the app during testing? That would be cheating, wouldn’t it? A few months later we had about 100 tests that were taking over 40 minutes to run. They were so flaky and unreliable that they would fail 75% of the time, even though nothing was wrong with the functionality of the app. We were spending time writing them, but they weren’t helping us finding issues.
But as John Dewey once said, failure is instructive…
Failure is instructive. The person who really thinks learns quite as much from his failures as from his successes.
And we did learn. We learnt that relying on real APIs during testing is a terrible idea. You have no control over the data that is returned, therefore you can’t set preconditions for your tests. Moreover, tests become exposed to failures in the external APIs as well as your internet connection. If the wifi stops working, you don’t want the tests to fail. In the end, you simply want to test that your UI functions correctly. If you rely on external APIs you are essentially writing integration tests, and that was not what we wanted to do.
Mocking is the solution
Mocking consists of replacing a real object with a mock one that mimics the behaviour of the real one in a way that you can control. This practise is mainly used when writing unit tests but it can also be useful when writing UI tests. You can follow different approaches to mock Java objects but using Mockito is one of the simplest and more flexible options. Below you can see an example of how you can create a mock version of the UsersApi class and stub one of its methods so it will always return a static array of usernames.
Once you’ve created a mock object you need to make sure the app uses the mock version when running tests, but still uses the real one during normal execution. This is the tricky bit. If your codebase is not architected in a test-friendly way, replacing real objects with mock ones can be very difficult or even impossible. Moreover, the code you want to mock needs to be isolated in separate classes. If you are calling a REST API directly from your Activity using HttpURLConnection (I hope you are not), this call will be very difficult to mock.
Think about architecture before thinking about tests. Finding tests hard to write, unreliable or impossible to mock, is often a clear sign of a badly structured codebase.
A test friendly architecture
There are many ways you can structure your code so it’s easy to mock and therefore test. Here I’m going to use the architecture we use at ribot as an example, but you should be able to apply the same principles to any architecture. Our architecture is based on the Model View Presenter pattern. We decided to mock the whole Model layer during UI tests so we have more control over the data and we can write more valuable and reliable tests.
The DataManager is the only class in the Model layer that is exposed to the presenters. Therefore, in order to mock the Model layer we simply have to replace the DataManager with a mock one.
Using Dagger to inject a mock DataManager
Once we know what classes to mock, we need to think about how to replace the real objects with mocks during testing. We solved this problem by using Dagger 2 — this is a dependency injection framework for Android. If you have never used Dagger I recommend you to read this guide before continuing.
Our applications will contain at least one Dagger Module and one Component, usually called ApplicationComponent and ApplicationModule. Below you can see a simplified version of these classes where I only provide an instance of DataManager. You could also achieve this by annotating the DataManager constructor with @Inject but here I’m using a provide method to make it easier to understand.
The application component is then instantiated in your Application class.
If you are already using Dagger 2 you probably have a similar setup. The idea now is to create a separate application Module and Component that will be used during testing.
The TestApplicationModule provides a mock version of the DataManager using Mockito. The TestComponent is just an extension of the ApplicationComponent but it uses a TestApplicationModule instead of the regular ApplicationModule. This means that if we are able to set a TestComponent in our Application class before running the tests, our application will use a mock DataManager.
Creating a JUnit rule that sets the TestComponent
In order to ensure the TestComponent is set in the Application class before starting any test we can create a JUnit 4 TestRule.
The TestComponentRule creates an instance of TestComponent, this overrides the apply method that returns a new Statement. This new Statement will:
- Set the TestComponent in the Application class.
- Evaluate the base statement (this is when the test gets executed)
- Clear the component so we leave the state of the application as it was before. This is a good practise so we prevent issues where one test case affects the execution of another one.
This rule also exposes the mock DataManger through the getMockDataManager() method. This will allow us to easily access the mock DataManager from the tests so that we can stub its methods. Note that this only works because the provideDataManager() method in the TestApplicationComponent is annotated with @Singleton. If it were not a singleton, the instance we get when calling getMockDataManager() would be different to the one being used by the app. Therefore, we wouldn’t be able to stub it.
Writing the tests
Now that we have Dagger setup correctly and our TestComponentRule ready to use, we only have one more thing to do, write tests! We use Espresso to write UI tests — it’s not perfect but it’s the fastest and most reliable testing framework for Android.
Before writing any test we need an app to test 😅. Imagine we have a very simple app that loads some usernames from a REST API and displays them on a RecyclerView. The DataManager would look like this:
The loadUsernames() method uses Retrofit and RxJava to load data from the REST API. It returns a Single that emits a list of Strings. We will also have at least one Activity that displays the list of usernames in a RecyclerView — we can call this Activity UsernamesActivity. If you follow MVP you would also have a presenter but that’s not relevant for this example.
Now we want to test this very simple Activity. There are at least three scenarios that we should test:
- If the API returns a valid list of usernames, they should all display in the list.
- If the API returns an empty list of usernames, the message “Empty list” should show.
- If the API call fails, the message “Error loading usernames” should display.
And this is how those three tests would look like:
As you can see above, we use the TestComponentRule we created as well as the ActivityTestRule provided by the Android testing framework. The ActivityTestRule will allow us to launch the UsernamesActivity from the tests. Note that we use a RuleChain to ensure the TestComponentRule always runs before the ActivityTestRule. This is to make sure the TestComponent is set in the Application before any Activity is launched.
You may have realised the three tests follow the same pattern:
- Set up a precondition. This is achieved by stubbing the loadUsernames() method. For example, the precondition for the first test is to have a valid list of usernames.
- Launch the Activity.
- Check views display the expected values for the given precondition.
This is a very powerful solution. It allows you to test all the different scenarios because you have complete control over the initial state of the application. If you try to write those three tests without using mocks, it would most likely be impossible because the real API will always returns the same data.
If you want to see a fully working app that uses this testing approach then you can check out the ribot Android boilerplate or the ribot app.
There are a couple drawbacks we’ve found with this solution. The first is that stubbing at the start of every test can become messy. Complex screens may require 5 or 10 different stubs at the beginning of each test. Moving some of the stubs to a setUp() method can help but quite often each test require different stubs. The second issue is that UI tests become quite coupled to the underlying implementation, meaning that if you have to refactor the DataManager you will also need to change the stubs.
Despite those disadvantages, we’ve been using this UI testing approach at ribot with a couple of decent sized production apps and it’s proven to be very beneficial. For example, our latest Android app has 250 UI test that run in ~3 minute with very little flakiness. We also have an additional 380 unit tests that focus on testing the Model layer and presenter classes.
And that’s it! I hope you enjoyed this article. UI testing can be very frustrating but hopefully this helps you write better UI tests and therefore build better apps!
If you are looking for someone to help you build a robust and well-tested Android or iOS application why not get in touch with us at ribot?
If you have any questions about UI testing on Android or you have figured out an even better way of writing tests, I’d love to hear about it.
