Make tests on Android projects I

Hi all!

As you might know if you follow my blog I'm working at Schibsted, and one of the projects we are developing is an Android application to access to the classifieds websites of the company (like custojusto.pt, bomnegocio.com, tori.fi, …). One of the ideas that defines our work method, Extreme Programming, is the continuous integration of the new developments (or iterations) of the project, so we have to include tests on every project we have in order to check the backward compatibility, and the Android project is not an exception.

Making tests for any Android application (I should say on any mobile application, but I've only seen Android and IOS) is a pain, and for a lot of reasons, including:

• To test any android application you must have an android environment up and running. That means that the tests are going to be slow because they are going to run in a complete android environment (that can be a virtual machine, an emulator or a real device).

  It also means that you have to get several environment (android version and devices) to test your application, because even with the same version of Android the execution can be quite different between on device and another (you find differences even between one device emulator and the real device!).

  Only this can make the tests veeeeeery slow.

• The test is a different project running in a different process. This means that the test have to compile against the project, so every change on the project lead to a clean and recompilation of all the tests. That's because ant is not so good as Makefile checking the dependencies of each file (at least by default in the Android projects, I supose it can be quite better if you customize it to resolve all your dependencies, but this is not the usual). Sometimes even I uninstall all the test project in order to delete from the emulator some garbage that adb/android didn't delete at the reinstallation.

  This also means that the test is responsible of running up the applicatiion, set up all the parameters you need and shut down the application. This is not bad, but I think the Android sdk should include a testing environment capable of setting up and down the application by you, so you don't have to do it in each test.

  Also, you have to know what actions have to be done at the test process and what actions should be done at the application process (or wait until the android test run an exception telling you that you have to run some action in the application process).

  Finally, if your application depends on an external application (like the gallery or an email application) it's almost impossible to test this part of the application, because you can test and control the application (or process) that you are developing, not an external application, so in some cases there are parts of you application impossible to test, because it's not possible (at least I didn't found how) to simulate an external calling.

• The documentation about testing is risible. Appart of some basic documentation and examples at the official android developers page there isn't any documentation of quality about the testing process, so it's very hard to learn how the tests works and to make a good testsuite of your Android application. I've found some posts that even recommend you to test your application directly on the physical device :).

For this reasons in the next few days I'm going to make a little tutorial about how to make some tests to your Android application

To be continued...