Tutorial: Android Testing In The Cloud With Android Studio, The New Gradle Android Build System and Manymo

June 17/2013, by Dan Grigsby


This tutorial shows how to use Android Studio, the new Gradle Android build system, and Manymo to test Android app Activities in the cloud.

Testing With The New Gradle Android Build System

Under the new Gradle Android Build System tests are no longer segregated in a separate test project; they're part of the app's src tree.

In the next steps we'll create a simple project and add tests to it:

Creating A Hello World App In Android Studio

Open Android Studio and create a New Project:

  1. Enter HelloWorld as the Application Name. Leave the other defaults unchanged, and click Next.

  2. Click Next to accept the defaults for the Foreground, etc.

  3. Click Next to accept the defaults to create a Blank Activity.

  4. Click Finish to accept the defaults for the Activity Name, Layout Name and Navigation Type and create the project.

Preparing The Activity For Testing

Activity testing often focuses on validating the on-screen representation of an app on many different device configurations.

In order to test things, we need programatic access to them. So, we'll modify the boilerplate app to include an id on the "Hello world!" TextView:

  1. From the Project pane, expand the folders HelloWorld > app > src > main > res > layout.

  2. Open the activity_my.xml layout; select the Design tab if it's not already selected.

  3. Click on the Hello world! TextView, and locate id attribute in the Properties pane and set it to @+id/hello_world_text_view.

Adding A Place For Tests To The Project

We'll continue by adding the folders that'll hold the tests to the project.

From the Project pane:

  1. Expand the folders HelloWorldProject > HelloWorld > src.

  2. Right-click src, choose the New menu item, select Directory enter androidTest for the directory name, and click OK.

  3. Right-click on the newly created androidTest directory and create new directory named java under it.

  4. Right-click on the newly created java directory and create new directory named com.example.helloworld.test under it.

The tests will live in the last folder we created in the above steps; its name should be the app's package name with .test appended to it.

Adding A Test Class To The Project

Next, we'll add a test class to the project:

  1. Right click on the com.example.helloworld.test directory, choose the New menu item, select Java Class, enter MyActivityTest for the name, and click OK.

Testing An Android Activity

Android provides the ActivityInstrumentationTestCase2 base class for testing Activities. Use this subclass to test that the "Hello world!" TextView is on-screen:

    package com.example.helloworld.test;

    import android.test.ActivityInstrumentationTestCase2;
    import android.widget.TextView;
    import static android.test.ViewAsserts.assertOnScreen;

    // Import the activity to test and the app's R
    import com.example.helloworld.MyActivity;
    import com.example.helloworld.R;

    public class MyActivityTest extends ActivityInstrumentationTestCase2<MyActivity> {
                                                                          //^Activity to test

        private MyActivity mMyActivity;
        private TextView mHelloWorldTextView;

        // Be careful about letting the IDE create the constructor.  As of this writing,
        // it creates a constructor that's compiles cleanly but doesn't run any tests
        public MyActivityTest() {

        protected void setUp() throws Exception {

            mMyActivity = getActivity();
            mHelloWorldTextView = (TextView) mMyActivity.findViewById(R.id.hello_world_text_view);

        // Methods whose names are prefixed with test will automatically be run
        public void testTextView() {
            assertOnScreen(mMyActivity.getWindow().getDecorView(), mHelloWorldTextView);

Testing On Multiple Devices, Each With A Unique OS Versions And Screen Size

There are a dozens of different Android OS version and screen size combinations. Android Instrumentation Testing really shines when it's used to confirm that presentation and functionality on all of them. To that end, we'll run our tests on multiple emulators, each with a unique OS and screen size combination, all running in Manymo's cloud.

Manymo Setup

If you don't already have one, sign up for a free Manymo account – no credit card is required.

You'll be able to run tests using any Manymo service level, including Free. That said, the number of emulators you can launch per month is limited by service level, and testing can use them up quickly. To give you more runway, you can use this link to give yourself a free month of our Pro level after you've signed up.

You'll need to install or upgrade the Manymo Command Line tool to use the Manymo plugin for the new Gradle based Android Build System.

Configuring Gradle To Run Tests In The Cloud On Manymo

Next, we'll locate the correct Gradle configuration file and update it to run tests on Manymo's cloud-based emulators:

  1. Locate the build.gradle file in the HelloWorld folder (not the top-level one in HelloWorldProject) and open it for editing.

  2. To add the Manymo plugin as a dependency and plugin, and configure the plugin, replace the contents of the file with the following:

    buildscript {
        repositories {
        dependencies {
            classpath 'com.android.tools.build:gradle:0.4'
            classpath 'com.manymo:gradle:1.1'
    apply plugin: 'android'
    apply plugin: 'manymo'

    dependencies {
        compile files('libs/android-support-v4.jar')

    android {
        compileSdkVersion 17
        buildToolsVersion "17.0.0"

        defaultConfig {
            minSdkVersion 7
            targetSdkVersion 16

    manymo {
        // which devices to run the tests on; use "manymo list" to list all available
        devices "17_WXGA800-7in_x86", "17_WXGA720_x86"

        // how many emulators to run tests on in parallel; use 1 for Free or Basic, 2 for Pro, and 12 for Ultimate
        maxThreads 3

        timeOut 30

Configuring Android Studio To Run Tests

The final setup step is to add a Run Configuration to configure Android Studio to run the tests:

  1. From the main Android Studio menu click Run > Edit Configurations...

  2. From the modal dialog, make sure that Android Application is expanded and that app is selected.

  3. Click the + to Add New Configuration and select Gradle.

  4. Name the configuration Run Tests On Manymo.

  5. Select the top/project level build.gradle – not the one we edited in the previous section – for the Gradle project field.

  6. Enter manymoInstrumentTest in the Tasks field.

  7. The configuration should look like the image below; click OK to save it.

Running The Tests

To run the test:

  1. Click the Run Configurations drop down and select the Run Instrumentation Tests On Manymo item.

If there are any problems (e.g., any of the tests fail) their log items will be highlighted with red text.

Reviewing Results

An HTML report will be compiled in the build/reports/instrumentationTests/device/manymo folder when the run completes. Open index.html in a browser to explore the results.


Having a problem you can't lick? Contact support.

More to Come

We've got lots more planned for testing with Manymo!

Stay tuned by following us on Twitter and Google+.