Automatic Testing for Cordova/Phonegap Webview on Android

Question

I'm currently looking for a solution to test Android Apps automatically, native Android apps as well as hybrid Cordova/Phonegap apps.

The UI Automator from Google looks pretty fine, but can not deal with WebViews, which makes it unusable for me. Then I found the promising automation framework Appium based on Selenium, which should also be able to deal with hybrid apps, but I couldn't get it to work with WebViews, it seems to not be implemented fully yet, as also several posts prove.

After trying Appium I found a hint for Selendroid, that also should work for hybrid apps, but it seems pretty much buggy and keeps crashing my whole computer all the time.

So, does anyone here know a working solution for testing Android Apps with WebViews? I'm working on Windows 7. Thanks a lot in advance for any help.

Answer 1

In first case we need to know what kind of tests you like to drive. I think you like to run functional tests on the UI? Thats what i can think about after reading your question.

So lets split this by the given environment:

native iOS

• unit tests -> do it in xCode. http://www.apple.com
• functionality tests -> http://appium.io/

native Android

• unit tests -> JUnit, http://developer.android.com/tools/testing/testing_android.html
• functionality tests -> http://selendroid.io/

native Web like phonegap/cordova Apps

• unit tests -> jsUnit, http://jsunit.berlios.de/
• QA Testing and Checkstyle -> jsHint/jsLint, http://www.jshint.com/ & http://www.jslint.com/
• functionality tests -> http://docs.seleniumhq.org/ & https://npmjs.org/package/ripple-
• functionality tests (E2E) -> https://angular.github.io/protractor/#/ or https://github.com/jasmine/jasmine
• Here is a article about JavaScript QA / QS

native X - The out of the Box solution for all Platforms

• Here we are. There is none atm. But .. footage is that one here:

Gappium based on appium uses json for writing your tests. As i get the thread right, it was testet with Web/iOS for this moment. Android/Win seems not be integrated for this moment:

https://github.com/appium/io.appium.gappium.sampleapp

Note for phonegap & apache cordova functional tests:

Its possible to use ripple and selenium for testing hybrid apps. Keep in mind that some native function/plugins (hehe native functions on hybrid apps ... =) cant be tested or need to be mocked on a unknown way. Thats the footage for this moment.

Maybe usefull for you Continous Integration Server:

https://github.com/linslin/PHPUnit-selenium-HTMLGallery

Answer 2

I faced this challenge with our own Cordova app ( sourceforge.net/p/ustadmobil/codehtml5cordova/ci/master/tree/ ) ; and I needed a way to integrate this with Jenkins CI. We have Cordova specific code (e.g. filesystem access) so just using PhantomJS and Grunt is not an option.

Essentially what I did was to launch the app with the test page as index.html in the emulator, run an http server, and then have the test page pass the results back using AJAX.

1. Script launches an emulator with the package
   $ /opt/adt/sdk/tools/emulator-x86 -avd $AVDNAME -qemu -m 2047 &
   $ EMULATEPID=$!
   $ adb wait-for-device #wait for device to start $ adb shell input keyevent 82 #unlock screen

2. Make a test page using qunit, which when finished will communicate the test results with an AJAX call.

3. Make a basic nodejs server script that will listen for that ajax call and save the results to a file. A bash script can then read that file and determine if the tests passed or failed : see http://sourceforge.net/p/ustadmobil/codehtml5cordova/ci/master/tree/setup/node-qunit-server/node-qunit-server.js
   
4. In the build script wait for the result file to come, then kill the emulator and the NodeJS server - see http://sourceforge.net/p/ustadmobil/codehtml5cordova/ci/master/tree/setup/android/unit-test-setup-android.sh

Also worth noting: this approach works for the cross platform nature of Cordova: it should be possible to take the same approach running iOS, Windows Phone emulators etc (I also use this test method against our NodeWebKit desktop version which shares most of the code).

Hope this helps,

-Mike

Answer 3

Generally, testing hybrid apps is quite a challenge with API-based approaches. It can be done, but typically you need to integrate the UI technologies (e.g. WebView, Native, OpenGL, ...) with several APIs or some kind of wrapper (like Selendroid).

Here at TestObject (I'm one of the co-founders) we've created an Image-based UI testing approach specifically for cases like hybrid apps, interaction between apps, and so forth. This black-box approach is solely based on capturing the screen - we don't care what happens underneath.

Feel free to give it a try (http://testobject.com/) with your hybrid Cordova/Phonegap app and let me know how it worked out in comparison to UI Automator, Selendroid, Appium. Let us know if you stumble over any issues.