Testing Mobile Applications (CITCON 2010)
The CITCON 2010 London session about software testing on mobile devices was well-attended with a reasonable distribution of experience across Android, iOS, Blackberry, proprietary platforms and people with experience of cross-platform tools.
- Testing as a “lagging phenomenon” (or as an afterthought…)
- look at the time delay between web application development and emergence of web-based tools
- Some testing frameworks beginning to emerge for mobile e.g.
Types of testing and cost/benefit
- Manual — for usability
- On device — actual performance and speed
- Emulation — for scaling to hundreds or thousands of test runners and for when devices aren’t yet available or cost too much
Book based on mobile testing experience at Google:
Julian Harty, “A Practical Guide to Testing Wireless Smartphone Applications”, Morgan & Claypool Publishers, 2009 (ISBN 1608452530)
Popularity in different markets
Financial: Blackberry and iOS
Consumer: Android and iOS (as of November 2010 Android has just eclipsed sales of iOS handsets.)
Tomtom’s Story
Worked to partition legacy code out into 3 layers:
- UI
- App domain
- platform abstraction layer
The plan:
1) Keeping UI and platform abstraction layers as thin and separate as possible.
2) Separate build and test of middle layer — especially routing algorithms and navigation data.
Separation of concerns and basic engineering concerns still important for mobile development. Perhaps more so when targeting multiple platforms from a shared codebase.
Causes of platform fragmentation
- manufacturers e.g. button placement, memory, screen resolution, other hardware devices
- network operator custom builds & restrictions
John Resig - mobile testing presentation - “can’t emulate what network operators put in the platform” (Paraphrasing)
Cross-platform Strategies
- problem: like early days of the web where presentation and domain logic mixed up — for example PHP/JSP style mix-up
- tools:
- Phone Gap
- JQuery touch
- Rhodes/rhomobile (ruby-based)
- Titanium
- javascript inside VM’ed environment: http://stackoverflow.com/questions/2444001/how-does-appcelerator-titanium-mobile-work
- (apparently) painful place to run Jasmine BDD for testing
advantages
- get app out early
- gauge demand quickly
- get rapid feedback
disadvantages
- not best UI
- not best performance
Other cross-platform testing related ideas
- idea: collecting data to empirically improve the app:
- get usage data
- get demand data
- to find out what worked/failed?
- getting to market early: building up a lead over the native approach
- Software Craftsmanship Robot game example:
- Getting into the game early much more important than perfection
- Built up a budget of points to spend on a rewrite
- webkit in/on most mobile platforms
- means good AJAX/Javascript/HTML 5/JQuery Mobile support
- Chrome as a 90% complete indicator for iOS/Android compatibility
See also: TDD and CI for iPhone
