Cross-device Optimisation

netmagazoneThis month’s net magazine (May issue to be exact) has a great article on 15 tips for cross-device optimisation. Craig Sullivan gives some rules to remember when testing cross platform.

Four insights of particular use for mobile developers…

  • Track people, not devices. Many people use their phone to find items to buy and complete the purchase on the desktop or laptop where it’s easier to checkout. Use tools such as Google Analytics user view to track the user rather than the device to gain a better appreciation of whether leads are coming via mobile or desktop.
  • Don’t ignore the wider context. Think about varying the functionality depending on the context. Craig gives the example of an airline app where what the user needs might be very different 48 hours before a flight than it is when it’s used at the gate.
  • Don’t assume everyone uses iPhones. Avoid concentrating on what happens to be your personal device. Look to user base numbers and ideally recent domain specific metrics.
  • Segment testing by device class. Different devices might look very different and might need different testing.

Latest Android Installed Base

opensignalOpenSignal have a new version of their study into Android Fragmentation. It’s the multi-coloured graphic, that I won’t be reproducing here, you often see when some blogger starts talking about fragmentation. Instead, I find the variation of OS version market share over time to be more interesting as it helps define what minimum version to target for new projects…

androidfragmentation

I believe this data to be more useful than declared device shipments as it a) represents the installed base and b) tells us the Android version. It’s also useful to compare with Google’s own dashboard.

It has been the case for a while now that there’s no point supporting pre Android 4.0.3 (Ice Cream Sandwich). Having worked on many past projects that had to go down to 2.3, this is great relief as it simplifies development and testing. Despite the hype over fragmentation, carefully selecting test devices that cover all the Android versions (v4.03 and later), top brands, dpi bands and main screen sizes is usually sufficient on most projects.

There are still a few outlying ‘strange’ current devices to consider. For example, there’s the Samsung Galaxy Pocket 2 that runs Android 4.4.2 yet only has a ldpi 240×320 screen. I find it’s a useful device to test apps at the lowest end of the device spectrum.

Mobile, Agile, Lean and Startups

jeffpattonFor the last few years I have mainly worked with startups. Of the many companies I have worked with, very few, only about 10%, have used Agile. Does it matter?

agileAgile

On the one hand, we have a few Agile tech speakers and consultants who will tell you if you don’t do it their way you’re stupid or incompetent. On the other we have people such as Jeff Patton, who previously won Agile Alliance’s Gordon Pask Award for contributions to Agile Development, saying common Agile practice isn’t for startups. Jeff’s article is thought-provoking because he says, for many startups, learning velocity is more important than delivery velocity.

leanstartupLean startup

Martin Fowler has an related article where he says “You don’t do agile or lean you do agile and lean”. I am not sure this is always true as, in practice, Agile invariably results in a significant upfront investment in additional unit testing – but I suppose it strictly need not when mixing Agile and lean.

You really need to consider your project. Is learning velocity important? Agile and better testing came out of a time when there were huge projects that were failing. Is your mobile project that huge/complex (it can be) or are you over-engineering? What’s your initial budget? What’s the anticipated lifespan of your software? Will the code ever have to be handed over to others to continue to develop? Is the project mission critical? How many people will be developing the project and how experienced are they? How much do you trust your current development process if you have one? How quickly do you expect things to change and by how much? Answering some of these questions might help you decide how much Agile matters for your project.

Android Fragmentation

android.gifThe Register reports from Google I/O that Android will offer an i-Phone-like App Store.

The more interesting part of the report is at the end – after all, how will such an app store determine what applications are compatible with what phones?…

"They can add to it. They can remove from it. They make it their own," Rubin said. "They can rip out all the Google stuff and put in all Yahoo! stuff."

This goes for entire APIs (application programming interfaces) as well – an obvious a point of concern for some developers mulling apps for the platform. "I see Android and I see all its APIs. What’s to stop someone from turning off all those APIs?" one developer asked Rubin.

There’s nothing to stop them, Rubin said. But he indicated that Google will provide tools that enable developers to easily verify the make-up of each Android handset. "We’re providing a piece of technology – I can’t go into a great amount of detail – that tests the APIs," he explained. "This will be a script that you’ll be able to run…and determine whether all the APIs are there."

As a developer, I don’t see how this will work. First of all, developers need APIs to determine individual capabilities, not a script that does this all at once. I need to do this at runtime because capabilities may change over time as the phone software is upgraded and updated.

Even if I know a capability doesn’t exist, what do I do? I can’t code around potentially thousands of missing capabilities.

Finally, even when capability tests exist and they indicate a capability is present, this won’t guarantee compatibility. There’s already a similar situation in the Java ME world where Sun Technology Compatibility Kits (TCKs) are used to test JSRs. There are thousands of very detailed tests and despite the fact that Java runtimes (Java Virtual Machine,JVM) on phones are run through them, it’s not possible to guarantee that a Java ME application that runs on one JVM will also run on another JVM.

Why is this? Despite the large number of tests, they cannot test everything. It’s not just the presence of an API that’s important, it’s also the behaviour. An API might pass a test but have an obscure bug that only becomes evident under particular circumstances. An API might also only implement a sub-set of a capability that’s either enough to pass the test or due to lack of some phone hardware capability. Also, no matter how hard humans try, with thousands of APIs, the expected behaviour of some of them will be to some extent ambiguous and open to interpretation. There are also types of test, particularly UI related, that are very difficult if not impossible to automate (The JAVA TCK’s rely on manual tests for these).

Hence, I currently believe there will always be Android fragmentation – and certainly more so than with more controlled platforms such as Symbian, Windows Mobile and the iPhone.

Mob4Hire

mob4hire.gifI came across Mob4Hire this week. A new site that provides "Crowd Sources Mobile Application Testing". The developer submits projects involving specific handsets and network operators. End users bid on projects and get paid via Paypal.

It’s often the case that an application is tested but fails in real-life use due to limitation of some random network operator. Mob4Hire can help reduce this problem by allowing developers to reach a greater breadth of network operators and phones. I also see it as a much more targeted solution than beta testing where typically only fraction of percent of users submit useful observations.

On the other hand, I can see some companies having problems with confidentiality and the inevitable variable quality of testers. Also, this site doesn’t solve the problem of ongoing support and testing – where the majority of problems surface. If you/your company didn’t test the particular variant then it may be difficult to fix.

The Mob4Hire site/process is still beta and a bit clunky and ambiguous in places. However, I think Mob4Hire may have something useful here if they evolve the service based on developer and tester feedback.

Perpetual Beta

On the Nokia Beta Labs blog there’s an interesting post and discussion on the meaning of ‘beta’. Many people accept that in the web world ‘beta’ now means the product or service will work as expected and continually evolve. However, it’s my experience that many ‘mass market’ users will actively avoid trying an installable mobile application if it is beta. The corollary of this that if you wish a large number of people to try your working downloadable application, then don’t mark it as ‘beta’.

CppUTest Unit Test Framework

symbian.gifArtem at Agile Software Development has extended the CppUTest unit test framework to support Symbian descriptors and leaves. There’s also a heavily commented example how to use the framework.

If you are looking into unit test tools then you might also like to consider Penrillian’s Symbian OS Unit and Symbian Test Unit.

paca Mobile Center

paca.gifAt one of the London Mobile Mondays I came across the paca mobile center. It’s a facility in Marseille and Sophia Antipolis, France that fills the gap between buying all test devices yourself or outsourcing to a 3rd party test house.

For 300 Euro per day or 6000 Euro/year (both +VAT) startups can gain access to between 400 and 500 devices. There’s also a facility in Glasgow, UK that also provides similar facilities free of charge to paca members.

Facilities such as this and 3rd party test houses are great for low budget startups. However, they don’t help when it comes to resolving support issues quickly when an application is in the field… as you might not have the particular phone to reproduce the problem. Also, while the application might have worked on the particular phone during testing, issues due to different firmware or (more usually) due to different network operators or (even more usually) user error can often stop your application from working. Hence, if you can afford it, then I believe it’s best to keep your testing in-house so that you can reproduce any problems.

Update 21 Sept: I received an email from paca Mobile Center (PMC) with the following clarifications…

  • PMC is a not-for-profit organisation and the fees just cover handset purchasing.
  • Wireless innovation in Glasgow is an incubator. Their service is restricted to their companies and they have 50 handsets.
  • Handsets aren’t donated by manufacturers or operators. Instead 30 handsets are purchased each month.
  • There are also services to allow PMC to carry out tests so that you don’t have to actually travel there.
  • PMC founders are mobile application specialists and its possible to link companies to solve some issues.
  • Customers tend to buy the top 20 handsets but haven’t time to source the whole market.
  • PMC spends a lot of time identifying and sourcing new handsets – much more time than you would purchasing a phone already in the market.