Android Fragmentation

android.gifThe Register reports from Google I/O that Android will offer an i-Phone-like App Store.

The more interesting part of the report is at the end – after all, how will such an app store determine what applications are compatible with what phones?…

"They can add to it. They can remove from it. They make it their own," Rubin said. "They can rip out all the Google stuff and put in all Yahoo! stuff."

This goes for entire APIs (application programming interfaces) as well – an obvious a point of concern for some developers mulling apps for the platform. "I see Android and I see all its APIs. What’s to stop someone from turning off all those APIs?" one developer asked Rubin.

There’s nothing to stop them, Rubin said. But he indicated that Google will provide tools that enable developers to easily verify the make-up of each Android handset. "We’re providing a piece of technology – I can’t go into a great amount of detail – that tests the APIs," he explained. "This will be a script that you’ll be able to run…and determine whether all the APIs are there."

As a developer, I don’t see how this will work. First of all, developers need APIs to determine individual capabilities, not a script that does this all at once. I need to do this at runtime because capabilities may change over time as the phone software is upgraded and updated.

Even if I know a capability doesn’t exist, what do I do? I can’t code around potentially thousands of missing capabilities.

Finally, even when capability tests exist and they indicate a capability is present, this won’t guarantee compatibility. There’s already a similar situation in the Java ME world where Sun Technology Compatibility Kits (TCKs) are used to test JSRs. There are thousands of very detailed tests and despite the fact that Java runtimes (Java Virtual Machine,JVM) on phones are run through them, it’s not possible to guarantee that a Java ME application that runs on one JVM will also run on another JVM.

Why is this? Despite the large number of tests, they cannot test everything. It’s not just the presence of an API that’s important, it’s also the behaviour. An API might pass a test but have an obscure bug that only becomes evident under particular circumstances. An API might also only implement a sub-set of a capability that’s either enough to pass the test or due to lack of some phone hardware capability. Also, no matter how hard humans try, with thousands of APIs, the expected behaviour of some of them will be to some extent ambiguous and open to interpretation. There are also types of test, particularly UI related, that are very difficult if not impossible to automate (The JAVA TCK’s rely on manual tests for these).

Hence, I currently believe there will always be Android fragmentation – and certainly more so than with more controlled platforms such as Symbian, Windows Mobile and the iPhone.

Mob4Hire

mob4hire.gifI came across Mob4Hire this week. A new site that provides "Crowd Sources Mobile Application Testing". The developer submits projects involving specific handsets and network operators. End users bid on projects and get paid via Paypal.

It’s often the case that an application is tested but fails in real-life use due to limitation of some random network operator. Mob4Hire can help reduce this problem by allowing developers to reach a greater breadth of network operators and phones. I also see it as a much more targeted solution than beta testing where typically only fraction of percent of users submit useful observations.

On the other hand, I can see some companies having problems with confidentiality and the inevitable variable quality of testers. Also, this site doesn’t solve the problem of ongoing support and testing – where the majority of problems surface. If you/your company didn’t test the particular variant then it may be difficult to fix.

The Mob4Hire site/process is still beta and a bit clunky and ambiguous in places. However, I think Mob4Hire may have something useful here if they evolve the service based on developer and tester feedback.

Perpetual Beta

On the Nokia Beta Labs blog there’s an interesting post and discussion on the meaning of ‘beta’. Many people accept that in the web world ‘beta’ now means the product or service will work as expected and continually evolve. However, it’s my experience that many ‘mass market’ users will actively avoid trying an installable mobile application if it is beta. The corollary of this that if you wish a large number of people to try your working downloadable application, then don’t mark it as ‘beta’.

paca Mobile Center

paca.gifAt one of the London Mobile Mondays I came across the paca mobile center. It’s a facility in Marseille and Sophia Antipolis, France that fills the gap between buying all test devices yourself or outsourcing to a 3rd party test house.

For 300 Euro per day or 6000 Euro/year (both +VAT) startups can gain access to between 400 and 500 devices. There’s also a facility in Glasgow, UK that also provides similar facilities free of charge to paca members.

Facilities such as this and 3rd party test houses are great for low budget startups. However, they don’t help when it comes to resolving support issues quickly when an application is in the field… as you might not have the particular phone to reproduce the problem. Also, while the application might have worked on the particular phone during testing, issues due to different firmware or (more usually) due to different network operators or (even more usually) user error can often stop your application from working. Hence, if you can afford it, then I believe it’s best to keep your testing in-house so that you can reproduce any problems.

Update 21 Sept: I received an email from paca Mobile Center (PMC) with the following clarifications…

  • PMC is a not-for-profit organisation and the fees just cover handset purchasing.
  • Wireless innovation in Glasgow is an incubator. Their service is restricted to their companies and they have 50 handsets.
  • Handsets aren’t donated by manufacturers or operators. Instead 30 handsets are purchased each month.
  • There are also services to allow PMC to carry out tests so that you don’t have to actually travel there.
  • PMC founders are mobile application specialists and its possible to link companies to solve some issues.
  • Customers tend to buy the top 20 handsets but haven’t time to source the whole market.
  • PMC spends a lot of time identifying and sourcing new handsets – much more time than you would purchasing a phone already in the market.

Motorola Z8 Testing

uiqtesting.gifOne of the problems of application development is that it’s often not possible to test on phones that are about to be released. This is especially the case when you are in a country where phones arrive late to the market.

Sometimes it’s possible to ask phone OEM’s for pre-production phones but often there aren’t enough units to go round. Unless you have an application that is compelling for the OEM then you won’t get a device.

UIQ have solved this problem for the Motorola Z8 by offering to test applications. What’s more it’s free of charge. This offer is open until 30 September 2007.

This has made me wonder why phone OEMs don’t offer this informal service as a matter of course. OK, it would cost the OEM something – but a few people informally testing applications would be negligible in the overall scope of things. It would help improve the quality of released applications (see Ewan’s recent comments for some surprising observations), encourage developers and also allow the OEM to identify potential partners. Microsoft do a similar thing via ‘Evangelists’ who seek out and help developers in this and many other interesting ways. Note however, the kind of thing I am talking about shouldn’t be confused with Symbian Signed testing – it’s more about informal collaboration.

Testing Across Devices

The London Mobile Monday Yahoo group has an interesting thread on phone testing. The problem is the need to test against a large number of different handsets and networks. It’s obviously not possible to buy every phone and visit every country to test every network. The following diagram from Zanan shows how the increasing number of services and handsets results in an explosion in the number of testers…

testingexplosion.gif

Some comments from the MoMo Group…

  • DeviceAnywhere.com – Has over 300 handsets across 10 carriers. 3 hr free trial.
  • Nothing beats the actual phone in the real network.
  • Orange’s developer partner www.orangepartner.com  testing labs can provide a very wide range of handsets.
  • zandan.com offers testing across live networks across the world.

My personal experience is that you should not under estimate the problems that can be caused by different network operators in addition to those caused by different phones.

One new service not mentioned is Nokia’s new free Remote Device Access. However, this just covers S60 phones and there’s no carrier network connectivity (some phones may have WLAN access where the phone supports it).

GMail via Java ME

google.gifI just had a play with Google’s new Java ME application that allows faster and easier access to GMail.

The press release says…

"It is currently compatible with all J2ME-enabled phones in the U.S. and works with a variety of carrier service plans."

I am in the UK but I thought I’d give it a go anyway. The WAP download page said…

"We are not sure the Gmail application will work correctly on your phone"

I managed to download and install on a Sony Ericsson M600i. I was surprised the app was untrusted as it then required explicit user permission to connect every session. On connecting I got the error "An unexpected error occurred during program execution". It seems other people have had similar problems.

Update :  The app does work in the UK… just not with my GMail account. I tried my wife’s (newer) account and it works. The instructions about compatible countries/networks previously led me to believe it was something to do with being in the UK. Incidentally, I have also found a phone compatibility listing.

It seems Google have a problem with some types of account. Maybe it’s to do with different types of GMail account or an account’s settings. More meaningful error messages might give us a clue.

As an aside, if you have a M600i or any other smartphone for that matter, just use GMail POP access from the phone’s built-in email application. It works very well and is much easier to use than mobile web access. Received email remains in your GMail account.  Furthermore, if you use the GMail SMTP (smtp.googlemail.com) to send email, your sent email will also end up in your GMail ‘Sent Mail’ folder. Phones such as the M600i also allow you to poll periodically for messages which gives the same benefit of having push email (with higher data use of course).