Monday, October 28, 2013

Stop submitting App Studio created apps to the store - you're just highlighting the poor test process

App Studio is a tool from Microsoft that will allow you to "take your app from idea to Windows Phone in record time". Unfortunately the apps it creates shouldn't pass the store's certification testing.

App Studio is still in Beta but already many people are using it to build apps and submit them to the store. Seems reasonable enough as that's what it's for. The only issue is that the apps that it generates contain bugs that mean they shouldn't pass the testing that is done as part of the application certification process. Every time such an app is certified it highlights a failing of the publisher and the store test processes to identify the issues and stop them reaching the consumer.
Even if the person using the app can get past the, arguably, poor design of the generated apps, no one deserves an app that crashes.

So what's the issue?

We'll, the biggest and easiest to demonstrate is an issue that should cause the app to fail the technical certification requirement 5.1.2:

The app must handle exceptions raised by the any of the managed or native System API and not close unexpectedly. During the certification process, the app is monitored for unexpected closure. An app that closes unexpectedly fails certification. The app must continue to run and remain responsive to user input after the exception is handled.
Recommendations
When handling exceptions, an app should provide a user-friendly error message. You may present a message that is relevant to the context of the app.
So how do the apps fail this requirement?
Just follow these simple steps to recreate a crashing scenario:
  1. Launch the app
  2. Tap on an item loaded from the internet.
  3. Pin that item (via the link in the app bar)
  4. Disconnect the phone's network connection (enabling flight mode is easiest)
  5. Tap on the pin created earlier.
  6. Observe the app open for a few seconds without displaying any content before crashing/closing.

I documented this and other issues previously but they all appear to still exist.


Two important lessons:
  1. Don't assume that a generated app is good enough to release to the store. Test it thoroughly yourself!
  2. All apps will be used in an occasionaly connected environment. You MUST test the apps behaviour in all network connection scenarios.
If you want to use App Studio then by all means go ahead. Just be aware that if you want to submit the generated app to the store then be aware that there will be a need to modify the generated code before you do.



1 comment:

  1. This comment has been removed by the author.

    ReplyDelete

I get a lot of comment spam :( - moderation may take a while.