Dr David Parsons

davidparsons.ac.nz

Category: Software Testing

Posts related to various aspects of software testing

Software Testing

Why regression testing matters

Sep 28, 2014 #Apple iOS bug, #regression, #regression testing by daveparsonsnz

Quite some time ago I posted a whinging blog about the software I have to use a editor of a journal. Over the last couple of years I have to acknowledge that the occasional improvement has occurred. For example it is now possible to see how many review requests each reviewer has responded to, and the dates of the reviews they returned. This is great, though one issue is that these changes just appear, without any warning. At least when your phone updates its apps, it does at least tell you it’s doing it, even if it doesn’t bother to ask you if it’s OK unless you have to approve various permissions.

Anyway, to the point of this post, which is regression. Of course the term has a number of meanings. I’m not referring here to sexually disturbing regression in Freudian analysis, or the statistical process for estimating the relationships among variables. I mean, of course, regression in the software sense, where an update to software reveals bugs that were not there in the previous version. A recent update to the journal software saw a complete change to the appearance of the main web page. Whether this was an improvement or not from an aesthetic or functional perspective I’m not sure, but the main thing I noticed was that the system has now forgotten all of the authors of all the papers submitted to the journal. This struck me as a not unimportant feature of a journal submission system.

The point of the above observation is of course that it highlights the great importance of regression testing. The danger of software maintenance is always that it may inadvertently break something that worked perfectly well before. Without regression tests you may not find out what you have broken until it is too late (e.g. you have forgotten every author of every paper in your journal submission system.)

Similar stories have been circulating about the rather more high profile Apple iOS 8 operating system update, which among other failures prevented people from making phone calls, still surely the main point of having a mobile phone. http://www.theguardian.com/technology/2014/sep/24/apple-ios8-update-withdrawn-iphones-bugs
Not the first time an Apple update has broken a major feature, as I discussed in a earlier post about problems with FaceTime.

Effective and comprehensive regression testing is not easy, and not something that can be tacked on at the end of the testing process. Rather it has to be part of a full testing strategy that starts with unit testing, and follows though integration testing, acceptance testing, performance testing, exploratory testing, security testing etc. However it is the last barrier between the software bugs and the customer, and deserves more attention that it appears to be getting from some software development teams.

Mobile Software Development, Software Testing

Faking your location – how and why

Apr 29, 2014 #fake GPS, #Google Maps API, #mobile app testing by daveparsonsnz

One of the many interesting types of app you can download on your phone is one that will fake your GPS location. There are quite a few of these on the app stores, but they all work in much the same way. If you turn off the network based location services on your phone, but leave the GPS service on, these apps can fool any other app that uses your GPS location into thinking you are somewhere else (and there are plenty of these – think of all those permissions you say ‘yes’ to without reading them every time you install a new app.) When you run a fake GPS app, all you have to do is click on a map of where you want to pretend to be and hey presto, your boss thinks you’re at a client, your partner thinks you’re working late, your accounts department thinks you’re staying in a cheap hotel and your friends think you’re on holiday in the Cayman Islands.

Well, those seem to be the kind of use cases that people suggest on the Web. I had a rather more humdrum requirement. I was updating a mobile learning game that we have been developing for a long time, and converting it to use the most recent version of the Google Maps API. Unfortunately this renders it impossible to test map based apps in desktop device emulators without some rather nasty hacks, which are technically illegal. The only sensible option is to test your apps on a device, but how do I test a location based app when I’m sitting in the office with the phone connected to the laptop through a cable?

The answer is. of course, to fake your GPS location with one of the aforementioned apps. I used ‘Fake GPS Fake Location’ by Andev, but there are plenty of others (with very similar names). First, I ran the app, then just moved the red dot to where I wanted to pretend to be. In this case, I wanted to generate some screen captures of the game being used at Hanyang University in Seoul, Korea, where we had been testing the game, so I positioned the dot there. Here’s what the screen looks like in the Fake Location app.

Image

Next, I ran my own app, and it was fooled into thinking I was in Seoul. Here’s my app, running on the device in Auckland, reading the GPS as normal, but getting fooled.

Image

Faking your GPS location may have all kinds of strange and sneaky uses, but from the perspective of testing location based mobile apps it’s a great tool.

Software Testing

Testing, Testing…

Jan 9, 2014 #Apple iOS bug, #LG television, #regression testing, #TDD, #test driven development by daveparsonsnz

I’ve recently been interested to see a couple of examples where the concepts of test driven development and regression testing were clearly not applied, or at least not applied as they should have been.

The first example was FaceTime on my old iPod Touch. Having worked perfectly happily for years, communicating with an identical device in the UK, it suddenly started failing, and for a while I was reduced to using the much flakier Skype. As you do, I stared surfing the web for answers to the problem. I soon found a growing number of people interacting on various on-line forums complaining about a similar issue. Clearly, the common denominator seemed to be a regression caused by a recent upgrade to the iOS operating system on iPod Touch 4th generation. Interestingly, the response from Apple, if the experiences of those on the forums were anything to go by, seemed to be one of total disinterest. However, a few weeks later, Apple quietly released the following fix for a problem that had affected ‘some users’ (i.e. many):

http://support.apple.com/kb/DL1701

Now I know, from painful experience, how hard it is to test for this type of fault, but Apple should have had regression tests in place for changes that might affect popular standard apps like FaceTime.

The other interesting example was this one, about LG televisions sending personal data back to the company, even when a privacy setting had been turned on:

http://doctorbeet.blogspot.co.uk/2013/11/lg-smart-tvs-logging-usb-filenames-and.html

Now you would expect, if this software had been test driven, that a test would have been written that ensured that activating the privacy setting meant that data could not be sent. Instead, data was still sent, along with an indication that the privacy setting was turned on. Either that wasn’t tested at all, or it was driven by a stupid test. The alternative interpretation, that LG wants your data whether you like it or not is, I’m sure, not the case.

These two examples are instructive because they emphasize how important it is to (a) drive your designs with unit tests and (b) avoid the entropy of your software with regression tests.

 

 

IT Education, Software Testing

Global Day of Coderetreat

Dec 18, 2013 #coderetreat, #Conways Game of Life, #pair programming, #TDD, #test driven development by daveparsonsnz

gdcr_ad_textGlobal Day of Coderetreat

Last weekend I participated in the 2013 Global Day of Coderetreat, joining the session running at the Xero offices in Wellington, New Zealand. Along with 2,000+ other software developers across 165 locations on all continents, I spent the day honing my software craftsmanship, pair programming with other developers, using test driven development (TDD), and working within a range of changing and challenging design constraints. A coderetreat is an opportunity to look at the same programming problem (typically Conway’s Game of Life) from multiple perspectives, without the pressure to create a finished product but using the opportunity to reflect on how we build software. I would recommend that all software developers, at whatever level of skill and experience, take a look at the coderetreat.org web site and keep a look out for upcoming coderetreats in their local area. If you can’t find one, why not run one yourself? All the information you need is on the web site. The only people who won’t gain anything from it are those who just want to show other people how great they (think they) are. Fortunately, these people are a tiny minority of the software development community. Most of us will embrace the opportunity to challenge themselves and learn from others, in a day of coding that is surprisingly enjoyable.

Software Testing

Testing mobile apps with geolocation and i18n

Jul 9, 2013 #geolocation, #Google Maps, #i18n, #Mobile app, #testing by daveparsonsnz

Testing mobile apps can be tricky, particularly if they use geolocation. Sometimes there’s no substitute for manual testing with a mobile device in situ. For the last couple of years we have been developing a mobile learning game that uses Google Maps to locate geotagged learning tasks. Late last year I was testing it at the University of York in the UK. Instead of the elegant new campus at Heslington East, all Google Maps could show me was a muddy building site from several years previously. I have just returned from a few weeks working at Hanyang University in Seoul, Korea. This time it was the zoom level available in Google maps that was the problem – the satellite pictures of Seoul on Google maps don’t have a suitable level of detail. A further complication in Korea was the task of translating all the game’s text resources into Korean. It was useful to prove to myself that UTF-8 really does work for i18n (internationalization)! However many of the tools on my laptop could not cope with the character set, so kudos to Microsoft Word, with its handy viewer that shows you what your document will look like using different encodings before you load it, and to oXygen’s excellent XML editor that picked up my markup errors. I was also reminded, after some painful debugging, that sometimes the only bulletproof way to render an ampersand (&) in XML is ampersand hash 038 semicolon (no matter how else I tried to write that it rendered it as an ampersand – foolproof!)

Finally, if you’re thinking I could test all that in an emulator from the comfort of my office, we simultaneously tested the app with three similar Samsung phones (after all we were in Korea!)  All three exhibited different behaviour, in particular in their response to the GPS signal. As I said, sometimes there’s no substitute for manual testing with a mobile device in situ.