Dr David Parsons

davidparsons.ac.nz

Tag: regression testing

Software Testing

Why regression testing matters

Sep 28, 2014 #Apple iOS bug, #regression, #regression testing by daveparsonsnz

Quite some time ago I posted a whinging blog about the software I have to use a editor of a journal. Over the last couple of years I have to acknowledge that the occasional improvement has occurred. For example it is now possible to see how many review requests each reviewer has responded to, and the dates of the reviews they returned. This is great, though one issue is that these changes just appear, without any warning. At least when your phone updates its apps, it does at least tell you it’s doing it, even if it doesn’t bother to ask you if it’s OK unless you have to approve various permissions.

Anyway, to the point of this post, which is regression. Of course the term has a number of meanings. I’m not referring here to sexually disturbing regression in Freudian analysis, or the statistical process for estimating the relationships among variables. I mean, of course, regression in the software sense, where an update to software reveals bugs that were not there in the previous version. A recent update to the journal software saw a complete change to the appearance of the main web page. Whether this was an improvement or not from an aesthetic or functional perspective I’m not sure, but the main thing I noticed was that the system has now forgotten all of the authors of all the papers submitted to the journal. This struck me as a not unimportant feature of a journal submission system.

The point of the above observation is of course that it highlights the great importance of regression testing. The danger of software maintenance is always that it may inadvertently break something that worked perfectly well before. Without regression tests you may not find out what you have broken until it is too late (e.g. you have forgotten every author of every paper in your journal submission system.)

Similar stories have been circulating about the rather more high profile Apple iOS 8 operating system update, which among other failures prevented people from making phone calls, still surely the main point of having a mobile phone. http://www.theguardian.com/technology/2014/sep/24/apple-ios8-update-withdrawn-iphones-bugs
Not the first time an Apple update has broken a major feature, as I discussed in a earlier post about problems with FaceTime.

Effective and comprehensive regression testing is not easy, and not something that can be tacked on at the end of the testing process. Rather it has to be part of a full testing strategy that starts with unit testing, and follows though integration testing, acceptance testing, performance testing, exploratory testing, security testing etc. However it is the last barrier between the software bugs and the customer, and deserves more attention that it appears to be getting from some software development teams.

Software Testing

Testing, Testing…

Jan 9, 2014 #Apple iOS bug, #LG television, #regression testing, #TDD, #test driven development by daveparsonsnz

I’ve recently been interested to see a couple of examples where the concepts of test driven development and regression testing were clearly not applied, or at least not applied as they should have been.

The first example was FaceTime on my old iPod Touch. Having worked perfectly happily for years, communicating with an identical device in the UK, it suddenly started failing, and for a while I was reduced to using the much flakier Skype. As you do, I stared surfing the web for answers to the problem. I soon found a growing number of people interacting on various on-line forums complaining about a similar issue. Clearly, the common denominator seemed to be a regression caused by a recent upgrade to the iOS operating system on iPod Touch 4th generation. Interestingly, the response from Apple, if the experiences of those on the forums were anything to go by, seemed to be one of total disinterest. However, a few weeks later, Apple quietly released the following fix for a problem that had affected ‘some users’ (i.e. many):

http://support.apple.com/kb/DL1701

Now I know, from painful experience, how hard it is to test for this type of fault, but Apple should have had regression tests in place for changes that might affect popular standard apps like FaceTime.

The other interesting example was this one, about LG televisions sending personal data back to the company, even when a privacy setting had been turned on:

http://doctorbeet.blogspot.co.uk/2013/11/lg-smart-tvs-logging-usb-filenames-and.html

Now you would expect, if this software had been test driven, that a test would have been written that ensured that activating the privacy setting meant that data could not be sent. Instead, data was still sent, along with an indication that the privacy setting was turned on. Either that wasn’t tested at all, or it was driven by a stupid test. The alternative interpretation, that LG wants your data whether you like it or not is, I’m sure, not the case.

These two examples are instructive because they emphasize how important it is to (a) drive your designs with unit tests and (b) avoid the entropy of your software with regression tests.