Step Four - Testing
Software has bugs.
Even our most mature software programs absolutely refuse to deal with things that they weren't taught to deal with explicitly, and they tend to refuse in the most childish of ways. They babble incoherent error messages you don’t understand, and they do it at the most inopportune times – like just as you’re showing the boss the new software. Most of the problems usually come down to a single line of code somewhere with a tiny almost insignificant oversight in it – the programmer never imagined that anyone would ever enter a negative number in the payment field (it wasn’t in the SRS after all). Sometimes it’s just a typo, or a comma in the wrong spot. Given that most (non-trivial) software programs have thousands of lines of code in them, it is no surprise that bugs happen.
The job of our testing team is to find these bugs before you do.
Finding bugs takes a lot of time and a lot of hard work. Good quality testing takes about 20% of the time spent programming. Like good detective work, testing is hours of tedious leg work and occasional payoffs. Good testers push the limits – they push keys and enter numbers that no one would have thought of, and they exercise the software until it breaks. Then they record the bug in our bug tracking system, along with the steps needed to reproduce it, and continue their systematic pursuit of imperfections.
We use testers because programmers are the worst people to test their own software. Why? For the same reason most people are not good at proof reading their own writing. We all have blind spots that keep us from seeing obvious typos. We know what we meant to say, and we allow our minds to skim over words that we already know are correct – even when they’re not. Programmers get tunnel vision. They know how their programs are supposed to work and do not spend tons of time thinking of all the ways uninformed users might use their programs. And frankly, it’s really not their job. Writers need editors. Programmers need testers.
There is a healthy tension between programmers and testers. Programmers dream of delivering bug-free code so they don’t have to suffer the indignity – however modest – of having to watch their programs crash. But they are also secretly grateful, because they will receive praise down the line from happy customers when their program works as expected.
We employ full-time testers who have been trained in the disciplines of regression testing, integration testing, use case development, test plan development and test plan execution.
We don’t want to pay for testing. For various reasons, some clients decline testing. This choice shifts the burden of testing from us to the client, but unfortunately doesn’t change the number of bugs. It only changes when and where they get found. To be clear, no one likes bugs – they are no fun for programmers or clients. We try very hard to write bug-free code, and if the client doesn’t want to pay for testing, we will try even harder to find bugs in the time we have. But as we mentioned previously, all programs have bugs. They are a natural part of the life cycle of all software. The choice is whether to pay for testing upfront, or pay for more bug-fix time later on.