It was a bad weekend for me, my fantasy team lost, with regards to the fantasy team, it has shown to me that no matter how well you prepare, how well you plan, there are things that are going to happen that you just can not predict.
Which leads me onto the title of this post, you can never guarantee bug free software. You can highlight the areas of risk and highlight what you have and haven't tested, to make business owners aware of what issues may arise, but you just can't guarantee bug free software being released. There are far too many variables that affect what you can and can't test unfortunately. Software is an extremely complex system, and this is ignoring hardware, which makes it even more complex.
Going to an extreme, NASA have shown that it's possible to make "virtually" bug free software, but mistakes are still made, which are sometimes unfortunately, fatal.
One solution is, as I said, to highlight the areas of risk, highlight what you have tested, and highlight possible things that you deem low risk, and make the business aware of the implications of what you ahve and haven't tested, as unfortunately, there isn't time to test everything.
Something that definitely helps in highlighting areas of risk is a good set of requirements, we don't always have this, but if we don't we need to say early and often.
Another thing that helps, is TDD, by testing early and testing frequently, you are far more likely to catch bugs before they enter production. It also encourages developers and QA to ask questions about requirements, so definitely feeds into the above paragraph too.
Every once in a blue moon, you may get an issue arising that you had marked as low risk, but this is life, it is full of unknowns. What you mustn't do, is completely change your approach, you are a good tester, you need to stick to what you know, and probably tweak your approach so that the issue or something similar doesn't happen again, but it would be foolish to throw away the game plan that has proved successful in the past. The important thing is to learn from it.
This reminds me of an interview I once held, I asked "What's the biggest bug that you've let go into Production?" to which he replied "None", I immediately thought this was strange, we've all made mistakes, the important thing is how we learn from our mistakes, and ensure that it doesn't happen again.
Which is what I am going to try and do with my fantasy football team, I'm going to stick with David Wilson, stick with Dwayne Bowe, but tweak the lineup slightly to ensure that I have the best chance of success in Week 2!
Which leads me onto the title of this post, you can never guarantee bug free software. You can highlight the areas of risk and highlight what you have and haven't tested, to make business owners aware of what issues may arise, but you just can't guarantee bug free software being released. There are far too many variables that affect what you can and can't test unfortunately. Software is an extremely complex system, and this is ignoring hardware, which makes it even more complex.
Going to an extreme, NASA have shown that it's possible to make "virtually" bug free software, but mistakes are still made, which are sometimes unfortunately, fatal.
One solution is, as I said, to highlight the areas of risk, highlight what you have tested, and highlight possible things that you deem low risk, and make the business aware of the implications of what you ahve and haven't tested, as unfortunately, there isn't time to test everything.
Something that definitely helps in highlighting areas of risk is a good set of requirements, we don't always have this, but if we don't we need to say early and often.
Another thing that helps, is TDD, by testing early and testing frequently, you are far more likely to catch bugs before they enter production. It also encourages developers and QA to ask questions about requirements, so definitely feeds into the above paragraph too.
Every once in a blue moon, you may get an issue arising that you had marked as low risk, but this is life, it is full of unknowns. What you mustn't do, is completely change your approach, you are a good tester, you need to stick to what you know, and probably tweak your approach so that the issue or something similar doesn't happen again, but it would be foolish to throw away the game plan that has proved successful in the past. The important thing is to learn from it.
This reminds me of an interview I once held, I asked "What's the biggest bug that you've let go into Production?" to which he replied "None", I immediately thought this was strange, we've all made mistakes, the important thing is how we learn from our mistakes, and ensure that it doesn't happen again.
Which is what I am going to try and do with my fantasy football team, I'm going to stick with David Wilson, stick with Dwayne Bowe, but tweak the lineup slightly to ensure that I have the best chance of success in Week 2!
Always an important thing to make sure non-testers (heck, maybe testers too) are aware of. My turn of phrase when asked is usually we can "make quality more likely" or we can "make critical issues less likely". Maybe weasel words to some, but it's the truth!
ReplyDeleteFor assessing risk areas in critical systems I apply Failure Modes and Effects Analysis (FMEA) on occasion. Then build tests around those we identify, along side the usual Functional type testing of course.
Mark.
Awesome article! I have gradually become fan of your article and would like to suggest putting some new updates to make it more effective.
ReplyDeleteMAC Address Spoof