Consider the situation:
A company develops embedded devices, operated by a remote control. The remote has gone through very detailed testing and is assumed to be error free. And it has been for over 10 years of development and testing. Good. Very few updates have been made to the remote over the time, very minor changes in wording, etc. Again tested, again assumed error free and no problems are found. Good.
Now there is a significant change to the remote, how the internal system is configured and such. Visible changes to the end-user would be minimal. We analyzed the situation, planned and executed tests, learned, tested again, etc. Everything we have tried still works. But we are not satisfied that we have tested enough. So the following situation takes place :
Tester : “What could we test ?”
Me: “Well, we have covered what we thought of already, maybe we should try another angle ?”
Tester : ” Hmm… maybe we should just do what most users would do ?”
Me: “Yeah, so change programs up/down” and added jokingly ” or just change volume up and down”
As I said that I had the remote on the table, pointing at a logging device, and started grotesquely pressing vol-up and vol-down buttons one after the other. We both laughed. Ha-Ha.
Few moments later, when I had already stopped, I noticed that the logger was still showing incoming signals. What the hell ?! Did I do it so fast that I overloaded the buffer and its now showing me it ?! We waited for a minute, but the signals still kept coming. I rebooted the logger, thinking it must have crashed – it is after all an old program. That also did not help.
I covered the remote so that it’s IR signals would not reach the logger – And the logger stopped showing incoming signals.
After some investigation it turned out that when pressing the volume buttons with certain timing the remote went into complete deadlock – and kept sending out signals forever. Developers did some code sniffing and found out that the problem was introduced about 4 years ago !
The Morale :
First: Joking is very good defocusing technique. Feeling free to say ridiculous things might trigger very good testing ideas.
Second: Logging tools are invaluable. Yes, they can ba nuisance to use or maintain, but they are Huge benefit.
Third : When designing the tests have a product at hand to try spontaneous ideas.