Yep. And they love it. Especially the ones who have hard time solving it.
( If you have not played the dice game and want to play it, this might spoil some of the game for you.)
So what have I done now ?
First I challenged a guy whom I have shared puzzles before. I knew he would desperately want to solve it and he did brilliantly. I talked him through the traps and skills. He was satisfied but I was not.
I wanted all of the developers to play (at least those who want to learn). I wondered a bit about ‘how to get them to want to play the game’. In the mean time I asked another guy to play, but he was busy working. Then another one, who agreed.
I wanted to keep the solution a secret, so I guided him to the comfortable chars at a corner of the office, where no-one could easily overhear us. Which was a lucky move, as it turned out. People passing by saw us playing with odd-looking dice and got curious. After a few games the word had started getting around. During the games I saw people poking their heads around the corner to see for themselves – what are we doing there. And the guys who had finished the game were all secretive, but positive about it. Most of them saying to me that all developers should play the game. And soon enough developers started to ask me : “I want to play too, when to do you time ?”
So I got what I wanted.
So what exactly I did with them ?
I used the first three classic Dice Game patterns (which I shall not utter here). First I said this is a kind of ‘debugging situation’ and explained the rules. Then, after the first round I commented on what they did good (naming their techniques and explaining them in general) and highlighted the traps they fell into. And then explain the skills of focusing and de-focusing. Another thing I suggest is that, when they have hypothesis of the algorithm – they should try to test against it. ( This is harder than it sounds in some cases, but for example, if you think I count the sides of the dice, show me the dice so that I could not see any sides and see what do i do then.)
For the second pattern I intentionally rearranged the pattern to exclude the ones they had already used (easy to do if you have lots of different dice). After this round I commented again on what they did and explained the problem with inputs. All the possible dimensions of them.
The third pattern ended with talking about outputs – that there are more than the Real Number I tell them – the way i look the dice, how long i wait before answering, etc – to show that they not knowingly used that information. The same way any system could be monitored. During the game I tried to mislead them too, but not too much, by looking at the dice from different angles or taking a bit longer to answer than normally, and such. So I explained that too and made the point that they should not relay too much on that kind of information.
During the game if they got stuck for too long, i started asking questions to point them in the right direction. For examples, ‘what do you already know‘ , ‘how many different tests have you tried‘ , ‘ have you considered all the input dimensions‘, ‘what does your test data tell you‘, and so on.
So how did they do ?
Variably. And surprisingly. The following are some notes I have taken.
- Every time I thought ‘now I have seen all the ways for people to get stuck‘ – they came up with a new one. Brilliant – more things for me to learn and help others to learn.
- Statistically, so far, from the 15 (or so) games, the average time is around 90 minutes. But it depends on how eager they are to solve the problem – if they look really desperate then I started sooner with helping questions.
- Most problematic are the first and the third patterns, seldom the second one.
- There are very few people who write down the number they tell me, and those people generally solved the third one really quickly.
- There have been two guys who discarded the rules from the start, but one of them still could not figure out the first pattern in 30 minutes (with the special dice).
- Most number of tests that have been done is – irrelevant – more than 280 000. How, you ask ? By using scripted testing.
- Some people do mistakes when writing down test data – and then fail to re-test them, even when they see that only that input is inconsistent with all others.
- The assumptions are very strong – one guy assumed that the number he tells me has no importance ( actually, I think that he thought that the number is not an input, but a guess of what I would say), and was constantly surprised during the third pattern when giving numbers like 35623, and then hearing that he had been VERY CLOSE to the real value.
- I have seen different tactics with tests: stacking the dice, turning them, hiding other sides, etc
- People are affected by their daily work – one guy took 5 minutes between rolling dice at the beginning of the first pattern. Wierd, huh? But whenever he has to do test his own code it takes him 15-30 minutes to see the results, so he is used to really thinking through all the tests he does. He did solve it quickly after he got around that assumption.
- Interrupts really confuse some people – any distraction from the game and they need time to get back into it, sometimes more than few minutes. This is very visible if they did not write down their test data.
- and so on.
I’ll post an update, when i have played it with all of them.
What about the Testers ?
Well, to be honest, I have not played it with them yet. I’ll do it later and be much more demanding with them, and intimidating. They have more experience as testers and we have been through a number of different testing exercises before, and I have trained them in the art of rapid testing too. I’ll post about that too.