This summer I went to an extraordinary museum about cars. In the Louwman Museum in The Hague cars, which are landmarks in the history of the automobiles, were shown. The range was from motorized carriages to more familiar cars on the road. The owner had also a special interest in strange cars. I discovered, that electric and hybrid cars were already used at the beginning of the 20th century. Then a piece of furniture drew my attention: the desk of Dr. Toyoda. He worked for Toyota, which has a special approach for Lean Management.
For the test project I had two iterations, which took 3 weeks each. My planning for an iteration was simple: two weeks for the functional tests and the last week for the user acceptance test. To be more precisely 40 hours in the last week for 3 end users. I mailed the test plan to everyone and I got polite decline from the end users. In a normal week they had each 8 to 12 hours left in a week. So ideally 36 hours of testing would be filled in. But it was still too short. Overwork was no option. It was time to reschedule the activities.
A few months before I had bought The Toyota Way. It was mentioned several times by experienced Lean practioners, so I bought after browsing it. Heijunka, Level out the work load, came to my mind. If I somehow could move the testing hours of the end users towards the beginning of the test iteration, then the planning problem would be solved. Suppose function A was successfully tested during the Functional Acceptance Test (FAT), why should I wait to let this function tested by the end users? If FAT went well, then the first functions could be tested by the end users in the first week of the iteration. So I made new calculations: in worst case I have 3 end users, who have 8 hours left in 3 weeks, then I come to a total of 72 hours of testing. The new test planning was easily accepted by the end users.
“How much time do you need to test this? “ my manager demanded.
“I cannot give an estimation, because I do not know, how it has been corrected.”
Annoyance came in her voice:
“Han Toan, what are you talking about?”
A pressing silence followed.
I looked around for a simple object to make my message clear. A powerful trick I learned from a senior business consultant. My eye fell on the door.
“Suppose, that this door handle had been broken. The supplier fixed it.”
She nodded slightly.
So I continued: “What would you test, if they only replaced the door handle?
The door? The room? The floor? The wing? The building?”
The atmosphere changed in the room: she understood it.
At that moment I did not need to use plan B:
“If the building has been replaced, would you test the structure of the building before testing the door handle?
If the complete wing has been replaced, would you check the water pipes and electrical wiring before testing the door handle?
If the door has been replaced, would you also check the lock?”
At the beginning of this year I was in a restaurant. It was possible to buy a drink. So the software, which took care for the supply, was good. Also the training of the people was good, because I bought more than I intended to. There was one problem: how can I empty the bottle with this straw? My first thought was: did you test it long enough? As a tester I have heard many variations: What were you testing? How did you spend your time? Or did you pay attention during testing?
The sound of nuisance
At eleven o’clock in the morning George joined the meeting of the software testers.
“Good morning. My name is George. Your boss asked me to help you. I know a lot about software testing, but not much about cars. “
“Hi George, my name is Jack. At the moment I have an important problem with the tires of sold cars. The tires wear off faster than it was the case with the previous models. So car dealers began calling to the head office. And my boss got the question: “Did you test it long enough?”“
George looked confused: “What is the relationship between tires and software? “
“Nowadays everything is computerized. The automated suspension takes care for a comfortable ride, so not every stone in the road is felt by the passengers. There is something wrong in the software of the suspension. We were able to pinpoint the problem: it is the combination of a wet surface and steep road. “
“What did you do with this information?”
“I went to my boss and I explained the problem. Up till now the suspension software was one of the most reliable parts of the system, so it was not tested intensively. We tested the cars on a wet flat road and a dry steep road. We did not test the on a wet and steep road, because time is money. “
“What was your answer on the question, whether you tested it long enough? “
“I told that at the end of the test all criteria were discussed in depth. Every stakeholder was involved. At that moment it was the best decision we could take. Furthermore I suggested to change the criteria for the suspension software. He agreed with me on the spot.”
´Why did you change the criteria?”
“Because the tires wore off.”
“What you are actually saying, is:
“I wait, until something unpleasant happens. Then I change the criteria.”“
“Is there a way to change the criteria before testing? “
“Sure, I checked the release notes beforehand. I noticed, that some components had been changed. But the functionality was basically the same. “
“Did you determine the impact of the changes of the components?”
“No, because the quality of the software was good and the functionality did not change.”
“The supplier probably did some refactoring. So the basic questions you should pose as a tester are: What did you change? and how can I test it? Then you can determine, whether the criteria must be adjusted and extra tests must be added.”