Category Archives: Branding software testing

Explaining exploratory testing with a table

Tables loaded with food and a class of kids playing on a lawn.

Another dad and I picked an all favourite Dutch subject: work.

“What makes a good tester?” the other parent informed.
“A good tester knows about exploratory testing.”

I saw wrinkles on his forehead. This was a bad start for this subject. I had to switch to his context. He was a police agent. Okay, second try.

“Suppose you ask for a driving license.”
I opened my imaginary jacket and pulled out an imaginary object.
“I place a gun on the table and”
I noticed a sudden sharpness.
“then I show you my driving license.”
This time I retrieved a thin imaginary object between my thumb and index finger.

“Would you be interested in the gun?”
“Yes, of course.”
He was constantly switching attention between my hands and the invisible gun on the table.
I continued with
“I would ask questions like
“Do you not feel safe?” or
“Is this your gun?””
He nodded.

Then I explained that a tester adjusts her or his activities based on observations during exploratory testing.

The focus would be on the gun instead of the driving license.

Test Twilight Zone

When I grew up, there was a TV program called The Twilight Zone. It was about people getting in really strange situations. Logic and laws of nature did not seem to apply.

The reason it appealed to me was that it could happen to me, the writer. Ordinary situations became unordinary situations.

This TV program had a tune. There are 3 notes which I still remember, to create a gloomy atmosphere. Those 3 notes became the tradenark of this zone.

Then the voice over would start like:
“This is a story of a woman. Let’s call her A. A is a very good UX designer or User Experience designer. She takes care, that people can use a program on first sight.

If she would design a kitchen tool, she could easily skip the manual. The product is so intuitive the moment you see and touch it.

She thinks that this is normal for her and not for other people. But she is about to enter the Twilight Zone.”

UX

A looked to me and asked me:
“How many bugs did you find?”
I mentioned a number above 40.
She swallowed.

I had no specs and not enough domain knowledge. Only a briefing from my PO or Product Owner. Still I had found some strange things.

She said that she had found about 20 bugs.
“What did you find?” she asked.
I described some bugs I had found. There was a bug with an input validation. I had just enough domain knowledge to point this one out.
I told about the details of the elements, which had confused me as a user. And …
“You are doing UX.”, she exclaimed.
“I am only testing.”

Then the voice over would come back: telling about the UX designer’s meeting. Mentioning the morale and The Twilight Zone.

“Next.”
“Huh.”
“Next story.”
“There you read” => PM

Then the voice over would start with:
“Mister X has more than 10 year of experience in “.

Let’s skip the scary part.
Okay with you?
Works on my web site.

Mister X and I were sitting at a table. We were talking about business on Sunday. That is typical Dutch by the way. Somehow the word migration was dropped and I could not stop myself to tell my deployment plan story.

My telling triggered the attention of Mister X. I told how I merged 3 plans and how I set up a meeting to discuss the result. X asked familiar questions which I had already covered in 2 piece Q and A.

“What was your role?”
“I was a tester. I wanted this project succeed, so I became the chairman.”
[Actually I made a mistake: I was a test coordinator. But still …]
Ten minutes in my story I heard:
“You were doing project management!”

At that moment I made contact. The next time he would remember my story about the migration. I would not be another faceless tester, which is to be avoided according to James.

PS

During a meeting last week I spoke up:
“I do not have a clue, but I have a weird idea.”

I almost forget the tune.

Finding courage

After more than 10 minutes of discussion the work item was clear to the developers and the product owner. Then the standard question was posed: “Are there any more questions?”

As a tester I had digested the information. I was not sure about the solution, so I raised my hand. Everyone looked at me.

What’s Up, Doc?

Currently I am the only tester in my team. If something has impact on testing or a quality related attribute, then I talk about it. It is something some people take for granted.

In the past people started rolling their eyes, if I questioned something. Until the main stakeholder supported me. Look who’s talking?

A few years ago I heard about a demo for a certain project. I tried to invite myself. My project leader objected with: “There is nothing to test.”

I persisted and attended the demo. Every now and then I posed a question. After the demo I heard no more objections about my presence.

Invitations for the remaining demos were even sent to me. The stakeholders obviously valued my input. Look who’s listening?

No harm intended

Last month the Skype rehearsal was not that successful. I had one month left to improve the exercises. They were crucial for my workshop at TestBash NL.

During the session I zoomed in on some exercises. In hindsight they were too big to handle in 1 go. Agile people might call them epic.

By breaking them down I got digestible mini exercises. I liked the idea.

Fast feedback for me and you.
[On the melody of Tea for two]

Some exercises had the complexity of my daily work. Using simple tests I might overlook edge cases. So let me complicate things please.

At the beginning of this section I wrote that the exercises were not that successful. I expected that the exercises went more smoothly than experienced.

Luckily there was useful feedback to improve the exercises. I had something to act upon. Things could only improve now. Also by writing down my thoughts and actions in this post.

During the preparation of every talk or workshop of mine there is a moment I think: “I cannot tell this.” And then the presentation is getting better. These experiences form my word of comfort.

The Skype rehearsal reminded me of #30daysoftesting: Lisa Crispin had doodled about experimenting. She was struggling, how to fit it in.

I tweeted her:
“There is no failure. There can’t be, if your only mission was to “see what happens”. ”
@sivers

An Oracle, Two Systems, And Four Situations

On September 26, 2015 Justin Rohrman ‏tweeted:
“Anyone know of the origin of the word ‘Oracle’ in software testing? Who started using it, what other words were used prior, …”

I answered with:

  • in the past in the Greek city of Delphi there was a special person you could ask question: Oracle.
  • The answer was sometimes cryptic. When the Persians invaded Greece, the answer was to hide behind
  • wooden doors. There was a smart man, who interpreted the answer right. He advised to build
  • a fleet of wooden ships, which beat the Persian fleet.

An oracle is a source of information, which I can use during testing. It can be used to determine, whether the observed situation is right. Right?

A dialogue from a movie with John Cleese about choosing a direction (while driving a car):
“Right?”
“Right?!”
“Left?”
“Left!”

Are you sure?

“Hey, I think I found a bug.”
“Okay. Tell me about it.” The other tester requested me.
I told as objective as possible my observations and my conclusion.
“Did you look in the previous version?”

Some systems are so complex. A fast way to sift possible bugs was to look at the system used in production. Of course afterwards it was always possible to read the specs or ask the PO, Product Owner.

One tester was able to pinpoint the version, where a bug was introduced, within minutes. He had the latest versions of the system available.

So I learned,

  • how to install more  than 5 different releases on my PC.
  • how to rollback a release including the database.
  • how to prepare the proper database and configuration for testing.

Where did you see it?

During the test I noticed, that the test version showed status information, which was not shown in the release in production. I asked other testers, who noticed the same. So my test environment was OK. There was still a doubt, whether the customers would miss the information.

The most simple way was to ask help from the help desk.

In the afternoon a blond woman was standing at my door post:
“You are right. The situation occurs in production.”
It was time to make a bug report.

Just don’t let it happen again

“I have not enough information to test the issue.” I complained.
“Did you test in the previous version?”, another tester remarked.
“No.”
“If you reproduce the steps from the issue, then you notice, what will go wrong. The issue has been solved, if the observed situation does not occur.”

During one project I had to test a system with service desk agents and PR colleagues. There were issues to be tested. I did not have time to make test cases for them.  If I let the testers test on their own, there was a chance  that they would check, whether only the bug was removed. I needed information about similar situations and connected cases.

I opened an issue in the defect registration system and read the description. “I would test this …” Then I wrote down the test ideas in the comment. Before the test I gave some instructions to the testers:
“In the comment situations are described, which must be tested. If you are ready, write in a comment, what you did. Then you assign the issue to me.”

After the start of the test the issues came back to me. I looked at the feedback. Sometimes I realised, that more situations had to be tested. Then I added them to the comment of the issue and assigned it back. The nicest thing, that happened, was that extra test ideas were added by the other testers.

Did you have a good look at it?

During the regression test I had to test the export and import function. I had two systems installed with different databases. The only things, what I did, were export data to a file and import the file in the other system.

I had two screens, so it was easy to compare the data: I placed the systems on different screens. If there was a difference between the source and target system, then I made a print screen. The picture contained images of both screens. Hey look mom! With one finger!

Next stop was my word processor, where I pasted the image. Then I used the keyword BUG followed with a small description. Sometimes I analysed the data file to pinpoint the source of the strange situation. If the file was wrong, then the source system might have some glitches. Otherwise it was the target system.

During the stand up I mentioned, that I had found a number of particular situations. This resulted in a session, during which I spoke with a programmer. The following pattern was repeated:

  • Search for “BUG”.
  • Resize the image to show the differences between the systems.
  • If necessary, look in the file.
  • Determine, whether it was a bug.

During the bug removal phase I got two kinds of feedback from the programmer: it was fixed or it cannot be fixed. The latter surprised me, but a study of the specifications changed my view. The format of the file did not support all types of information. There were semantic limitations. Something like: country is not supported, so this cannot be exchanged.

Does it fit?

Fancy restaurants have little delicious dishes and standard spoons, which might not fit.

Take that

“We cannot combine the automated test and the performance test.”, Scott stated. “They don’t fit. It’s a waste of time.”
Alice looked hopeful.
George brushed it away:
“Instead of testing you are talking. You’re the one, who wastes time.”
Scott was a man, who did not mind a good discussion:
“We’ll lose a lot of time, if we do not split the tests. ”
“My decision is definite.” George answered.
“Don’t you understand?” Scott shouted.
Alice started to look white.

“You can leave the room NOW.”
“You cannot run away from decisions.”
George stood up:
“You bet. I will have a little talk with HR right now.”
He went out the room and slammed the door.
Alice looked miserable.

The door opened slowly. George looked inside and said:
“That did not go quite right.”

Take Me To St Louis

“Scott, do you know a person you never shout at and who knows as much about automated tests and performance tests as your manager?” George asked.
Scott was silent for a few moments.
“My mother. ….!? Wait a sec. You are not involving my mom in this discussion!”

“Hi mom”, Scott began.
“What is the matter, son?”, George answered.
“I want to thank for the apple pie you made for me.”
George raised an eye brow, Alice smiled.
“Did you come to talk only about the apple pie? ”
“No, for my work I have to combine automated tests and performance test.”
“I am sorry, son, but I cannot help you with this. Just do, what they told you.”

Scott was shifting gears in his head.
“A performance test for a car is to look, whether it can handle the load.”
Alice put a thumb up.
“I understand: if I can put all ingredients for an apple pie in the car, then it can handle the load.”
“No mom, that is the purpose of an automated test. That one is focused on the functionality of the car: can I open the door, put the ingredients in the car, close the door, and drive away?”
George looked puzzled:
“What kind of load do you mean?”

George nodded.
Scott made another attempt:
“A load for car software are heavy conditions. Does the car ride well with four adults, luggage for long holiday on slippery roads in a dark rainy night?”
“I only fetch my apple pie ingredients in my car during daylight, when the weather is good. ” George explained.
Scott threw his arms up in the air.
A frown appeared on Alice’s forehead.

Take 5

“You are close.”, George assured Scott.

“You know, mom, that a car has a lot of software.”, Scott stated.
“You say so, son.”, George answered. Scott continued with:
“The dashboard must show the right information at the right moment, if I am speeding on the highway. In this case the computers in the car must work very hard. That is the purpose of the performance test.

An automated test can be used to check, whether all buttons and displays of the dashboard work properly. If I drive fast, I am not using every button on the dashboard. So the automated test and performance test cannot be combined. It does not fit.”
Alice put her right hand on her chin.

“Thank you for the explanation, son. You definitely earned an apple pie.”
Alice started laughing merrily. Scott joined in.

After Alice had stopped with laughing, Scott looked at Alice:
“Do you need more arguments, Boss?”

Alice sobered up. She got a focused look:
“You know, that there is a lot of pressure. So I tried to take a shortcut. I notice, that you have a lot of anger. I assume, that you have a need to be understood. Am I right?”
Scott just nodded.
“This basically means, that a choice must be made between quality and functionality. “, Alice continued.
“At the moment Quality has the best chances.”

Alice turned away from Scott.
“George, I really want to thank you for ..”
George’s chair was empty. He had left the room.