Category Archives: Test automation

Fast Forwarding Arrange

At that moment I had a test, which would always pass. This was not the right starting point.

Putting

The first phase of a test is to arrange, that things can be tested. Somehow I had to land on the Login Page on https://the-internet.herokuapp.com/. In other words put the automation tool on the same page. And off I coded.

What is a way to identify this screen?
Okay, the header is “Login Page”.
Now I can use Red, Green, and Refactor to make the final step to this screen.

Red or making a failing test was simple, because nothing could be seen.

@Test
public void testSuccessfulLogin(){
  assertTrue(loginPage.getHeader()
               .contains("Login page"),
               "You entered the wrong page.");
}

Green: now I had to make enough code to pass the test. The code was quite simple.
In LoginPage.java I made a method for the class LoginPage:

public String getHeader(){
  return "ran394594";
}

Refactor: the code looked simple. So there was no need for cleanup.

There was a drawback. I had a continuous passing test. This reminded me of the last unsuccessful attempt. In that case I started with a failing test of the Assert. Also in this case I ended with test code unconnected with the code under test.

Things

Then I realised that I had repeated the same wrong steps. This walk was clumsy. I started at the end of the Arrange and wanted to walk back with Red, Green, and Refactor.

In the upper part there is a rectangle "Arrange". Under this rectangle there are two groups rectangles "Red", "Green", and "Refactor" under an arrow pointing to the left. The left group is lower than the right group!

A lot of people state that you should start with the desination and then try to figure out what you need to get there. I had the feeling that I was rather awkward.

Let us assume that I have to provide some information to a wondering visitor of The Netherlands.
Imagine me walking backwards from the Palace on the Dam to the Rijksmuseum in Amsterdam.
They might mistake me for being a tourist. I hope.
Better to take some extra pictures. A habit of tourist. I think.

In the right order

It was better to start with Red, Green, and Refactor at the beginning of the Arrange. And from there I would slowly walk in the right direction.
Under the rectangle Assert there are two arrows RGR pointing to the right. The left RGR is higher than the right RGR!

So I needed to have code to get to the home page. But LoginPage.java extends BaseTests.java. This basically means that all code in BaseTests.java also applies to LoginPage. I discovered that the code to get to the Home Page was already present.

@BeforeClass
public void setUp(){
  System.setProperty("webdriver.chrome.driver", "resources/chromedriver.exe");
  driver = new ChromeDriver();
  driver.get("https://the-internet.herokuapp.com/");

  homePage = new HomePage(driver);
}

My first step was to put the code with getHeader in a comment.

@Test
public void testSuccessfulLogin(){
  /*
  assertTrue(loginPage.getHeader()
               .contains("Login page"),
               "You entered the wrong page.");
  */		   
}

Red: now I had to test that the Login Page was not shown. There was no code in place, so my test would always fail. I checked with my own eyes, that the Login dialog was not shown

Green: then I made some code to get there

@Test
public void testSuccessfulLogin(){
  /*
  assertTrue(loginPage.getHeader()
               .contains("Login page"),
               "You entered the wrong page.");
  */
  LoginPage loginPage = homePage.clickFormAuthentication();
}

I also needed some code for the method clickFormAuthentication in HomePage.java.

private By formAuthenticationLink = By.linkText("Form Authentication");

public LoginPage clickFormAuthentication(){
  driver.findElement(formAuthenticationLink).click();
  return new LoginPage(driver);
}

I executed the code again. And I saw the Login dialog.

Refactor: the code contained some comment with old code.

@Test
public void testSuccessfulLogin(){
  /*
  assertTrue(loginPage.getHeader()
               .contains("Login page"),
               "You entered the wrong page.");
  */
  LoginPage loginPage = homePage.clickFormAuthentication();
}

I deleted the comment:

@Test
public void testSuccessfulLogin(){
  LoginPage loginPage = homePage.clickFormAuthentication();
}

Also I noticed, that there were only manual checks and no automatic tests in the code of login page. Somehow I let my attention slip away. Again.

It is time for a flash forward. A few practice runs later:

@Test
public void testBackspacePressed(){
  // Arrange
  // String header  = "Random text";
  var keyPressesPage = homePage.clickKeyPresses();

  String header  = keyPressesPage.getHeader();
  Assert.assertEquals(header, "Key Presses", "You entered the wrong page.");
}

Why should I bother to use Red, Green, and Refactor so intensively?
This was the only way for me to fully experience this new way of developing. Later on I could always change things.

Okay, back to the point where I was trying to test a login form.
There I was. Ready to make actual tests.

This exercise was so small, that I started at the wrong point twice.

  1. At the end of the Assert
  2. At the end of the Arrange.

The whole way of programming in the right direction was still not a natural thing for me.

Somehow I forgot the time being a test coordinator for performance tests. My team members had to make scripts for complete unknown websites. My task was to simplify this by asking for click paths.
“Would you please provide screenshots, which are shown to to the user, and the input of the users?”

Another side note

In music there is a saying that the tape does not lie. If you are a good musician, then the recording will prove it. For software development I could translate it to “the code does not lie”. For doubtful people I could change it to the “the version control system does not lie”.

I could write about an error free test automation experience, which is not real.
My coding in TDD in test automation was still in progress. I was learning.

My next blog post is about fixing the Assert.

Fast Forwarding Slow Down

On the day of my talk about TDD in Test Automation the meetup was postponed.

Painting

One month later I was still exploring Test Driven Development or TDD. The talk is still available. In this blog post I will write about my latest experiences.

Test automation is about programming and TDD is about programming. In a talk Kent Beck told about crazy ideas, which would bring him laughter. TDD in Test Automation made me laugh. What was holding me back?

“What is holding me back?”
Kent Beck about Explore at YOW! Conference 2018.

In TDD there is a continuous loop of Red, Green, and Refactor. Red is writing a failing test. Green is writing enough code to let the test pass. Refactor is clean up the code.

Refactor is like something like “Let me put that recurring piece of code in 1 method”. This sounds easy, but it is still tempting to optimise during coding or Green. I already made the optimised code, so why should I spend more time on refactoring? There might be a chance that I miss 2 out of 3 improvements.

A post it with the text Red posing to the post it with the text Green, pointing to the post it to the post it "Refactor". The Refactor post it points to the Red post it!

Myself

A common pattern in test automation is Arrange, Act, and Assert. The first phase is to arrange that everything is ready for testing. E.g. I am logged in with the right user name and password on the right website. The second phase is to act or do something, which must be checked. E.g. I have to press on this button, so a dialog will pop up. The third and last phase is to assert or check, whether the result is right. E.g. I can see a dialog.

At that moment I had two patterns to work with:

  1. Arrange Act Assert
  2. Red Green Refactor

On a high level I used Arrange, Act, and Assert for the general structure. On a low level I used the Red Green Refactor cycle to make the code. So the Assert block contained several Red Green Refactor cycles. A cycle can also be shown as a serie of actions.

In the upper part there is a rectangle "Assert". Under this rectangle there are two groups rectangles "Red", "Green", "Refactor". The left group is lower than the right group!

In the next image I add Arrange and Act. For my convenience I abbreviated to “Red, Green, Refactor” to “RGR”.

There are 3 rectangles, Arrange, Act, and Assert. Under the rectangle Assert RGR is written twice. The left RGR is lower than the right RGR!
In order to improve my skills I used the Selenium WebDriver with Java course of Angie Jones. It was very free, very good, and filled with useful code.

My deliberate practice exercise was to see, whether I could log in via the link Form Authentication on https://the-internet.herokuapp.com/.  See chapter 4.2 of the course.

In the corner

Maybe you noticed that I started with Assert. But why did I start with the Assert?
Frankly I cannot remember it. Maybe Assert was my final destination for my test script. Or perhaps I used Red Green Refactor on the wrong level. Any way I had a starting point for my test automation script.

public class LoginTests {
SecureAreaPage secureAreaPage = new SecureAreaPage();

@Test
public void testSuccesfulLogin(){
  Assert.assertEquals(secureAreaPage.getMessage(),
    "You logged into a secure area!",
    "Alert text is incorrect.");
}

In SecureAreaPage.java I made a method for the class SecureAreaPage:

public String getMessage(){
  return "ran394594";
}

So I made my failing test. Red.

Now I could easily pass this test by changing the return value in SecureAreaPage.java. Green.

public String getAlertText(){
  return "You logged into a secure area!";
}

At this moment I had a problem. There was no connection with the website. The test would always pass regardless of the correctness of the form authentication.

Looking back I could even use the code to test pizzas. The test results would be of no value. What about testing that gravity is pulling me sideways? Same story. I must have been sleeping on my right side. The wrong side.

I needed to make some code to connect the website and Assert.
In the next blog post I will start with Arrange.

Test Automation is a quick way to check items in a web site. Fast Forwarding.
Coding is an activity which still needs consideration. Slow Down.

Find That Thing In The Room

80 % is not good enough.

A lot of readers might wonder about this statement. 80 % is quite good. Unless your boss does not accept any mistakes. But Agile is the new norm, so learn and adapt. Errors are some great and uncomfortable way to learn.

On the other hand a test automation result should not be right in 80 % of the cases. Especially when it was executed several times within 15 minutes. It is difficult for me to interpret these results.

My draft FAQ for test automation

This one is not perfect, but it will be better at the end of this blog post.

Q: What to do, if I always have a No Such Element Exception?
A: There is a good chance, that the web element cannot be found. So my way to locate the web element is bad. There are several ways to find a web element.

A: What to do, if I always have a Stale Element Exception?
A: There is a high chance, that TestNG is used. The element to be used cannot be found any more. In this can use a findElement as late as possible.

Q: What to do, if I have a No Such Element Exception once in a while?
A: My solution was to retry the required action. After an exception I would wait and try again.

Boolean individualOrderNotFound = false;
For (int i = 0; i < 3 && !individualOrderNotFound; i++){
  try {
      findElement(xpath).click();
      individualOrderNotFound = true;
  }
  catch(NoSuchElementException e){
      Thread.sleep(2000);
  }
}

Q: What to do, if I have a Stale Element Exception once in a while?
A: See the answer on the previous question.

Q: Would you like to write a blog post about it?
A: You are reading one of the blog post serie.

Q: Do you have anything to add?
A: Sure. How about from 80 % to 100% reliability?

Q: Why is 80 % not good enough?
A: If you perform a test, then the result cannot be trusted.

Q: I would just execute the test several times. 80 % of the results would point in the right direction.
A: So you take extra test execution for granted. And what if 90% of the test results are the same. Do you still execute tests until you get to 80 %?

Q: This is supposed to be a frequently asked questions.
A: You are right. Now it is a dialogue. Does it matter?

Q: I am the person who should be posing the question.
A: Why?

Pretty sure

The morale of the following story is that statistics should be used carefully to help your customers. So you can go straight to the next chapter with more Java and Selenium stuff.

Content warning: the following story contains a scene about a date with an unfortunate ending.

Imaginary situation in the cinema more than 2 decades ago. I am recognized by one of the employees.

Me: “I want to have a Cola.”
Employee: “Here you are. And the nachos and the hot sauce.”
Me: [puzzled] “How did you know that?”

Employee: “You always have the Nachos, if you order the Cola.”
Me: “Yes, you are right. I also want to order a Lemonade.”
Employee: “With a chance of 80 % …”
Employee: [Writes something on the cup and places the drink with a wink] “Here you are.”

Me: [Surprised] “What did you write??”
Brunette: [Walks to the counter] “Amy! Who is Amy?”
Me [Turning to the lady with a cramped smile.]
Employee: “Hi Amy.”

Me [Turning to the blond lady with a cramped smile.]
Brunette [Turning to the blond lady] “I am 5 minutes away and …”
Brunette: [Picks up my drink and throws it over me. Walks away angrily.]

Me: “And Amy, did you figure out the situation with the pointers?”
Young man [Approaches counter]: “So, that’s why you are interested in programming.”
Young man [Picks up the drink of Amy and pours over me. Walks away angry]

Amy: [Sighs] “I finally found a young man, who wants to talk about programming.”
Amy: [Picks up my hot sauce and pours it over me. Followed by the nachos]
Amy [Walks away angrily]

Employee: “If this get worse, I pay you bill. 100 % chance of no way.”
A TV crew pops up. A woman with a microphone: “People do the strangest things to stand the desert heat of the desert scene of Monsters Unlimited. More at 9. What is the cooling effect of Nachos and the hot sauce?”

Employee: [Shows receipt with a message: the receiver gets a lifetime membership of Cinema Hype VIP.]
[The sound of a paper being shredded.]

[The end]

Say it again Sam

Back to reality.
In my office I had the following quality check of my automation scripts. If a script would pass in 4 out of 5 cases, then it would be good. OK next test.

Now I had my honour to make my test results 100 % reliable.

The biggest advantage of the Cinema Hype VIP application was the two stage order. First an order could be placed for the movie tickets, so the seats were reserved. On the day of the movie drinks and snacks could be added to the order.

Let me try to remember what happened.
On first sight the actions were not that complicated:

  • Select a day.
  • Select a movie.
  • Select a time slot.

What did I see on screen?
There were 7 buttons to select a day. I implemented this step flawlessly.

The next step was to select the right movie in a combo box. I really liked Outside In, which would be shown from Thursday.

 
WebElement moviesComboBox = driver.findElement(By.xpath(contains(@id,
                                     “moviesComboBox”));
filterCombobox.click();
selectMovieOption();
filterField.sendKeys(“Outside In”);
filterField.sendKeys(Keys.ENTER);

The selection in the combo box went wrong: Stale Element Exception.

Maybe a loop might solve this problem.

Boolean individualOrderNotFound = false;
For (int i = 0; i < 3 && !individualOrderNotFound; i++){
  try {
      WebElement moviesComboBox = driver.findElement(By.xpath(contains(@id,
                                     “moviesComboBox”));
      filterCombobox.click();
      selectMovieOption();
      filterField.sendKeys(“Outside In”);
      filterField.sendKeys(Keys.ENTER);
      individualOrderNotFound = true;
  }
  catch(StaleElementException e){
      Thread.sleep(2000);
  }
}

After some tweaking I got No Such Element Exception. I also added this exception, but it did not help.

 
Boolean individualOrderNotFound = false;
For (int i = 0; i < 3 && !individualOrderNotFound; i++){
  try {
      WebElement moviesComboBox = driver.findElement(By.xpath(contains(@id,
                                     “moviesComboBox”));
      filterCombobox.click();
      selectMovieOption();
      filterField.sendKeys(“Outside In”);
      filterField.sendKeys(Keys.ENTER);
      individualOrderNotFound = true;
  }
  catch(StaleElementException e){
      Thread.sleep(2000);
  }
  catch(NoSuchElementException e){
      Thread.sleep(2000);
  }
}

I found my monster in the room. So I asked for help.

A tester from another team replaced the loop for the movie selection by a couple of sleeps.

Thread.sleep(2000);
filterCombobox.click();
Thread.sleep(2000);

selectMovieOption();
filterField.sendKeys(“Outside In”);
filterField.sendKeys(Keys.ENTER);

The code was a bit slower, but more reliable. 5 out of 5 passed. I better got used to this.

Hypothetical causes

Retrospecting my coding was not pleasant. I did not stick to my own FAQ. And this can happen.

Now let me focus on the working code.
My fellow tester gave me the tip to use sleep. This way any delays would be handled.

There were two statements “filterCombobox.click();” and “selectMovieOption();”, which were preceded with sleeps. Therefore there were two places where exceptions could occur. In my loop I assumed that there was only one place for exceptions. The worst part was that I picked the wrong action.

Let me illustrate, what would happen, if I handle exceptions by starting over at the wrong spot.
I have a recipe for baking. I have enough flour and more than enough eggs. First I find the flour. Everything goes right: there is no exception. Then I put 200 gram of flour into a bowl.

For the second step I pick an egg. I break the egg and the smell is bad. That is an exception. The egg goes into the wastebin. So I start all over. I empty the bowl.

First step I have to weigh the flour, but there is no flour any more. Now I have an exception and I cannot finish the recipe.

A much better way to find a good egg without throwing away the flour.

This is something I need to research.

Final Fantast I C Assert

At the end of every regression test there should be one final check. A simple yes or no would indicate, whether the test has passed. For the cinema reservation system this was quite simple. Has the order for the tickets, drinks, and snacks sent to the cinema?

I let Selenium set the search field to Order Id and enter the Order id. I saw a Stale Element Exception. Time for my FAQ. I used findElement at the latest moment. Stale Element Exception.

I let Selenium switch to another menu and back to the right page to continue.
Stale Element Exception.

Which other steps were possible?
I let Selenium log off and log on. A search of the Order Id followed.
The order was not found, so it was sent. Test passed.

Another look at Continuous Delivery

[Update author: my opinion is not the same as the author of the referred characters, but I believe in the goodness of the good characters.]

It was like delivering a message. If one way did not work out, another way was explored. What had a mail delivery person, a milk delivery person, and a mail delivery teacher in common?

They all wanted to deliver a letter to Mr. H. Potter. Persistence paid off. And be nice to the messenger. The delivery story had a nasty tail.

Deliberate Practice

Of course this post should end with some smart code.

While blogging I was going through the chapters of the free course about Selenium WebDriver and Java of Angie Jones on the Test Automation University. In Chapter 9 about Wait Strategies I found FluentWait. One of the cool things about this command, that it could ignore exceptions.

A part of the solution was:

Thread.sleep(2000);

selectMovieOption();

This code was not optimal. If the environment or network is changed, then the sleep time might be adopted each time. A FluentWait would provide a robust and fast solution in this case.

FluentWait wait = new FluentWait(driver)
  .withTimeout(Duration.ofSeconds(8))
  .pollingEvery(Duration.ofSeconds(1))
  .ignoring(NoSuchElementexception.class);
wait.until(ExpectedConditions.visibilityOf(
  driver.findelement(filterCombobox)));
selectMovieOption();

Also Alan Richardson released a paid course about test automation on LinkedIn: Advanced Selenium: 3 Synchronization Strategies. If I need more tricks, I consider to have a close look.

My improved FAQ for test automation

Q: What to do, if I always have a No Such Element Exception?
A: There is a good chance, that the web element cannot be found. So my way for to locate the web element is bad. There are several ways to find a web element.

Q: What to do, if I always have a Stale Element Exception?
A: There is a high chance, that TestNG is used. The element to be used cannot be found any more. In this case I use a findElement as late as possible.  Another option is to add some additional steps like logging out and logging in.

Q: What to do, if I have a No Such Element Exception once in a while?
A: Use a FluentWait.

Q: What to do, if I have a Stale Element Exception once in a while?
A: See the answer on the previous question.

Q: Do you have anything to add?
A: Have a look at Deliberate Practice. I mean the way of learning and not the previous chapter.

Selenium, I think this is the beginning of a beautiful friendship.