Category: Testing Challenges

My first pair testing session – Day 21 of 30 Days of Testing

buggylina1 Comment

PAIR TEST WITH SOMEONE

Some time ago I’ve read a great article by Katrina the Tester on Pair Testing. This topic since then would often pop up, but I never had a chance to actually try it out.

I am very grateful and lucky because I have super supportive colleagues when it comes to 30 days of testing challenge. Not only that they show a sincere interest and ask about the progress, but they also agree to get involved into some of the challenges. At the very start of July, I was talking to one of our great front-end developers and mentioned pair testing as one of the challenges. He immediately said “I can do it with you!”.

Ironically, today I did not have any specific task which we could pair test, so we thought of testing an existing feature.

In the very start it was a lot about seeing how he does testing. It is fairly different than my testing, so it was very beneficial to me. He showed how he configured dev environment locally for testing so he could fix any coming up issues as well immediately.

Then, we tried to specify on how this feature should be tested. This involved some UI tests verifying requirements, but not only that – we looked at monitoring, requests being sent. Usually, when I test, I concentrate on UI when the feature is mainly seen on the UI, but it was great to see how differently the very same feature is viewed by a developer who knows what events are being recorded and which requests should be sent for the feature to work. He as well showed me some of his unit tests for that feature.

To sum up, we asked when we can say that we have tested the feature completely. How much is enough? What should we do to be sure that it’s well tested?

This question is a challenging one. We decided that in addition to what we discussed, we should as well just try to be users for a while and try to use the product. Funny enough, trying a few of scenarios – we found a bug. In a tested released feature in production.

Here came in a very important point in testing: while testing one of the features you can have blind spots on some of inconsistencies on other existing features, so it’s best to stay alert and check for regressions which concentrate not only on the new feature.

Pair testing definitely exceeded all my expectations: I learned a lot new ways how to test, and, seeing the product with the help of different set of eyes even helped us to find a bug!

And, it was all about improvisation and sharing knowledge rather than dictating the rules.

128istqb

Illustration above taken from brilliant Cartoon Tester.

 

 

 

 

Gruyere’s paradise for security testing – Day 20 of 30 Days of Testing

buggylina2 Comments

FIND A GOOD PLACE TO PERFORM SOME SECURITY TESTS

In many sources, security testing is called ethical hacking. It means that you create a controlled attack to evaluate the security status of your product.

To familiarize more with the concept of security testing, I read a very detailed tutorial. Then, after some googling, I found a wonderful place to learn more and actually perform security tests: Google’s Gruyere!

Gruyere is not only a great playground for security testing, but it has lessons and challenges to help you find vulnerabilities. If you get stuck, there are hints on how you should proceed.

I have read and tried out first few parts in Gruyere and had a great fun! It feels very nice to find security flaws. For example, I changed my account to be administrator account which enabled option Manage this server. Now I could edit any user.

manageserver

Security testing always looked like a very complex and difficult area to me. I have never performed any security tests before, so this seemed like one of the biggest challenges. However, after actually trying to perform some security tests, I can say that it’s actually not that difficult. It is hacking in a way, but it’s rather knowing the product you are testing really well and trying to think of the ways how to unleash the security flaws. I will definitely come back to this exciting topic and I am very grateful to Gruyere for being such a great place to learn and execute some security tests!

P.S.  Web Application Security Testing Cheat Sheet can come in handy for further learning.

Struggles and discoveries using te52 – Day 19 of 30 Days of Testing

buggylina1 Comment

FIND AND USE A NEW TOOL

Today I decided to try out tellurium 52. It’s a web-based simple automation tool. The main features of te52 are:

  • It’s free (this is always a plus!)
  • Tests are written in “Plain English”
  • Cross-browser testing
  • Scheduling

It sounded great, so I got my hands dirty.

In the start, it can be confusing on how to write a test in “Plain English” when web elements are way more complex than that. I write Selenium tests, so for me this more business-orientated way of writing tests was both something new and challenging.

Trying to write my own test, I found out that there is a Tellurium Chrome extension which works rather the same as Selenium IDE: you can record your browser activity and get a “pre-made” test. I have had my share of disappointments with Selenium IDE, so I did not expect much from a recording tool, but… it was way worse than I expected.

The extension would constantly crash and make the whole page which I was recording unresponsive. I would often blame the sites for being not te52-friendly and changed the sites for my test idea multiple times.  Screen Shot 2016-07-19 at 6.14.15 PM

After a lot of struggles with the extension and eventually inspecting web elements manually, I created a test for Urban Outfitters Quick View: open Quick View, assert that Add to Bag is visible and take a screenshot.

My final test “script” looked as follows:

For the url http://www.urbanoutfitters.com/uk/catalog/category.jsp?id=WOMENS-DAY-DRESSES-EU#/
When the mouse is moved over the first “Kimchi Blue Katrina Polka Dot Shirt Dress” image
When the first Quick Shop link is clicked
Once the “action-buttons ng-scope” div is present
Then the text “Add to Bag” is present
Then take a screenshot

As you can see, plain English definitely is very easy to understand and implementation details are hidden. Definitely, test could be even more obvious if my elements were chosen wiser wouldn’t be as specific (but I made them to be so to get it done!). The screenshot was successfully taken and saved displaying the Quick View:
screenshot-0001

Some final observations about the tool:

  • Idea is very good: you get to do quick checks online which are written in an easy to understand language and anyone could check them out.
  • Different browsers support is still very weak. To run tests online in the simple way you can do it only on Chrome, Firefox and IE11 (Tellurium grid). And, this is what I got trying to run it on Firefox:
    Screen Shot 2016-07-19 at 6.57.05 PM.png
  • Chrome extension is a pain to use: it crashes and freezes the whole page. You should be quite familiar with inspecting elements in order to write tests in te52 and cannot rely on the extension to help you.
  • Import from Selenium IDE works: I tried to import Selenium IDE recorded scenario and it worked fine. Did not dare to import real Selenium test.
  • Some web elements are not easy to capture. This is a constant struggle with web UI testing – it changes, it’s dynamic, implementations very much differ from site to site. However, some of elements which Selenium IDE grasped fine did not get recognized on te52.

In the end, I have had quite a few struggles with te52 and I think it still has room for improvement. This table demonstrates it well:

Screen Shot 2016-07-19 at 6.28.59 PM.png

I may use it in the future for scheduled automated smoke checks of UI as it allows to take screenshots easily, run tests on the regular basis and it all is for free. However, at this moment I don’t feel that I’m ready to bother with a lot of limitations that te52 still has.

 

Easy way to find broken links – Day 18 of 30 Days of Testing

buggylina1 Comment

FIND A BROKEN LINK. AND REPORT IT

During some previous testing challenges, I encountered broken links (they are pretty common!). However, I decided to try testing a different website.

Again I had to choose – I like pretty websites, so I found Woven Magazine.

There are free online tools which scan the whole website for broken links and create nice reports. So, I just used Dead Link Checker. No need for me to blindly try to find broken links when I can get a full list of them.

Possibly, magazine does not run a dead link checker because they have 11 broken links:

dead_links

I did check some of those broken links and they hide pretty nice bugs. For example, you go to http://wovenmagazine.com/blog/ and there is a button to Collaborate. You click on it and this is what you get:
Capture.JPG

Not only that you cannot really Collaborate, but you also get Page 1 of 0.

Broken links happen very often in various kinds of websites because:
Things get outdated or they just change, while link remains.

You can see in the list for Woven – there are links from articles which were written more than a year ago and initiatives about which they were written do not exist anymore.

I did send an e-mail with my findings to the magazine so I’m sure it’s a win-win situation. Today’s challenge was definitely very fun to do – I love using broken link checkers!

Quote that inspires me – Day 17 of 30 Days of Testing

buggylina1 Comment

FIND AND SHARE A QUOTE THAT INSPIRES YOU

One of my absolute favorite quotes is:

“Ever tried. Ever failed. No matter. Try again. Fail again. Fail better” – Samuel Beckett

This definitely relates to the job of a tester (and not only). We must make mistakes in order to learn and improve.

By the way, back in 2014, I wrote a blog post about neglected bugs and used the very same quote to illustrate the importance of failing.

A little break from testing – Day 16 of 30 Days of Testing

buggylina1 Comment

GO TO A NON-TESTING EVENT

Today’s challenge is particularly easy.

Saturday may make the challenge rather difficult as meetup events usually happen on week days (you can find more info about how to find events in a blog post which I wrote before). However, I interpreted this day as taking a break because whatever we do in daily life can be treated as a non-testing event.

It has been raining in Budapest, so my “events” included some Saturday shopping, official laundry event and a movie night! A more event-like event definitely will happen in the future: I enjoy attending other non-testing meetups, for example, product management, hiker’s club or just self-improvement events.

Sometimes it’s good to take a little break, but I’m looking forward to Sunday’s testing!

 

Easier task than expected: testing e-commerce website – Day 15 of 30 Days of Testing

buggylina1 Comment

FIND A PROBLEM WITH AN E-COMMERCE WEBSITE

I must admit that today’s challenge was a low-hanging fruit for me. At work I test e-commerce websites and often get teased by my colleagues that I’m constantly shopping.

Talking about e-commerce websites, the main purpose of online stores is… to sell. This makes some functionalities super important because if they do not work – user cannot purchase.

When in online store, in order to make a purchase, user must be able to:

  1. Choose a product
  2. Select a size (color, other features of the product)
  3. Add To Bag
  4. Check Out
  5. Pay successfully using possible way of payment

There can be multiple issues in steps #1 and #2 like product filtering not working properly, color selection not working, etc. However, they are not as major as #3-5.

When choosing a site to test, I wanted to take a look at a website which I haven’t visited before. For that, I decided to search for a nicely looking website and found an article with 20 of the Very Best E-Commerce Websites. This article lists some stylish e-commerce sites.

My choice was a known name (which definitely should have a properly working online store) – Reebok.

I did check global Reebok site first which already had several issues like broken product links, but it did not have an online store, so I changed to the most powerful market when it comes to e-commerce – US.

Major functionalities test had to be executed first and as a real user, I tried to make a purchase using 5 steps I mentioned above with biggest interest in critical functions (3, 4 and 5). And, to my surprise, it took me 5 minutes to find a problem.

In US site of Reebok Check Out using Paypal does not work

  1. Go to a wanted product in the US site of Reebok
  2. Add the item to bag
    Must mention that in this step I did not expect to be redirected to the Shopping Bag because this discourages further shopping and tries to make user complete the Check Out, but this happened!
  3. Click on PayPal CheckOut (any of two button instances)
    Verify that nothing happens – URL gets Redirect addition but the page itself does not change. Trying to pay with PayPal is not possible for a user.

Screen Shot 2016-07-15 at 6.24.21 PM.png

Notes:

  • I reproduced this issue on both latest Safari and Chrome versions on Mac OS, as well as latest Chrome on Windows 10.
  • I did check different market site (the one in Netherlands) and the user experience was definitely inconsistent: for example, step 2 just gave a popup if you want to Check Out or continue browsing and Paypal redirection did work.
  • Also, if you actually Check Out in US market and then click to Check Out with Paypal – you are redirected back to Shopping Bag.

This video illustrates displays the issue and points mentioned in this post:

 

 

 

 

Where to start if you want to be a speaker? – Day 14 of 30 Days of Testing

buggylina2 Comments

STEP OUTSIDE OF YOUR COMFORT ZONE

After attending a few conferences, I realized that a lot of us may feel Impostor syndrome when it comes to professional knowledge. Its definition in Wikipedia states:

Impostor syndrome is a term referring to high-achieving individuals marked by an inability to internalize their accomplishments and a persistent fear of being exposed as a “fraud”.

This should not happen, because we all have unique stories and career advice to share. So, why not to step outside of the comfort zone and become a speaker?

In one of the conferences I attended the after-party and a couple of speakers shared their road to becoming a speaker. What inspired me the most was that they stressed that you already have a “No” by default, so why not to try applying to the conference and suggesting your topic until you get a “Yes”? It cannot get worse because you already have a “No”. This is how many speakers actually became speakers – by trying to use each chance they have and not giving up.

Working on becoming a speaker has been recent stepping outside of the comfort zone for me. I still need to work very hard and actually achieve my goal, but I have collected a few valuable tips which can help you to get started as well.

  1. Think of your knowledge and passions to decide what topics you could talk about.
    As a professional you definitely have tools or methodologies you use. These could be easily transformed into a useful presentation for others. Especially, if they are made of advice and tips on how others could improve their work. Some examples that I have noticed often are “Responsive Design” or “How to improve your Selenium tests”. These topics are specific enough and a lot of people may be interested in hearing more about it. Also, you could think of something which is not that common but you are passionate about. It can be your own success story, or, area that interests you. For example, maybe you’re passionate about Internet of Things and could share what you’ve learned about its testing so far?
  2. Read and watch online resources
    Don’t be afraid to search for information online on how to become a speaker. Recently I found a great blog post on how to write an appealing abstract for the conference, and, noticed a soon approaching webinar on Conference Speaking 101 by Lee Copeland. These resources are free, but definitely very valuable.
  3. Find a mentor
    There are many speakers who are willing to help you to become a speaker. One of the great ways to find them is to apply for Speak Easy programme which is free. There you get an experienced mentor who helps you to build your talk and gives you a lot of useful advice. I have joined it and my mentor has been giving great thought-provoking feedback (some of it inspired this blog post). Also, recently Maaret Pyhäjärvi  volunteered to help mentor new speakers and organize Signature Webinar Series with Ministry of Testing.
  4. Practise speaking in your company and local events
    The best way to get experience in public speaking is to start little by little and grow your audience. In my company every two weeks we organize a Secret Gathering where anyone from company can share their discoveries or work details if they want. This helps both to understand the team better and also practise your own presentation skills. When you are better in presenting in smaller groups, why not to give a go giving a speech in a local meetup? If you’re in Budapest and would like to talk about something QA related – feel free to contact me and join Budapest’s QA meetup as a speaker.
  5. Don’t give up
    Becoming a speaker definitely requires a lot of patience and hard work. I am sure that I will first get multiple rejections before getting an invitation to actually speak. However, only trying we can actually achieve our dreams.

Testing community is great and the more people we get involved in it – the better. Join me trying to step outside the comfort zone and bringing in some new voices to conferences.

 

How a user decided not to use Wizzair app – Day 13 of 30 Days of Testing

buggylina1 Comment

FIND A USER EXPERIENCE PROBLEM

UX problems are very common in all kinds of products. Each of us has faced them with products we tend to use often. It can be as simple as too small button, and as a user you may misclick or mistap it which can be very frustrating.

Techniques for usability testing definitely would not fit to this post, but I must include a honorable mention of heat maps. I just love images of them and user behavior: how there are hot areas which users interact with way more.

In Software Development, UX problems usually are marked as “nice-to-have“, however, sometimes, especially with new products, UX problems are major ones to solve: if user is not comfortable using the product – they are discouraged to keep using it.

For today’s challenge, I chose to test Wizzair‘s mobile app on my Android phone. Usually, I do all my traveling related actions in desktop version, but today I decided to see how users can Check-in to the flight via app.

I booked my flight in one booking before, so I found my booking in the app and tapped on it. Screen opened with my both ways flights. Even if I do believe that it would be great to Check-in to both flights at once, but web client works the same way, so it may be by design.
I noticed this UX problem:
“Please check-in!” is shown only on 1 out of 2 flights which are actually both not checked-in yet.
This is confusing for a user (I had to re-check online to see that I really did not check-in to the second flight of my trip).

checkin

Moving on, I did check-in to the first flight. Then, I just went back a few times (because I knew that second flight is waiting there to be checked-in to again). I got to the same screen as I inserted above, it confused me – I just checked-in, so why is it still asking to check-in! I clicked once again to check-in to the first flight and ended up again in Check-in window, but this time after first Proceed, I got my Boarding Pass window. Well, this was quite buggy, so I moved again back. This time it refreshed the state accordingly and displayed this:

all_passengers_checkedin

It is clear for me as a user that I did check-in to the first flight, but what about the second one? Why doesn’t “Please check-in!” appear there?

If I had many flights, this could be very frustrating not to know in which flights I did check-in. Showing message only to one of flights and not others may mean that others don’t have the same state. 

In the end, because of this UX problem, I may stick to the desktop version of Wizzair.