Getting to know more testers – Day 24 of 30 Days of Testing

buggylina1 Comment

CONNECT WITH A TESTER WHO YOU HAVEN’T PREVIOUSLY CONNECTED WITH

30 Days of Testing definitely introduced me to multiple testing community members whom I did not connect with before. It has been very rewarding to feel the sense of community!

Nevertheless, today I spent some time connecting to possible speakers for Budapest QA meetup. Sometimes it can be very tough to find new speakers, so a lot of research has to be put in it. So, I have looked at some nearby testing meetups and connected to some of the organizers/speakers from there.

A wonderful thing was that I got very positive replies and now we will try to keep in touch to make sure to arrange a talk when they are here.

If you are interested in speaking at Budapest’s QA meetup and tend to come here sometimes – let me know, we would like to get to know you.

600_442114348

How I helped the account manager to test better – Day 23 of 30 Days of Testing

buggylina1 Comment

HELP SOMEONE TEST BETTER

Quality of a product is a very important topic to anyone in the company, so ability to quickly check real-time status of major product’s KPIs is a great skill to have.

In my company, very often some of the checks are done by our account manager. Imagine a situation where a partner writes in with a problem with web-service which your company is delivering. Account manager is usually the first person to read this, check the problem and reply to the partner. In other scenarios, account manager may just need a quick check of certain partner service quality before making any kind of communication step with them. Have in mind that sometimes other colleagues are not able to help immediately, so account manager would clearly benefit from knowing how to execute some checks independently.

Our account manager is a great learner and has been very open to get to know more about the product, so we arranged a meeting to talk about my beloved monitoring tool – New Relic.

I have explained to him some basics of New Relic: where to find dashboards, how to see queries behind each dashboard, how to make your own queries. We created some queries together to check a few important questions and modified existing ones to get the grasp of how to build queries if you don’t know where to start.

Of course it will take some time for the account manager to get used to this new tool, but I’m sure that he got a good background now and will be able to check out some of important questions real-time.

It was very fun for me to share knowledge and we even found out together that one of device types for our web-service in the last 2 weeks was a car, specifying which car it was we learned a fun fact – our web service was used by Tesla car owners.

 

Ode to New Relic – Day 22 of 30 Days of Testing

buggylina1 Comment

SHARE YOUR FAVORITE TESTING TOOL

I must admit that I love using analytics and monitoring in my testing. This is why my favorite tool is New Relic! It’s a software service which monitors web and mobile applications in real-time.

You may wonder why I call it a testing tool: I believe that monitoring and analytics are a huge part of regression testing. If numbers are still correct (or better), it can be a very valuable KPI to measure the system change. And that is not the only benefit in testing that New Relic can give.

I am going to list 3 reasons why I love using New Relic Insights (that’s the part that I tend to use the most):

  1. You can collect statistical significance and evidence of reported bugs.
    In most of the cases, I can check real-time data and see how many users were affected by a certain issue. For example, if found issue happens for a specific set of items, I can check the percentage of all requests which were made on those items. The bug will definitely need to be fixed if the impact is high.
  2. You can find out your user demographics and devices/browsers they use.
    It is very important to know who is your audience. If most of your users are based in India, it is likely that they use different devices than the users in the USA. This can help you to decide which devices and browsers you should test on and which location settings are important. For example, the following screenshot displays what are the most popular desktop and mobile browsers among the users of the product I test:
    Screen Shot 2016-07-18 at 5.46.25 PM
  3. By monitoring the system, you can find the issues easily.
    You can create all kinds of monitoring dashboards. Our product is a service, so knowing that service is functioning correctly means a lot. If there is a drop in one of its functions – we have a problem. For this reason, I have created a dashboard for myself including all important KPIs to me. Every day I check to see if system is in tact, if not – I query for more specific examples and investigate what could be the reason. Nevertheless, after releases, I always doublecheck the dashboards for any possible regressions. It helps me to make sure that system continues working as expected real-time.

New Relic Insights has given me more confidence in bug reporting because I can add a real-life value and importance to them. Also, it helps me to do quick regression checks and investigations which I was not able to do this easily pre-New Relic.

 

 

My first pair testing session – Day 21 of 30 Days of Testing

buggylina1 Comment

PAIR TEST WITH SOMEONE

Some time ago I’ve read a great article by Katrina the Tester on Pair Testing. This topic since then would often pop up, but I never had a chance to actually try it out.

I am very grateful and lucky because I have super supportive colleagues when it comes to 30 days of testing challenge. Not only that they show a sincere interest and ask about the progress, but they also agree to get involved into some of the challenges. At the very start of July, I was talking to one of our great front-end developers and mentioned pair testing as one of the challenges. He immediately said “I can do it with you!”.

Ironically, today I did not have any specific task which we could pair test, so we thought of testing an existing feature.

In the very start it was a lot about seeing how he does testing. It is fairly different than my testing, so it was very beneficial to me. He showed how he configured dev environment locally for testing so he could fix any coming up issues as well immediately.

Then, we tried to specify on how this feature should be tested. This involved some UI tests verifying requirements, but not only that – we looked at monitoring, requests being sent. Usually, when I test, I concentrate on UI when the feature is mainly seen on the UI, but it was great to see how differently the very same feature is viewed by a developer who knows what events are being recorded and which requests should be sent for the feature to work. He as well showed me some of his unit tests for that feature.

To sum up, we asked when we can say that we have tested the feature completely. How much is enough? What should we do to be sure that it’s well tested?

This question is a challenging one. We decided that in addition to what we discussed, we should as well just try to be users for a while and try to use the product. Funny enough, trying a few of scenarios – we found a bug. In a tested released feature in production.

Here came in a very important point in testing: while testing one of the features you can have blind spots on some of inconsistencies on other existing features, so it’s best to stay alert and check for regressions which concentrate not only on the new feature.

Pair testing definitely exceeded all my expectations: I learned a lot new ways how to test, and, seeing the product with the help of different set of eyes even helped us to find a bug!

And, it was all about improvisation and sharing knowledge rather than dictating the rules.

128istqb

Illustration above taken from brilliant Cartoon Tester.

 

 

 

 

Gruyere’s paradise for security testing – Day 20 of 30 Days of Testing

buggylina2 Comments

FIND A GOOD PLACE TO PERFORM SOME SECURITY TESTS

In many sources, security testing is called ethical hacking. It means that you create a controlled attack to evaluate the security status of your product.

To familiarize more with the concept of security testing, I read a very detailed tutorial. Then, after some googling, I found a wonderful place to learn more and actually perform security tests: Google’s Gruyere!

Gruyere is not only a great playground for security testing, but it has lessons and challenges to help you find vulnerabilities. If you get stuck, there are hints on how you should proceed.

I have read and tried out first few parts in Gruyere and had a great fun! It feels very nice to find security flaws. For example, I changed my account to be administrator account which enabled option Manage this server. Now I could edit any user.

manageserver

Security testing always looked like a very complex and difficult area to me. I have never performed any security tests before, so this seemed like one of the biggest challenges. However, after actually trying to perform some security tests, I can say that it’s actually not that difficult. It is hacking in a way, but it’s rather knowing the product you are testing really well and trying to think of the ways how to unleash the security flaws. I will definitely come back to this exciting topic and I am very grateful to Gruyere for being such a great place to learn and execute some security tests!

P.S.  Web Application Security Testing Cheat Sheet can come in handy for further learning.

Struggles and discoveries using te52 – Day 19 of 30 Days of Testing

buggylina1 Comment

FIND AND USE A NEW TOOL

Today I decided to try out tellurium 52. It’s a web-based simple automation tool. The main features of te52 are:

  • It’s free (this is always a plus!)
  • Tests are written in “Plain English”
  • Cross-browser testing
  • Scheduling

It sounded great, so I got my hands dirty.

In the start, it can be confusing on how to write a test in “Plain English” when web elements are way more complex than that. I write Selenium tests, so for me this more business-orientated way of writing tests was both something new and challenging.

Trying to write my own test, I found out that there is a Tellurium Chrome extension which works rather the same as Selenium IDE: you can record your browser activity and get a “pre-made” test. I have had my share of disappointments with Selenium IDE, so I did not expect much from a recording tool, but… it was way worse than I expected.

The extension would constantly crash and make the whole page which I was recording unresponsive. I would often blame the sites for being not te52-friendly and changed the sites for my test idea multiple times.  Screen Shot 2016-07-19 at 6.14.15 PM

After a lot of struggles with the extension and eventually inspecting web elements manually, I created a test for Urban Outfitters Quick View: open Quick View, assert that Add to Bag is visible and take a screenshot.

My final test “script” looked as follows:

For the url http://www.urbanoutfitters.com/uk/catalog/category.jsp?id=WOMENS-DAY-DRESSES-EU#/
When the mouse is moved over the first “Kimchi Blue Katrina Polka Dot Shirt Dress” image
When the first Quick Shop link is clicked
Once the “action-buttons ng-scope” div is present
Then the text “Add to Bag” is present
Then take a screenshot

As you can see, plain English definitely is very easy to understand and implementation details are hidden. Definitely, test could be even more obvious if my elements were chosen wiser wouldn’t be as specific (but I made them to be so to get it done!). The screenshot was successfully taken and saved displaying the Quick View:
screenshot-0001

Some final observations about the tool:

  • Idea is very good: you get to do quick checks online which are written in an easy to understand language and anyone could check them out.
  • Different browsers support is still very weak. To run tests online in the simple way you can do it only on Chrome, Firefox and IE11 (Tellurium grid). And, this is what I got trying to run it on Firefox:
    Screen Shot 2016-07-19 at 6.57.05 PM.png
  • Chrome extension is a pain to use: it crashes and freezes the whole page. You should be quite familiar with inspecting elements in order to write tests in te52 and cannot rely on the extension to help you.
  • Import from Selenium IDE works: I tried to import Selenium IDE recorded scenario and it worked fine. Did not dare to import real Selenium test.
  • Some web elements are not easy to capture. This is a constant struggle with web UI testing – it changes, it’s dynamic, implementations very much differ from site to site. However, some of elements which Selenium IDE grasped fine did not get recognized on te52.

In the end, I have had quite a few struggles with te52 and I think it still has room for improvement. This table demonstrates it well:

Screen Shot 2016-07-19 at 6.28.59 PM.png

I may use it in the future for scheduled automated smoke checks of UI as it allows to take screenshots easily, run tests on the regular basis and it all is for free. However, at this moment I don’t feel that I’m ready to bother with a lot of limitations that te52 still has.

 

Easy way to find broken links – Day 18 of 30 Days of Testing

buggylina1 Comment

FIND A BROKEN LINK. AND REPORT IT

During some previous testing challenges, I encountered broken links (they are pretty common!). However, I decided to try testing a different website.

Again I had to choose – I like pretty websites, so I found Woven Magazine.

There are free online tools which scan the whole website for broken links and create nice reports. So, I just used Dead Link Checker. No need for me to blindly try to find broken links when I can get a full list of them.

Possibly, magazine does not run a dead link checker because they have 11 broken links:

dead_links

I did check some of those broken links and they hide pretty nice bugs. For example, you go to http://wovenmagazine.com/blog/ and there is a button to Collaborate. You click on it and this is what you get:
Capture.JPG

Not only that you cannot really Collaborate, but you also get Page 1 of 0.

Broken links happen very often in various kinds of websites because:
Things get outdated or they just change, while link remains.

You can see in the list for Woven – there are links from articles which were written more than a year ago and initiatives about which they were written do not exist anymore.

I did send an e-mail with my findings to the magazine so I’m sure it’s a win-win situation. Today’s challenge was definitely very fun to do – I love using broken link checkers!

Quote that inspires me – Day 17 of 30 Days of Testing

buggylina1 Comment

FIND AND SHARE A QUOTE THAT INSPIRES YOU

One of my absolute favorite quotes is:

“Ever tried. Ever failed. No matter. Try again. Fail again. Fail better” – Samuel Beckett

This definitely relates to the job of a tester (and not only). We must make mistakes in order to learn and improve.

By the way, back in 2014, I wrote a blog post about neglected bugs and used the very same quote to illustrate the importance of failing.

A little break from testing – Day 16 of 30 Days of Testing

buggylina1 Comment

GO TO A NON-TESTING EVENT

Today’s challenge is particularly easy.

Saturday may make the challenge rather difficult as meetup events usually happen on week days (you can find more info about how to find events in a blog post which I wrote before). However, I interpreted this day as taking a break because whatever we do in daily life can be treated as a non-testing event.

It has been raining in Budapest, so my “events” included some Saturday shopping, official laundry event and a movie night! A more event-like event definitely will happen in the future: I enjoy attending other non-testing meetups, for example, product management, hiker’s club or just self-improvement events.

Sometimes it’s good to take a little break, but I’m looking forward to Sunday’s testing!

 

Easier task than expected: testing e-commerce website – Day 15 of 30 Days of Testing

buggylina1 Comment

FIND A PROBLEM WITH AN E-COMMERCE WEBSITE

I must admit that today’s challenge was a low-hanging fruit for me. At work I test e-commerce websites and often get teased by my colleagues that I’m constantly shopping.

Talking about e-commerce websites, the main purpose of online stores is… to sell. This makes some functionalities super important because if they do not work – user cannot purchase.

When in online store, in order to make a purchase, user must be able to:

  1. Choose a product
  2. Select a size (color, other features of the product)
  3. Add To Bag
  4. Check Out
  5. Pay successfully using possible way of payment

There can be multiple issues in steps #1 and #2 like product filtering not working properly, color selection not working, etc. However, they are not as major as #3-5.

When choosing a site to test, I wanted to take a look at a website which I haven’t visited before. For that, I decided to search for a nicely looking website and found an article with 20 of the Very Best E-Commerce Websites. This article lists some stylish e-commerce sites.

My choice was a known name (which definitely should have a properly working online store) – Reebok.

I did check global Reebok site first which already had several issues like broken product links, but it did not have an online store, so I changed to the most powerful market when it comes to e-commerce – US.

Major functionalities test had to be executed first and as a real user, I tried to make a purchase using 5 steps I mentioned above with biggest interest in critical functions (3, 4 and 5). And, to my surprise, it took me 5 minutes to find a problem.

In US site of Reebok Check Out using Paypal does not work

  1. Go to a wanted product in the US site of Reebok
  2. Add the item to bag
    Must mention that in this step I did not expect to be redirected to the Shopping Bag because this discourages further shopping and tries to make user complete the Check Out, but this happened!
  3. Click on PayPal CheckOut (any of two button instances)
    Verify that nothing happens – URL gets Redirect addition but the page itself does not change. Trying to pay with PayPal is not possible for a user.

Screen Shot 2016-07-15 at 6.24.21 PM.png

Notes:

  • I reproduced this issue on both latest Safari and Chrome versions on Mac OS, as well as latest Chrome on Windows 10.
  • I did check different market site (the one in Netherlands) and the user experience was definitely inconsistent: for example, step 2 just gave a popup if you want to Check Out or continue browsing and Paypal redirection did work.
  • Also, if you actually Check Out in US market and then click to Check Out with Paypal – you are redirected back to Shopping Bag.

This video illustrates displays the issue and points mentioned in this post: