Quick Start Guide…To Public Speaking

Public speaking is a bit like jumping into a pool. There is no half-way point; you either jump or you don’t. Sadly many people let the fear of public speaking put them off sharing great stories. Nobody is born as a great public speaker, it is a skill that everyone develops with practice.

Start small. Practice giving presentations to your colleagues or friends (if you have the sort of hobby that suits presentations..). Learn how to tell a story with visual aids rather than being tempted to read each slide line by line. Watching presentations from others is a great way to develop. Take note of how the speaker engages with the audience, uses slides, and the many different presentation styles that people use. Experiment with these ideas as you find your personal presentation style.

The only way to develop your public speaking is to practice. Write your talk in advance and then practice it until you know it inside out. The audience have turned out because they’re interested in your topic. Make sure that you do the preparation required to deliver them a great story.

I find the very beginning of a presentation to be the hardest. You’re likely to still be processing the whole being on stage looking at the audience thing. You might be blinded by the lights or freaked out by the foreignness of the microphone. All these things will work to try to make you lose your way. Have a scripted introduction and beginning to your talk that you can use if you do find yourself overwhelmed. I promise that once you get started you’ll forget your nerves.

What’s the worst that could happen? Huib Schoots passed on a piece of wisdom (that in turn came from Michael Bolton; if you’re feeling nervous before a presentation think about ‘Why’? Are you worried about getting facts or figures wrong? If so, practice. Are you worried about mis-pronouncing a word or losing your place? If so, does it really matter? Have you actually ever seen an audience attack someone for making a mistake like this?

Are you worried that you’ll be hit by stage-fright? Learning to manage your nerves is an important part of succeeding as a public speaker. It’s probably safe to assume that all public speakers are nervous to an extent. After all you’re about to stand up and share an idea, and opinion or a story. You want the audience to enjoy it, agree with it, remember you fondly but you also know that there will always be doubters ready to question what you think. Controlling nerves comes in part from rationalising them, and in part from learning to accept that they are part and parcel of public speaking. Use the adrenaline to bring energy to your talk and don’t be surprised if you feel a little wiped out by the end of it all.

There are a couple of mistakes that new public speakers make. As with everything else you are more likely to avoid these as you become more accustomed to public speaking but I thought they were worth mentioning. 1) The number 1 rule of a good presentation is don’t read the slides. Your audience can read too, and they can read a whole lot faster than you can speak so it’s likely they’ll have read the whole slide before you’ve even begun. 2) Don’t go too fast. Nerves can have the horrible habit of making you rush through your carefully rehearsed talk at a hundred miles an hour. Don’t be afraid of silences, the pauses can be a really great way to emphasise a point. They also give you some time to actually breathe. I find it helpful to write ‘Pause’ in my speaker notes to remind myself. Take a few moments to be silent and then continue. Taking a drink is also an easy way to build pauses into your talk.

Dive in, have a go and then make sure you do it again. Like any skill you need to practice so no matter what happens go away and find some positives and some areas for improvements then come back and do it again. After a couple of outings you’ll find the whole getting on a staging and speaking thing a lot less scary and your talk will improve no end.

Any other tips? How do you manage your nerves?

Acceptable Weaknesses

Everybody has strengths and weaknesses. You are likely to be working in a job that suits your strengths, however that doesn’t mean you won’t be required to test something that seriously challenges your weaknesses. Weaknesses tend to fall into the acceptable and the unacceptable categories. Categories that subtly change depending on your company. Maybe when you’re with friends it’s unacceptable not to be able to name all the Number 1 hits from the music charts in the 90s. A skill that is unlikely to matter at work.

Unacceptable weaknesses are more of a challenge. People take it as a given that you can spot a bug in spelling or grammar but very few job interviews test for this.  

I once tested a Polish language system. I don’t speak a word of Polish but this was an acceptable weakness. The testing role I was performing didn’t require me to actually read Polish, I was a functional tester in a team which also had a native Polish speaker to test the copy. Don’t be mistaken into thinking that my testing didn’t involve checking any of the system text. I couldn’t read the text but by deliberately ignoring it, thinking that it wasn’t my job, I would have perhaps missed several places where English had been mistakenly used instead of Polish. Maybe I would have even missed the screen layout problems caused by words being longer, or breaking in different places, to English.

The same system required me to test a financial product through its complete life-cycle. My background is not in financial services and so I hit upon an unacceptable weakness. Only by reading up on the financial product, and by talking to experts in the company could I gain enough understanding to be confident in testing the system. My lack of knowledge required that far more time and effort was expended to make sure all test plans were accurate. Exploratory testing was useful for trying to break the system but I was lacking the knowledge to allow me to test the system functionality without a scripted test plan.

Not all weaknesses are as obvious as this one. Consider for a moment how competent you believe you are in usability testing of websites. Most people consider themselves to have a reasonably good understanding of what makes a website easy and enjoyable to use. However there is a fascinating cognitive bias called The Dunning-Kruger Effect which states that incompetent people overestimate their abilities whereas competent people underestimate theirs. The great challenge is correctly identifying which side you stand on. It seems plausible that someone who hasn’t studied the complexities of usability will consider the problem to be far simpler than a UX expert who knows the subject in-depth.

The Dunning-Kruger Effect makes identifying weaknesses problematic. It is likely that if you are truly weak at an area then you will overestimate your abilities. You’ll be testing using knowledge that you don’t have. On the flip side you might be more competent that you realise but the Dunning-Kruger Effect will have you underestimating you abilities and probably conceding to ‘experts’ when in fact you should be challenging their opinions.

Either way you’re likely to be testing with an incomplete set of information.

So what can you do? The easy answer is to be aware. Question your assumptions and those of people around you. Always ask questions no matter how certain you are of the answer.

The hard answer is to address those (perceived) weaknesses. Get some training, read some books, do whatever it takes to learn enough about the subject to allow you to question subject experts, and ultimately software, with confidence.

Finding your motivation

Everything you do, or don’t do, is driven by motivation. Maslow’s hierarchy of needs makes a frequent appearance in conversations or books on how to motivate people. As does Herzberg’s two-factor theory. The basic idea is that people need food and safety before they start forming relationships. Only after they have formed relationships will they start working towards achievements and recognition. In short you have to get the work environment right before you can expect workers who are creative and self-driven.

Once, long ago, I was a pretty average tester. I was lucky that I knew I loved testing; University had given me the time to read a number of books on testing and each one fascinated me. Unfortunately the reality of being a tester turned out to be a little different from my dreams. There were highlights when I would get the chance to perform some meaty testing but also long periods of boredom. Test environments needed time to be built and recovered, there were endless rounds of manual regression testing, or worst of all, re-testing all the ‘fixed’ bugs only to find that they weren’t even remotely fixed.

Nowadays the testing industry has improved no end. As well as many more great testing books it is easier than ever to hear and form ideas via Twitter, blogs and, of course, conferences. Despite this there will probably always be motivated testers trapped on projects where they are expected to ‘just test’ with no chance of influencing the actual process or culture (caveat to this sentence is that you should always try to improve culture).

Testing on the sort of project that just wants some ‘testing’ can be a horribly de-motivating experience so how do you cope? If you are committed to the role and know you have no chance of improving things to allow you to actually test in the way that you want to test then you have to seek a new motivation. I survived many uninspiring projects by focusing on what I was learning; New tools, domains, or methodologies will all improve your ability to test. In many cases I learnt more about what I didn’t want in a job than what I did but it’s all valuable experience.

Even dull, death march projects are working towards delivering something to a user. When I get on board an aeroplane I trust that the software will work well enough that it isn’t going to drop from the sky. I trust that the ATM machine is going to give me the correct amount of money and update my account correctly. Every time I use a piece of software, get into my car, or basically do anything at all I hope and trust that someone has done their job and created a safe and effective system.

As a tester you have the ability to influence the end product. Maybe you don’t make the final decision on whether something goes live or not but you certainly play a significant role in identifying who your users are and testing to the best of your ability. So no matter how boring that insurance software is always remember that somewhere out there a user is going to buy insurance and trust that your product works. Do your best for them.

Keeping it real

Over the recent weeks I’ve been reminded of some of the things which work to undermine all your testing efforts. Collectively I consider them to be filters that you test through. How much they limit your ability to see the real system depends on how opaque they are.

  1. Test environments must be realistic. I should really list this as number 0 because it should be a total given. If the test environment has a different configuration from production then what exactly are you testing? Systems do not work independently of their environment.
  2. Not knowing who your user is. If you think like a developer then you’ll test like a developer. Make it your business to know who your user is and think like them. If your user is an internal business user then go and visit them. Do they use a keyboard or a mouse to navigate? How much technical knowledge do they have? Are they likely to be using wildcards in their search queries or not? Use this knowledge to shape your testing.
  3. Not knowing the numbers. If you work on a website them it is pretty standard to be collecting data on usage. Find out which browsers your users use. Which mobile devices are they accessing your website on? Which email client is most popular? There is no point conducting all of your testing on Internet Explorer 8 if the majority of your users have upgraded to Internet Explorer 9.
  4. Testing on developer spec computers. Maybe you’re one of the lucky ones who has a seriously cruddy PC but many testers are using developer spec computers – I’m talking about the ones with masses of RAM, maybe even quad-core. Basically something which is powerful enough to run those Selenium tests. It is pretty unlikely that your users have the same sort of power so they’re far more likely to see how slow your system really is.
  5. Throw in a huge monitor and things get worse. It’s pretty easy to forget that the average user is likely to have a single 17inch monitor (probably). Many of your users will be on laptops or maybe even tablets. Again those usage stats will be useful here. Keep your set-up realistic otherwise you’re likely to miss some pretty obvious usability issues.
  6. Network latency. Test environments are likely to be running inside your office which means requests will be much faster than for people accessing the system outside. Again this will depend on who your users are and how realistic your test environments are. If possible run your test environment from the same location as the production one.
  7. Office LAN speeds. Related to the above but I find this one is easy to forget. Anything you test inside the office should feel super quick. If it doesn’t then the poor person sat at home with a 4mbps wireless connection is going to be extremely frustrated. Make sure you consider both the LAN speed and the network latency when you’re planning performance tests.
  8. Test Data. This one deserves a whole post to itself but in short if the test environment doesn’t contain realistic amounts of test data and if you don’t enter realistic test data then you’re likely to miss bugs.

So in summary, find the numbers to help you identify who your users really are. Fight to have a test environment and test machines which actually reflect what the system will be like in production. Perform your testing against realistic test data and above all make sure you do at least some testing on a slow PC with a small monitor.

A problem! But what is the actual bug?

I was recently helping a junior tester test an iPhone app. He had found a button which wasn’t doing anything when clicked. The button appeared to click, certainly the graphical interface gave the impression that it was being clicked but nothing happened.

I doubt he is alone in being flummoxed about what to do next. He could have just raised a bug and left it to the developer to investigate but a good tester tries to find as much information as possible before handing it over to someone else.

Here’s the steps we went through to identify the actual cause of the symptom.

  1. Is the problem repeatable? Repeated clicks on the same button led to the same result. So, yes.

  2. Are all buttons affected? We found other buttons responsive. So in this case, no

  3. Does the problem appear to exist for everyone? Testing in a different browser (if web based), a different mobile device (if a mobile app) or even just a different user account can reveal a great deal about the problem. With just a little testing we found the button problem was affecting the user account rather than the device (logging in as the same user on a different device displayed the same behaviour, and logging in on the same device with a different user didn’t).

  4. Now that we know the problem is dependant on user data or settings we can dig deeper. This time I knew the iPhone app was built on top of an API. Directly querying the API displayed the same problem; the number of results being returned didn’t match the API headers which was confusing the iPhone app. This is good news, we have now eliminated the problem from the iPhone app.

  5. We know that different users are experiencing different things. Some have a button that doesn’t respond and others have one that does. We know the problem is in the API or lower rather than in the actual app code. Now we start to investigate everything that could be different for our two identified users.

    In this case there are almost no user settings involved so the most likely cause of the problem is data. Identify the unique combinations of data and then eliminate them until you find the trigger. Use a technique such as equivalence partitioning to reduce the number of unique combinations. In this case the problem turned out to be caused by data sets changing ‘state’ without correctly updating all the information in the API.

With just 5 steps we can go from seeing a button apparently not work to identifying unique data groups which trigger the actual problem in a lower layer of the system. If you had just gone ahead and raised a bug about a button not working when clicked, the odds are that it would have worked for the developer. Then the endless ‘Works for me’ cycle begins. So when you see an issue take the time to investigate the actual cause.  

What is testing? And why do we care?

Testing. We talk about it all the time. I read books, tweets and blog posts debating how to do testing, how to manage testing and usually how to improve my testing. I frequently hear people exclaiming that we’re not doing testing the ‘right way’ or that such and such ‘isn’t really a tester’ but what do we actually mean?

Firstly is seems important to address the question of whether we really need to spend time thinking about this question or not. If you work for a company that has a test team and you have time and resources to test things in the way that you want to, without anyone ever questioning why or how you test then you probably don’t need to spend much time thinking about it. If, however you are trying to grow a test team, or introduce testing more widely to your company then it is worth having a think about.

Personally I think we are wrong to have roles or responsibilities in a team if we don’t understand what we hope to achieve by having them, for that reason alone I think it’s important to understand what you actually mean when you talk about testing.

I think it’s fairly safe to say that the role of a ‘Tester’ will usually involve far more than just executing tests. You’re likely to be working on a test strategy, performing test planning, and risk assessments. You’ll be executing tests, reporting bugs, and maybe even creating automated test suites. The actual act of getting your hands on the system and testing seems to be just one small part of the tester role.

To me testing is about making sure the system does the right thing. I am deliberately vague with those words. Obviously the system must meet all the original requirements but it must also meet many others including, but not limited to, performance, security, accessibility, and reliability. Some of those will hopefully have been specified as part of the actual requirements but I believe testers should be testing them regardless of what was specified. After all is it fair or right to release an insecure product just because the requirements didn’t specify the need to secure it?

The system must also not do unexpected things and this is where I believe the strength of a tester lies; it is impossible to specify all the things that a system must not do and it is also impossible to test that a system doesn’t do all of the things that it shouldn’t do. The tester must assess risks and manage time in order to test a system thoroughly enough for release managers to have enough information be be able to decide if a system is fit for release or not. Good testers use their knowledge of test techniques and risks to break a system in realistic ways. Obviously you might want to to test if the system breaks under unlikely conditions such as a power failure, or DDOS attack but in my opinion there is fairly limited value in testing whether it breaks under near impossible conditions such as if a user attempts to log in 10 times in quick succession. Under tight time constraints I will also focus my testing on scenarios which are likely to occur.

Good testers are able to understand the business goals as well as the user’s goals. They certainly break systems in realistic ways. They bring knowledge of bugs seen in other systems and use the system architecture to quickly identify risk areas. Testers do not have exclusive rights in being able to carry out testing but they do have a particular strength in dedicating their time to studying testing. Anyone who takes the time to read about and practice a skills is likely to be more thorough and have a better understanding of where risk is likely to lie than the causal super user.

The outcome of testing will always be information. Some of that information will be captured in bug reports (things that don’t work as expected), maybe some will be recommendations or feature requests, some will be in completed test plans or mind maps. The outcome of the testing can guide release decisions but it will not provide quality. Quality systems come from understanding the users, the code and making the right decisions about features, infrastructure, code, as well as testing.

Testing the Tests

I am seriously pro test automation. After working on several projects with scant, if any automated test coverage I really appreciate the benefits of having fast, repeatable tests running as part of the build pipeline.

Unfortunately (or fortunately depending on your viewpoint) test automation is not a replacement for manual testing. You really don’t want to be duplicating your testing by manually testing the same things as your automated tests but you do need to make sure you’re placing you trust in a sound automation suite.

When you implement a new feature you’ve probably had some conversations with developers, designers and product people. Hopefully you have a good idea of the risks around this feature. In an ideal world the developer would have added lots of unit tests and maybe a few integration tests to test this feature. In a perfect world you, as the tester, know exactly what has and hasn’t been tested.

Problem number 1 comes from automated tests needing exact expected results to be completely defined up front. Quite a lot of testing is about finding out more information, using your tests to answer questions. Automation doesn’t give you any help with this.

Problem number 2 comes from trusting your tests. How do you know that your automated tests will find problems? Will they fail when something goes wrong? Does that green build really means things are working? The only way to answer these questions is to manually test. Finding bugs in high risk aspects of the system indicates missing or incorrect automated tests. Keep reviewing your risk categories and make sure you have then tested with reliable, complete automation.

Next time you sit down to carry out some manual testing use it to test the tests. The outcome of the testing should shape your opinion of your automated test suite. Bugs found should be fixed and new tests added. Confirm that tests passed have suitable test scenarios covered by automation and then relax.

Why that production bug might be the best thing that ever happened to your testing

Don’t you just hate bugs that get found in production? You’ve been testing for days (maybe even weeks) and yet as soon as it hits production you see that bug. Then you have the hassle of regression testing and patch releases.

Maybe it wasn’t even you who found the bug. A mocking tweet or an angry email, sometimes a very confused user who can’t understand why you didn’t test that this thing actually works!?!

We all know that you’ll never achieve perfection and yet we always try. Each release is relentlessly tested until the deadline is upon us, each time you hope that just maybe you’ve got all the big bugs. Maybe you even did, but maybe a bug found in production will make you a better tester.

So how do bugs get into production?

You didn’t think to test that area of the system: Why not? Maybe you didn’t realise that it had changed. Maybe you didn’t even know that this part of the system existed. Why? Did you miss something in the requirements? Did the developer make changes that you didn’t know about? Maybe you trusted automated tests to check this for you. Whatever the reason, it didn’t work so think about ways you can avoid this happening again. Maybe checking commit logs or improving regression suites would help or maybe you could improve your communication with developers.

You didn’t know how to test this area of the system: Hopefully this is an extension of not knowing this area of the system exists. Ignorance is not a good reason to miss bugs. If this bug was reported by a user then you need to re-visit your test planning to understand why you missed it. If it was discovered by someone technical then this is a great chance to learn. Find out how they discovered the bug. What techniques or tools are they using? If they work within your company then some pair testing might uncover more bugs whilst allowing you to pick up new testing knowledge.

You weren’t able to test this part of the system: Maybe you’re lucky but I have never worked anywhere which had an exact replica of the production available for testing. Any difference between the two opens up the chance of bugs making it through to production. Different sets of data, different release processes and different setups will undermine your testing. Use production bugs as evidence for why you need to improve your test environment. If you really can’t create a realistic test environment then consider how you could safely test in production.

You didn’t know you could test this part of the system: Related to the above. If you’re working on a test environment with limitations then make sure you, and everyone else who is testing knows exactly what should and shouldn’t be expected to work. Nothing is more frustrating that realising that thing that wasn’t working is broken rather than just not working because the test environment doesn’t support it. In my experience this is most likely to happen around 3rd party integrations.

As testers we aim to find all bugs before the code is released to production. Deep down we all know this is impossible. Use Production bugs as a way to improve your testing and you should see a significant reduction in big production bugs (but no guarantees you won’t still have bugs in production – sorry!).

Choosing to Learn

As you grow up you are surrounded by education, you go to school, join clubs, maybe go to college or university. Then you start a job, learn a new process and perhaps attend training courses. All the time you are being provided with the chance to learn new skills, but one day this stops, you’ve probably been in your job for a few years, you know the ropes and each day you perform actions without having to question why. After a few more years you realise that you haven’t learnt anything new in a long time, maybe you badger your manager to send you on some training or maybe you start to prioritise learning. 

Have you ever wondered how some people fit it all in? They seem to find time to go to work, look after the kids, exercise and still have time to read all those books which you have on your reading list. I used to wonder if they had somehow worked out how to create 5 more hours in every day, or maybe they just never slept but whilst both of those might be true it is far more likely that they are just prioritising these things higher than you do. No one can do everything in life, maybe you are the sort of person who cooks meals from scratch each night, or has a family to look after, for every 1 of you there are 10 people out there who wish they could cook but who are prioritising something else. Life is simply a juggling game. 

So what do you do? You could prioritise learning, on the extreme scale maybe you could give up your job and return to full-time education? Or maybe on a more practical level maybe you could spend a hour or two a week actively learning? If you travel by train you could be reading or listening to a book, if you work out at the gym you could be listening to a podcast, if you’re walking the dog you could be thinking through how that recent bug escaped your testing. Still too much commitment? Then learn to be more aware of the decisions you are making, why did you press that button first? How are you going to present that data? Becoming aware of your decisions lets you develop your thought processes and over time you’ll start to identify weak areas or mistakes that you could have avoided. 

Each week I consciously note down what I have learnt, sometimes it might be an amazing fact about another country, hopefully it’ll be something about how I could have been a better person or a better tester,  on a really gret week I might get through some of the Code Academy training courses or read a new book. The point is to make sure you have learnt something new by the end of the week, if you really want to learn then you need to prioritise it into your life.