A look back at 2014

2014 was the year that I resolved to do less stuff. I have a terrible time turning things down. Empty calendars seem to fill me with fear, and I simply can’t help getting involved in things I believe in. Unsurprisingly I ended up busier than ever.

I attended, or spoke at some fantastic conferences – TestBash, Let’s Test, Agile Cambridge, BBC Develop, and EuroSTAR. At each event I met, or re-acquainted with fantastic testers from around the world. My conference highlight of year was being involved in organising Pipeline. It took months of hard work but it was hugely satisfying to see the conference unfold on the day.

A few difficult months of writing and editing finally saw my contribution included in Build Quality In. If you’re interested in Continuous Delivery this book contains some fantastic accounts of how different teams have approached it. Steve and Matthew did a great job of selecting contributors and editing the submissions.

After many years of deferring I finally completed the Black Box Software Testing Foundation certificate, and attended James Bach’s RST course in Brighton. One day of Security Testing training with Bill Matthews completed my training for the year. My greatest takeaway from all of these courses was a realisation that I do have the time, and the desire, to learn. Hopefully I’ll be building on this in 2015.

Unexpectedly I found myself co-facilitating Weekend Testing Europe. Meeting, and having the opportunity to learn with the Weekend Testing attendees has been a pleasure. Alongside Neil and Dan I’m excited about the sessions we have (almost) planned for 2015.

Podcasts seemed to be a theme of my year. Steve and Dan were kind enough to have me on Testing in the Pub twice and I wrapped up my year with a great chat with Rosie for The Minitry of Testing Dojo.

Throughout 2014 I have thoroughly enjoyed all the interactions with so many testers – thank you to all of you.

Resolutions for 2015? Well I think I might have another go at being just a little less busy…

Quick start guide to… Weekend Testing

I’ve recently teamed up with Neil Studd to re-launch the European chapter of Weekend Testing (WTEU). Both of us were inspired by Alessandra Moreira‘s excellent talk at Let’s Test conference in May. Weekend Testing, originally set up by Ajay Balamurugadas aims to facilitate peer-to-peer learning through monthly Skype testing sessions. I’m a little embarrassed to admit that before taking steps to resurrect WTEU I had never actually taken part in a Weekend Testing session, somehow I just always seemed to be too busy…

As a soon-to-be facilitator I stated joining testing sessions with the other Weekend Testing chapters in the vain hope that some of their experience would rub off on me. I discovered that not only is Weekend Testing great fun, It’s a fantastic place to meet like-minded testers from all over the world, and most of all, a great place to learn.

So what’s Weekend Testing all about?
Originally started by Ajay as a place for peer-to-peer test practice. There are now four chapters (India, America, Australia/New Zealand, and Europe). You are welcome to join in any session, the locations drive the session time rather than the entry criteria. I recently joined a fantastic session with the Australia/New Zealand chapter but it did start at 5am…!

How does it work?
Each session takes place over Skype (typing, rather than talking). Skype, Tweet, or email the chapter to let them know you want to join the session and be on Skype 20 minutes before the start-time to be added to the session. Each session has a theme or topic so keep an eye on http://weekendtesting.com/ for full details.

What can I expect from a session?
Expect to learn. Each session lasts for 2 hours and is roughly divided into four sections – Intros, an introduction to the session topic, some time to carry out the mission, a post-session discussion on what you learnt. Sometimes you’ll work in pairs, or small groups, sometimes you’ll be working alone and then re-grouping to share your findings. Each session is unique.

Who attends sessions?
Everyone! Each session has a mix of people from all kinds of backgrounds, most are testers but we welcome anyone who wants to learn about testing. You are not required to bring anything along other than your curiosity.

How do I know when session will take place?
Chapter’s generally try to hold their session on a regular slot but each session is announced on http://weekendtesting.com/ as well as over email and Twitter.

What if I can’t make the whole session?
That’s OK. Weekend Testing is very informal so feel free to come and go as you need to. Obviously if you can stay for the whole session you will get a lot more out of it but everyone understands that sometimes life has a way of interfering.

If you’re still not sure it’s for you then take a look through the write-ups of past sessions on http://weekendtesting.com/ or drop me an email at testingthemind@gmail.com

Visiting an Open Device Lab

When I first heard about TestPartners’ Open Device Lab on The Software Testing Club I was intrigued. Like many teams we try to design and test for the most popular web browser and mobile devices. Unfortunately as any mobile user knows the speed of change makes it difficult, and certainly expensive, to maintain a test device library.

Finally last week I had the need to visit the lab. After a couple of emails with Steve our booking was confirmed.

When you book in to the lab you have the option to reserve specific devices for testing. We didn’t have any specific requirements, really we just wanted to test some recent front end changes on a range of modern devices with varying screen sizes. When we arrived we were presented with a box full of tablets and phones. Each was carefully labelled and had a charger too – something that never happens with the devices I manage at work!

An information sheet provided full details of each device, including screen size and resolution. In addition we soon discovered that each device was set up with its own email address so we could capture and share screenshots of bugs and email URLs to the devices for quick checking.

We found TestPartners’ device lab to be extremely useful. The lab is located in Bank, London, close enough to our office to avoid major inconvenience. It certainly allows us to maintain a device library of only our most widely used devices and then supplement this with visits to the device lab.

I think it’s fair to say that we’ll be returning to the ODL in the near future!

Thanks to TestPartners for making their devices available for use.

What is the Business Value of Continuous Delivery?

Michael Bolton sparked a fascinating discussion on Twitter by asking what exactly is the business value of Continuous Delivery? I’ve attempted to capture the conversation on Storify but I also want to explore the topic a little further.

Michael quite rightly pointed out that beyond CD being technically possible and cool, it is difficult to see the business value of the approach. I agree with him that there are practically no systems that could ever need to release new features every few minutes. In fact I can’t think of a single system which would require such frequent updates.

Despite this I believe that Continuous Delivery can provide value to the business.

Let’s step back for a moment and consider what ‘Business value’ actually means. It obviously means ‘providing something of value to the business’. Usually this ‘something’ will be a new feature or maybe a bug fix. Being able to release frequently could certainly get either one of those out to the business, but CD is only one possible solution to this problem. Even more important that releasing frequently is releasing reliably.

Almost every business wants to know exactly when a feature will be useable by real users. Releases require code to have been written and tested. Deploy scripts need to be reliable. Code needs to be packaged for release. Release documents, notes, maybe even marketing materials need to be produced. It seems logical to think that with so many parts to a release, and most likely the need for multiple people or even teams to be involved, you should release as infrequently as possible to reduce impact. In reality the more infrequently you perform an action the more likely you are to make a mistake.

The more frequently you practice a series if actions the more natural they feel.

Painful systems requiring workarounds or insider knowledge are often accepted if the user only has to feel the pain once in a blue moon. Bringing the pain forwards is really the only way to motivate people to fix these issues.

Simon Stewart responded to Michael Bolton’s initial Tweet questioning the business value of CD with the benefit of cleaner code. To be successful at CD you will need to learn how to be disciplined about writing and maintaining code. Developers need to be adding automated checks, and builds will need to be manged. My experience of adopting CD showed that having a goal that the developers were excited to achieve could be be a great motivator to get people fixing test environments, taking responsibility for quality, and maintaining automated checks. Obviously there are others ways that you could achieve these benefits but it would be foolish to dismiss the impact that CD can have on a development team.

Maybe the value of CD only starts to appear when you have multiple development teams contributing to a single code base. With a single developer and a single codebase the thought of releasing multiple times a day points to poorly written and tested code being endlessly thrust upon the user. If you consider a CD approach for multiple small development teams working on a shared codebase then you can see that if you do 3 releases in a day you are not necessarily saying that each developer released in that day. In fact at Songkick our release problems increased as the development team grew. Each developer was spending around 1-3 days on building and testing new features but this alongside regular bug fixing and technical improvements meant we were exceeding our release capabilities.

We could see that creating a process that allowed developers to commit code freely and receive fast feedback would be a huge improvement. Once we had made the necessarily cultural and technical changes to our process, attitude, and environments, we realised that with just a few additional changes we could ship our code directly to production without increasing the risk beyond our comfort zone. Taking this final step was hugely motivating for the development team; we wanted to create an environment where everyone has access to the information they need, can make decisions based on their expertise, and has access to support when they want it. At Songkick we did this by giving all developers the freedom to ship code whenever they were ready.

Unfortunately it seems rare for a team to sit down and define their release process, instead they rely on some process which evolves over time to handle new tools or respond to problems. Business value can only be achieved if you have code that actually makes it to the hands of your users. Being able to release the code to production in minutes doesn’t remove the need to define, design, build, and test changes but it can help you to develop a reliable release process.

I consider CD to be a technical solution to the problem of release delays. As with all releases there is likely to be some impact to the user, consider what this means and then plan accordingly.

Look at the risks of every release and identify actions to monitor or mitigate them. CD gives you a hugely flexible release process, you have the means to release quickly and frequently, but you also have the choice over whether you use it.

Build Quality In

Build Quality In

I’m excited to announce that Songkick‘s experience of moving to Continuous Deployment will be included in the forthcoming book “Build Quality In” edited by Steve Smith and Matthew Skelton.

The book, available on Leanpub aims to gather together experiences, lessons learnt, and the highs and lows of building quality in. I’ll be contributing my experience of moving to continuous deployment and in particular taking a look at the impact the change had on testing and quality at Songkick.

Testing In The Pub – Part one of my interview about Continuous Delivery
Part one of my interview with Testing in the Pub is now live. You can download it from http://testinginthepub.co.uk/testinginthepub/


Here are a few links for further reading (can also be found in my comment on the Testing in the Pub website):
– The place to start – http://continuousdelivery.com/
– Etsy’s developer blog and in this post in particular – http://codeascraft.com/2011/02/04/how-does-etsy-manage-development-and-operations/
– There are some interesting webinars available on http://www.thoughtworks.com/continuous-delivery
– Videos from the Pipeline conference – http://web.pipelineconf.info/2014/05/29/videos-from-pipeline-2014-are-online/


In addition I personally enjoy the following blogs:


And soon there will be a great book of experience reports all about CD. OK so I might have a report included but It’s still going to be a great book 🙂


Please comment with links to other blogs or resources that you find interesting or useful.

Let’s Test 2014 – Reflections
This time last year I had just spent a painful week watching the Let’s Test attendees tweet their way through a seemingly amazing week. I decided that I would make sure I didn’t miss out this year.

Now having returned home and taking a few days to catch up on sleep and digest the many many conversations I can honestly say that buying a ticket to Let’s Test was one of my better decisions in life.

As James Bach said before the event this is a conferring conference. The talks and workshops were interesting and useful but definitely less polished that at typical conferences. Everybody seemed to be attending the event to meet people and share ideas rather than because there was a particular talk that drew them there.

The conversations that took place during the conference were invaluable. I’m going to be digesting all the brilliant ideas and suggestions for some time. Expect blog posts on more specific topics to appear over the coming months.

For now I thank Henrik, Johan, and Ola for creating such a fantastic experience and thank all the people who I shared this great conference with.

I’ll leave you with a few photos.

Image
Tim Lister opens the conference with his Keynote

Image
The Sketchnoting workshop gets serious with Ilari and beer.

Image
Just a regular start to Tuesday.

Image
Alessandra Moreira presents.

Image
Michael protects a (covered) bowl of bacon from some hungry testers.

Image
Learning skills that have no real application in life.

Image

Testing in a Continuous Delivery World

The video of my recent talk at the LondonCD Meetup group is now live. The talk was kindly filmed by BBC Future Media.

May 2014 – Amy Phillips – Testing in a Continuous Delivery World from Software Engineering Practice on Vimeo.

Slides are also available at

Instigating change
In my last post I defined different approaches that people adopt for dealing with bad practices. Consider a bad practice to be anything which is damaging to a project or your sanity. This could be something that has an obvious negative impact such as excluding testers from early feature conversations, or maybe a more personal issue such as feeling you are unable to test effectively because you have too many interruptions during the working day.

Many people just accept that company culture is set in stone, either taking for granted that things are good, or suffering through the bad times. Maybe you spend your evenings and weekends complaining to your loved ones or suffer from the Sunday night ‘back to work dread’. I’m a huge believer in individuals recognising and driving improvements to process and projects. No matter what your level within the company be pro-active in identifying changes and do whatever you can to make things better.

How you go about actually making changes is somewhat down to your personal preference. At the highest level you need to know what the problem is, have some ideas on how it could be solved or improved, and then take steps to gain buy-in from the team.

I’m trying to convince my boyfriend that we need a dishwasher. In truth I’ve been trying to convince him for about 18 months now. We both commute so washing up is ignored until we run out of something, usually the tea spoons go first. With a mountain of dishes to get through we end up spending a whole evening cleaning up. We both agree that the problem is washing up takes up too much of our free time.

Initially when I proposed a dishwasher as a solution he was a total no-no. We would lose a cupboard in the kitchen, we would need more plumbing, environmentally it was a terrible idea, and have you seen what they do to the glasses?! I left it and then gradually drip-fed him reasons why this would be a great idea. I told him about the new dishwashers that actually use less water than washing up by hand, I emphasised how much free cupboard space we have these days, I sighed over another evening spent washing up. Basically, without him even realising it he was hearing the benefits of making the change on a regular basis. Finally the glasses, I can’t disagree with him that dishwashers really do scratch glasses so I offered a compromise, even if we wash glasses by hand it will be better than having to do everything. We have reached agreement that a dishwasher is the solution to the problem. Now we just need to work out the practical aspects of achieving the goal.

Workplace problems can be resolved with a similar approach.

Identify the problem:

Make sure you are focusing on the problem and not the solution. Your problem will never be that you aren’t following Continuous Delivery or that you don’t get invited to a particular meeting. You might find that you can identify several quite different problems. In this case choose the most problematic, or the most visibly problematic to solve first. Try to understand the reasons why the problem exists, was the situation introduced intentionally and if so, why?

Find potential solutions:

Spend some time researching how you could solve this problem. Try to identify as many possible solutions as you possibly can, you might be surprised at how a seeming crazy idea can turn out to be a brilliant solution to the problem. Aim to come away with one or two potential solutions that you can use to gain buy-in from the team. Be careful not to have a fully defined solution that could make people feel like you’re trying to dictate change.

Blog posts, videos of conference talks, meetups etc can all offer ideas for solving problems even if they don’t immediately seem to have much in common. Take care not to fixate on a particular approach just because it seems trendy right now.

Set an end goal that the whole team agrees with:

To be successful you need at least the majority of the team to buy-in to your solution. How you go about this will depend on your team as well as on your personal preference. You may prefer to present to the whole team at a regular team meeting, or you might prefer to meet with people on a one-to-one basis to recruit foes to your cause. I find it helpful to sound out my ideas on several honest, and smart people within the team before presenting to everyone. Hopefully they’ll point out any obvious flaws and help you to build a stronger case. Try to address the whole team as soon as you feel confident about your solution.

There will always be resistance but there will probably also be valid concerns about making a change, or towards your proposed solution. Engage these people in conversation and make sure you understand their reasons for resisting. Work with them to help them overcome the fear of change, particularity if the proposed change will affect the way they work.

Don’t worry about how ambitious your proposed solution may seem. If everyone has a clear idea of where you want to end up then tiny steps will lead you to the goal. Keeping moving forwards and one day you will arrive at your destination.

Persevere.

Change is difficult. Do it anyway.

The conditions you face day-to-day are one of the key components to your workplace happiness. Most people have some opportunity to improve the way they work so look critically, dream big, and be courageous in driving change.

When good practices turn bad
Over time everything evolves. Project’s evolve as new people join and as new information becomes available. Testing evolves as we learn to think and reflect on past projects. What was considered good testing ten years ago is probably not considered good testing today. There are processes and ideas that I look back on and wonder not only how I could have ever considered it to have been a good thing but to actually wonder how on earth it even worked. In my defense I should say that at the time, on that particular project, it did work.
Sometimes you’re working with something that was always known to be a bit of a compromise. We can’t be perfect all of time, for most of us we can’t be perfect for any of the time. All of this leads to the inevitable situation where you’ll be following a process, or carrying out a task and you’ll know that this is actually a bad thing* to be doing.
For example I know that test plans take far too much time to produce and maintain. I know that they can actually limit your thinking and cause you to miss bugs. However I currently work in an environment where sometimes I need people who aren’t testers to carry out some checking. I have to accept that they are only comfortable doing this if they have something to follow. Since I believe our process is effective, and I know our process depends on people besides myself (the only tester) testing, I’m reasonably OK with the need to support test planning to some degree. Obviously I can work on different approaches to this test planning but right now I’m OK to write test plans if they get me to a better place in the short-term, by improving things for the end-user.
Estimating is another common area of contention. We all know that it is impossible to estimate accurately. The anchoring effect will make sure that any number at all could be the resulting estimate. Yet, despite this, there are many people estimating work on a regular basis.
An individual taking a stand is unlikely to have any impact on a project. If you suddenly decide to stop providing estimates, then I doubt everyone else on the project will sigh with relief and drop the whole charade. It’s far more likely that you’ll be excluded from a meeting which could contain genuinely useful conversations. You’ll probably also end up being assigned estimates for your work, and then somehow expected to meet them.
So what do you do? Is it enough to go through the motions in an I KNOW THIS IS BAD kind of way similar to the way you know the late night kebab is bad but you do it anyway?Do you adopt stealth tactics and slowly try to raise a rebellion? Depending on the activity and project this might be successful. Unfortunately there will usually be at least one supporter for every activity, no matter how antiquated it has become. Maybe you’ll be satisfied that you at least had a go at making a change even if it wasn’t successful. This approach seems to be particularly common if the person supporting the bad thing is your manager.
Stealth tactics can be effective for making the small early steps. They can be used gather some evidence to support your case, or for identifying who will be open to change and who will resist. Unfortunately at some point you will have to come out into the open and launch a full-scale mission to rid your project of the bad practice. Keith Klain has given some great presentations about how to go about this. The biggest takeaway is probably that you need a lot of energy and perseverance to succeed.
So what do you do? Are you a ‘accept it as it comes’, ‘stealth changer’ or a ‘full-scale mission’ kind of person?

*Obviously there are measures of bad, I’m reasonably open-minded to things which are mostly bad but give you some overall gains. But there are also things which are just bad, bad, where the perceived gains are actually just an illusion hiding the reality of your actions.