Category Archives: Work life

2025 in review

Leave a reply

My first year of retirement has passed so quickly and, of course, has been filled with different activities than when I was working part- or full-time. Keeping the tradition going, though, I’m taking the opportunity to reflect on the year that was 2025.

Vital statistics

As I mentioned at the end of 2024 when I essentially retired from the testing industry, I expected my activity on this blog to reduce considerably – and I only published 4 blog posts this year (including this one).

Surprisingly, my total views for 2025 were still around the annual average since I started blogging here in 2014. For the first time in many years, I didn’t critique the World Quality Report and those annual posts had been some of my most popular.

While I still have a presence on Twitter/X (and closed out the year with almost 1,200 followers), I’m no longer posting on X and never visit my feed there. I’m still pretty active on LinkedIn, which is where most of the action around testing now seems to occur anyway.

Writing my first Status Quo book

Many of you probably know that the British rock band Status Quo have been a lifelong passion (and somewhat reflected in my X and WordPress handle, therockertester!). I’ve run one of the most popular Quo websites for more than twenty years – Access All Areas – and maintain the best online gig history for the band there.

I’ve toyed with the idea of writing some kind of Quo book for many years. With more free time in retirement, 2025 became the year when the idea came to fruition! I’ve spent most of the year working on a detailed history of the band’s tours in Australia and New Zealand (of which there were 14 across a period of almost 45 years). It’s been a long process, but very enjoyable. The content is basically finished now and I’m hopeful of going to print early in 2026 (as an A4-sized full-colour hardback tome).

Work life

I haven’t engaged in any testing-related work this year, but have continued some one-on-one mentoring relationships. After a long break, I’m looking forward to sharing my testing knowledge and experience to help a Melbourne organization in an advisory capacity early in 2026 (and I remain open to other gigs like this).

Testing books

I was delighted to be involved in Taking Testing Seriously, the epic new testing book from James Bach and Michael Bolton.

I contributed chapter 20 “From RST to AST” in which I describe my personal experience with the Rapid Software Testing approach and how it led to my deep involvement with the testing community and particularly the Association for Software Testing.

I hope this book reaches a broad audience it deserves – it’s beautifully crafted and treats testing in a way that no other textbook has ever done.

Volunteering for the UK Vegan Society

I continued with my volunteer work for the UK’s Vegan Society by contributing mainly to their web research efforts.

The process of building a completely new website for the Society still continued this year and most of my efforts involved testing it. It was good to be “hands on” and providing value to the organization using my existing skillset. I also tested new versions of the VeGuide mobile app.

I published three new blogs for the Society in 2025:

I enjoy blogging on veganism – it utilises my writing skills and feels like a good fit in terms of vegan activism for me.

As a result of my travel blogs for the Vegan Society, I was interviewed for the World Vegan Travel podcast . I shared my tips for travelling as a vegan in and around Melbourne in this enjoyable interview with Brighde Reed.

Coffee blog

With more spare time in retirement, I started another blog – this time about coffee! I enjoy a great oat latte and my blog’s name reflects this, In Search of the Perfect Oat Latte. I blog about every new coffee place I try and I also post similar content on Instagram @theperfectoatlatte.

Reading

I’ve enjoyed using my extra free time to read a lot in 2025, again largely thanks to the great service from Geelong Regional Libraries.

Looking back on my review of 2024, I mentioned that Rolf Dobelli’s plea in his excellent book “Stop Reading The News” was one of the most impactful reads of that year. I’m happy to say that I have successfully broken my addiction to following the news and would strongly recommend it – there really is now downside.

I read 33 non-fiction books and 2 fiction (by a local author who literally lives a few doors down on our street!). In terms of themes, I found myself heavily into AI, veganism, vaccines and the COVID pandemic response.

My most impactful reads were pretty diverse this year.

Will Guidara’s excellent Unreasonable Hospitality was really inspiring. His approach to leading people and organizations is so refreshing and, while it’s a story based around running restaurants in New York, his ideas are of great value to anyone who’s tasked with creating a great place to work. (I blogged about Will’s book here.)

The Age of Surveillance Capitalism (by Shoshana Zuboff) was insightful and a welcome reminder to keep pushing back on the encroachment of tech into more and more aspects on my life. And, yes, I’m one of those people who still uses cash whenever I can – central bank digital currencies are just around the corner if you don’t resist and ask yourself whether you really want programmable money (when you’re not writing the programs…).

Dissolving Illusions (by Suzanne Humphries and Roman Bystrianyk) is an important book, challenging the historical narrative around vaccines. It’s well worth a read for people new to the medical freedom/vaccine space, especially those open-minded enough to accept the possibility that they’ve been lied to by the medical profession.

Controligarchs (by Seamus Bruner) was a great read. While I consider myself well-informed when it comes to many of the topics covered by Seamus, he constructs a compelling narrative and backs it up with a lot of research (and amazingly deep “following the money” threads). The book serves as a good wake-up call for anyone who still thinks “philanthropists” and worldwide organizations (WEF, WHO, etc) are actually trying to help us.

My reading for the year is detailed below:

Non-fiction

Very Bad People (Patrick Alley)
McMafia (Misha Glenny)
Toxic (Richard Flanagan)
Code Dependent (Madhumita Murgia)
Chill & Prosper (Denise Duffield-Thomas)
The Internet Is Not What You Think It Is (Justin E H Smith)
The Coming Wave (Michael Bhaskar and Mustafa Suleyman)
Braving The Wilderness (Brene Brown)
Dissolving Illusions (Suzanne Humphries and Roman Bystrianyk)
Unreasonable Hospitality (Will Guidara)
Futureproof (Kevin Roose)
AI Needs You (Verity Harding)
Follow The Science (Sharyl Attkisson)
The Age of Surveillance Capitalism (Shoshana Zuboff)
Australia’s Pandemic Exceptionalism (Richard Holden and Steven Hamilton)
The Art of Bleisure (Emma Lovell)
AI 2041 (Chen Qiufan and Kai-Fu Lee)
Techno Feudalism (Yanis Varoufakis)
Upstream (Dan Heath)
Brave New World (Aldous Huxley)
When The Body Says No (Gabor Mate)
Ikagai (Francesc Miralles and Hector Garcia)
You Look Like a Thing and I Love You (Janelle Shane)
Controligarchs (Seamus Bruner)
Data Grab (Ulises A Mejias and Nick Couldry)
How to Argue with a Meat Eater (and Win Every Time) (Ed Winters)
Four Thousand Weeks (Oliver Burkeman)
Eat To Live (Joel Fuhrman)
Hanging By A Thread (Erin Deering)
Eat For The Planet (Zachariah and Stone)
Main Street Vegan (Victoria Moran)
All In (Mike Michalowicz)
Tools of Titans (Tim Ferriss)
The Golden Years (Jamie Nemtsas and Drew Meredith)

Fiction

The Maw of the Beast (Rick Wilkinson)
Poppy Day (Rick Wilkinson)

In closing

My first year of retirement has opened up time for following other passions this year and I’ve thoroughly enjoyed working on my Status Quo book while continuing to volunteer within the vegan community.

I wish you all a Happy New Year!

(Featured image for this post by Joshua Michaels on Unsplash)

Sign saying "Don't look back, you're not going that way"

Why the testing industry is the way it is

Leave a reply

I was inspired to put virtual pen to virtual paper again by a LinkedIn post from my good mate, Paul Seaman, lamenting his experience of spending nine months looking for a new testing role in Melbourne (Australia):

During 9 months of job searching it was hard not to notice that the job market for software testers is broken. Not just a little broken, a lot broken…

…we have the job ads that ask for a million different things and tools. I was told by a recruiter that, in a market like the current one, it’s a form of filtering. We both agreed it’s a particularly poor filter. I suspect it’s more fundamental. Many companies seeking a tester do not know what they need so they resort to a “wish list”.

Paul asks how the testing industry got to be this way and that got me thinking. When you look at a system and it seems completely broken or makes no sense, it’s worth thinking about how it could make perfect sense just the way it is. The US “healthcare” system is a perfect example, it’s not broken for those who’ve architected it to be the way it is – far from it!

We know how systems become the way they are thanks to these sage words from Jerry Weinberg:

Things are the way they are because they got that way

So, what lens can we use to look at the current testing market and see it making sense? Who benefits from the way it is? Who decided it got this way?

I’m aware that many other folks in the testing community have charted the history of software testing in various different ways. What follows is my take on how historical events (and not just within testing itself) have led us to the current state – you may agree or disagree with my analysis and I invite further debate on the topic.

In my opinion, the testing industry has been shaped into its form today by the following factors (presented in somewhat chronological order). I will discuss them individually but, as will become clear, they’re intertwined and exert forces between each other as well as on the industry as a whole.

  • The Agile & DevOps movements
  • ISTQB certification
  • Commodification
  • SDETs
  • The “testing is dead” narrative
  • Keyword-driven recruiting
  • Surveillance capitalism

(No, I haven’t forgotten about AI, I’ll come to that in my closing remarks.)

The Agile & DevOps movements

The early 2000s saw the agile movement starting to gain traction, with DevOps coming into the mix towards the end of the first decade of the new millennium. I’m covering both of these movements together as their impacts have amplified each other in many ways, I think.

Both movements talk about faster feedback loops and don’t formally acknowledge the idea of testing being a speciality in terms of role. As both of these movements have become the dominant paradigms for modern software development (despite their adoption often not adhering to their foundational practices – yes, I’m looking at you, organizations with a “DevOps team”), it’s no surprise that testers have been devalued.

Organizations have institutionalized the utopian vision of machines rapidly & cheaply checking their software products instead of “slow & costly humans” critically evaluating them (and the conflation of human testing and “automated testing” is a consequence of the widespread organizational ignorance around testing).

Both of these movements have been very well-resourced and popular certification programmes further their financial clout, so there has been no shortage of high-profile coverage of the benefits of both Agile & DevOps in major IT and business conferences, industry publications and so on. You only need to look at the strong focus on these movements in CapGemini’s “World Quality Report” to understand their reach into the testing and quality management arenas. (I’ve critiqued these reports in previous blog posts: 2018/19, 2020/21, 2022/23, 2023/24 and 2024/25.)

That organizational decision-makers have gone “all in” on these approaches was an entirely predictable outcome – adopting them as the de facto way in which software development teams now operate across their organizations.

ISTQB certification

It’s over twenty years since the ISTQB was founded and they have issued over a million certifications in over 130 countries (according to their own data from May 2025). The lack of other software testing certification schemes created the perfect environment for the ISTQB’s offerings to flourish and they were highly successful in marketing their certifications as the “industry standard” especially in the 2005-2015 period (based on my own experience). Though they had no genuine authority, they created the “ISQTB as industry standard” narrative. While skilled practitioners questioned the value of these certifications, they provided an opportunity for candidate filtering that was too good to waste and they were subsequently viewed as mandatory for many testing positions for a long time.

The simplicity of obtaining the Foundation certification helped to create the illusion that testing is easy and, as such, anyone can be quickly trained to be competent. Treating testing in such simplistic terms inevitably helped it become seen as a commodity service (more on that later).

The ISTQB and its local boards actively promote the idea that they are non-profit organizations, but the accredited training providers associated with them are generally not – and are often owned or serviced by members of the boards (which would seem to be a conflict of interest). The market value of the certifications themselves along with the training courses around them is in the order of millions of dollars per year. This significant financial clout has been used to influence decision-makers especially in larger organizations, with a trickle-down effect on the industry more generally.

Commodification

With testing being seen as easy and capable of being performed by machines – via the forces of agile, DevOps, easy certifications, etc. – testing skill became conflated with deft operation of the machines or tools, rather than in the creative intellectual evaluation and exploration of the software.

It was then an inevitable “race to the bottom” for the humans left behind. This industrial revolution of testing resulted in competition only on price, with outsourcing to low-cost locations becoming more and more common.

SDETs

The SDET (Software Development Engineer in Test) role originated in the early 2000s and was popularized by Microsoft, who made a lot of noise about the fact that they no longer had testers, only SDETs.

Like sheep, other big players quickly followed suit, including Google with their version, the Software Engineer in Test (SET). As the big names talked up this new approach, many other organizations latched onto the idea and human testers all over the world found themselves out of favour (and often out of work).

The need for engineers who could both write code and the automated tests for it arose out of the agile and DevOps movements, but the move to SDETs critically missed where human testing added value (or ignored it in the interests of speed, automation, commoditization, etc.). The terrible user experience of Windows Vista released during the height of the SDET frenzy should have been taken as a sign that removing the human elements of testing was probably a bad idea.

SDETs, in practice, were likely to be much better developers than testers and the role seems to have fallen from favour in the last decade. It’s now common to see agile teams with developers and no SDETs or testers, based on the theory that developers can do all the testing, whether that be coding automated checks or performing human testing. I again see this notion as being based on other influences rather than facts, such as the devaluing of testing skill promoted by easy certifications or the perceived need to increase the speed of delivery.

The “Testing is dead” narrative (c.2011)

At the large STARWest testing conference in 2011, James Whittaker (then at Google) announced that “testing is dead” with testers no longer being required in a world of automated checks and automatic updates. A high-profile name from a high-profile company like Google guaranteed that the message would reach far and wide. It was music to the ears of the SDET fanboys and proof positive that human testers were a historical relic whlle the new, faster, better software development world marched on.

The death of (human) testing has been proclaimed so many times in my 25-odd years in the industry (for various reasons), yet human testers still exist in many software development teams. It’s almost as though the humans bring something to the table that the machines cannot, although some organizations are steadfast in their refusal to admit it.

Keyword-driven recruiting

This will probably feel alien to younger folks, but back when I first started work (and for some time afterwards), job ads were largely focused on broad capabilities like “problem-solving,” “communication” or “managerial experience”. Tools were often learned on the job and there was more on-the-job training, so it was uncommon for particular tools to be part of job ads.

With the internet boom in the 2000s, online job boards normalized searchable skill and tool keywords. Employers started to assume that general skills were not enough and applicants had to be “ready to go” with experience in the right tools for the particular job.

Over time, tools became more closely tied to workflows so experience in them was viewed even more favourably so that a new starter could “hit the ground running” in a Jira shop, for example. Companies that make such tools also push for credentialing and adoption, which filters into hiring norms.

With the digital transformation in full swing, tools then become more central and it was a perfect storm once Applicant Tracking Systems scanning for exact terms in resumes were employed en masse by recruiters – the age of “keyword-driven recruiting” was upon us.

Long laundry lists of tools rapidly became a feature of most job ads for testers and lazy recruiting practices were at least partly to blame. Smart testers learned how to manipulate the system, by using text like this as suggested by Michael Bolton:

I do not have an ISEB or ISTQB certification, and I would be pleased to explain why

But too many simply fell into the trap of focusing on toolsmithing rather than becoming excellent testers, just to feed the filtering beasts.

This keyword-based approach excludes many great candidates who are perfectly capable of picking up and learning new tools as required, but who haven’t used the exact tools to pass through the automated filtering process. It also overemphasises tools over core competencies and yet it is these more fundamental skills of the craft that are much more durable and essential to completing testing missions with credibility.

This move towards keyword-based recruiting has negatively impacted the hiring process for genuinely good testers, in my opinion.

Surveillance capitalism

“Surveillance capitalism” is a term used to describe a new economic system centered around the extraction, analysis, and commercialization of personal data. It was popularized by Shoshana Zuboff in her excellent book, The Age of Surveillance Capitalism.

One of the most obvious characteristics of surveillance capitalism is the commodification of the human experience. Human behaviour becomes a raw material: your clicks, likes, movements, conversations and even emotions are turned into products. These raw materials are the fuel used to predict and modify behaviour for the benefit of their actual customers (not their users who are merely seen as the sources of these raw materials).

The dehumanizing impact of surveillance capitalism is clear. Attempting to track, monitor, instrument, analyze, predict and modify every aspect of our world – from our virtual interactions (e.g. tracking our searches) out into to the real world (e.g. tracking our movements via GPS and mapping services) and right into our very beings (e.g. wearables and facial image recognition) – has become an accepted part of modern life. In doing so, these approaches feel less alien than they should and so removing humans from the picture in other aspects of life becomes normalized too. The move away from skilled human testers towards toolsmiths and machine operators thus seems completely natural to the current generation of software development professionals.

What about AI?

In the discussion above, I deliberately left AI out of the list of factors I think have contributed to the current state of testing. The factors I’ve identified have all played their part in my opinion, some more significantly than others. The impact of AI, though, is only just starting to hit our industry – and I fear that it will make all of these factors look very minor in terms of their impact. I realise that I’m writing this in the middle of a huge hype cycle around AI, but the “loading up” on all things AI is important to analyze, both from the viewpoint of the testing industry but also across software development & IT more generally.

The stage really has already been set for dehumanization as I’ve outlined above, so I’m not surprised that I don’t see too much resistance to the idea of “AI” replacing testers and other IT professionals. I don’t believe that skilled testers can be replaced by current AI systems, so I urge testers to navigate this time by focusing on being more human and not trying to behave more like the machines that look set to replace them. Being aware of the benefits and limitations of AI is important, as is seeing these systems as assistants or tools to help you do better or different testing, but not replacements for your humanity.

Who is the current system working well for?

Looking through the lens of testing tool vendors, the current state of the testing industry is looking good. More and more organizations are using more and more toolsets to assist with testing and agile & digital transformation projects tend to result in a move towards more tooling and less human testing. These vendors have deep pockets and can influence the testing space through their advertising, sponsorship of testing conferences and so on.

The AI vendors will also see the testing industry as being in a sweet spot for exploitation, with the stage set by years of talking about “testing is dead”, automating away the humans and surveillance capitalism’s normalizing of a dystopian world.

Recruiters seem to love the keyword-based filtering of applications, filtering down the massive number of applications (for fewer and fewer pure testing roles) to more manageable stacks to follow up.

For human testers, though, the moulding of the industry into its current form hasn’t been beneficial and, frankly, is likely to become even worse as AI hooks into more and more aspects of the development game.

So what?

The testing industry is what it is, shaped by many different forces over decades. For human testers, the time to be vocal about the value you offer is now – before it’s too late. The tidal wave of AI is heading your way and you can’t make a snorkel long enough to breathe through it – instead, head for higher ground where you can see the wave crashing in, while bringing your distinctly human skills to the table of those organizations still seeing value in what you bring.

There’s a big role to be played by those professional organizations representing testing as a craft, such as the Association for Software Testing. These kind of voices carry weight and are less easily silenced by the lobbying and financial weight of the players looking to dehumanize our craft.

Focus on being more human, not more like the machines, and build communities of like-minded folks. The industrial revolution transformed manufacturing with its factory model and many people were (and are) content to buy factory-produced low-cost goods. But there are also plenty of other people who want a more artisan, hand-made, craftperson experience behind their purchases. Excellent human testing is the same and there will, I believe, always be a market for the true craftspeople – go find it… or help to create it!

(Featured image on this post by Yusong He on Unsplash)

A cute dog peeks from under a bed

A privileged peek

1 Reply

Now that work doesn’t consume any of my days, I’ve been finding even more time to enjoy reading (making extensive use of the amazing service provided by Geelong Regional Libraries).

I’ve just finished Will Guidara’s excellent Unreasonable Hospitality and I found it really inspiring. His approach to leading people and organizations is so refreshing and, while it’s a story based around running restaurants in New York, his ideas are of great value to anyone who’s tasked with creating a great place to work.

The book centres on the fine dining restaurant, Eleven Madison Park – interestingly, it transitioned to being vegan in 2021 after Will’s time there and is one of the few such restaurants to achieve Michelin stars. I’ve been lucky enough to enjoy dining at one of the first vegan Michelin-starred restaurants, Kajitsu (also in New York back in 2014 and sadly closed in 2022), but haven’t had the good fortune to get to Eleven Madison Park – yet!

One line from his book stood out to me as a great takeaway in how to reframe bad experiences (in his case, talking about his terrible treatment by a chef in the early days of his restaurant career):

…while it was a terrible experience, it was also a privileged peek at a mistake I never wanted to make

I can relate to his message, both in my personal and professional life. While I’ve been incredibly fortunate to generally have supportive managers during my career, there have still been occasions where I’ve witnessed behaviour – both directed towards me and others – that have offered me that privileged peek into ways I never want to relate to others.

I hope I model this behaviour and pass it on to my mentees so that they too can help shape better workplaces in the future. (I’m still open to taking on mentees, by the way.)

PS: While I’m spending more of my time reading these days, I’m investing much more in my first retirement writing project…. but that’s a story for another day!

(Featured image by Yevhen Stienin on Unsplash)

2024 in review

Leave a reply

Another year flies by, so I’m again taking the opportunity to review the year that was 2024.

Vital statistics

I only published 7 blog posts this year (including this one), again not meeting my personal target cadence of a post every month. I haven’t been finding much in the way of inspiration to post, but 2024 has still set a record for total views which I find quite amazing after 10 years of blogging! The traffic is slightly skewed by the fact that my most popular posts – viz. my critiques of the World Quality Report – appeared twice during 2024, one a belated review of the 2023 report and the other for 2024’s effort.

I’m still on Twitter/X and closed out the year with just over 1,200 followers, slightly down from last year. I’m no longer posting on X and it appears most of the interesting testers I used to follow have already left the platform. I’m seeing almost all of my engagement coming from my posts on LinkedIn now.

Work life

I spent the year working part-time for SSW in my role as Test Practice Lead and all of it for the same government agency I started working with in 2023. I had a great experience in this agency and my tenure there and at SSW have now both just come to an end.

In my own business, Dr Lee Consulting, I again focused on my Mentoring offering and have found one-on-one mentoring very rewarding.

I’m not intending to look for new opportunities at this stage of life, but maybe an interesting project or two could tempt me away from more pleasurable pursuits in the years ahead!

Testing-related events

For the first time in maybe 15 years, I didn’t attend any virtual or in-person testing conferences or meetups during 2024, nor did I give any presentations. It’s perhaps a sign of my deliberate choice to wind down that I opted out of opportunities during the year after a long stint of contributing to the testing community.

Testing books

I wrapped up the content for the free AST e-book, Navigating the World as a Context-Driven Tester. This book provides responses to common questions and statements about testing from a context-driven perspective, with its content being crowdsourced from the membership of the AST and the broader testing community. The final version of the book contains 28 responses and continues to be freely available from the AST’s GitHub.

I didn’t publish an updated version of my book An Exploration of Testers during 2024 and the current version is likely to be the last. There were more purchases of the book, though, so I was happy to be able to make another donation to the Association for Software Testing’s excellent Grants program.

Reading

My strong reading habit continued during 2024, thanks to the great service from Geelong Regional Libraries. I again added a little fiction into the mix.

Of the 37 books I read this year, the most impactful were two very different reads. Firstly, Rolf Dobelli’s plea to “Stop Reading The News” was just what I needed to break my addiction to following the news cycle and I went cold turkey early in 2024, never to go back. I don’t feel like I’m missing out on anything, apart from mainstream media’s propaganda. Secondly, snooker player Ronnie O’Sullivan’s biography “Unbreakable” was an inspiring read. Although I’ve followed his entire career, his many challenges and work on his mindset were interesting to read about – and he remains at the very top of the sport despite his age.

My reading is detailed below:

Non-fiction

  • The Myth of Normal (Gabor Mate)
  • The Courage to Face COVID-19 (John Leake and Peter A. McCullough)
  • Stop Reading The News (Rolf Dobelli)
  • The Paradox of Choice (Barry Schwartz)
  • Life As We Knew It (Aisha Dow and Melissa Cunningham)
  • How Innovation Works (Matt Ridley)
  • The Locked-up Country (Shahar Hameiri and Tom Chodor)
  • Ageless Soul (Thomas Moore)
  • The Upside of Stress (Kelly McGonigal)
  • Viral (Alina Chan and Matt Ridley)
  • Superforecasting (Dan Gardner and Philip E. Tetlock)
  • Same As Ever (Morgan Housel)
  • Ultra-Processed People (Chris van Tulleken)
  • Cobalt Red (Siddharth Kara)
  • Deadly Medicines and Organised Crime (Peter C. Gøtzsche)
  • Living Plantfully (Lindsey Harrad)
  • Read Write Own (Chris Dixon)
  • Oxygen (Patrick McKeown)
  • Breaking The Habit of Being Yourself (Joe Dispenza)
  • Shoe Dog (Phil Knight)
  • Slow Productivity (Cal Newport)
  • Lies My Government Told Me (Robert Malone)
  • Making A Killing (Bob Torres)
  • The Influencer Industry (Emily Hund)
  • The Psychology of Money (Morgan Housel)
  • The Way of Integrity (Martha Beck)
  • Unbreakable (Ronnie O’Sullivan)
  • The Violence of the Green Revolution (Vandana Shiva)
  • The Bodies of Others (Naomi Wolf)
  • Die With Zero (Bill Perkins)
  • The New Confessions of an Economic Hitman (John Perkins)
  • Unsettled (Steven E. Koonin)
  • How Not To Lose $1 million (John Addis)

Fiction

  • Changing Places (David Lodge)
  • Apples Never Fall (Liane Moriarty)
  • The Truth Teller (Angela Elwell Hunt)
  • I Know My Love (Catherine Gaskin)

Volunteering for the UK Vegan Society

I continued with my volunteer work for the UK’s Vegan Society by contributing to their web research efforts (and I didn’t tackle any proofreading jobs this year).

I came up with recommendations for changes to the website’s “Key Facts” page after reviewing other sites to define what a modern layout and content should look like for the page.

The process of building a completely new website for the Society continued this year and most of my efforts involved testing it. It was good to be “hands on” and providing value to the organization using my existing skillset.

I didn’t publish any new blogs for the Society in 2024, but started a couple of posts and expect those to be finalized early in 2025. I really enjoy blogging on veganism, both to flex my writing muscles and also to more deeply engage with vegan content.

Working with The Vegan Society continues to be a joy, I’m blessed to work with great people there who really appreciate my efforts. I expect to contribute more fully in 2025 now that my time in the testing game has come to an end.

In closing

I remain grateful for the attention & support from the readers of my blog and also my followers on other platforms. I wish you all a Happy New Year!

As I move on from the testing industry, this blog might not see too many more posts but I may be inspired to write again once in a while…

(Featured image for this post by Igor Kasalovic on Unsplash)

2023 in review

1 Reply

It’s time to take the opportunity to review my 2023, a year that has flown by.

Vital statistics

I only published 10 blog posts this year, so didn’t meet my personal target cadence of a post every month. I still enjoy blogging but just haven’t had as many triggers to create posts as in previous years for some reason. My limited activity has probably contributed to the drop in my blog traffic, down by about 40% compared to 2022. Traffic was much heavier during the first quarter than for the rest of the year, reflecting a string of public testing presentations during this time.

One of the most popular posts each year is my critique of the World Quality Report (which I published for the 2020, 2021 and 2022 reports). I missed the recent release of the 2023-24 report, so maybe I’ll get to wading through the latest version early in 2024 and posting my findings as usual!

I’m still on Twitter/X and closed out the year with about 1,250 followers on Twitter, up slightly from last year. I wonder how many testers are still around on this platform and I note that I’m seeing more engagement with my posts on LinkedIn than X now.

Work life

I’ve spent the year working part-time for SSW in my role as Test Practice Lead, the last eight months or so of which have been for a government agency. As my first exposure to government work, it’s been great with an excellent culture and friendly & welcoming colleagues. I’m looking forward to continuing to add value in this role during 2024.

In my own business, Dr Lee Consulting, I’ve focused on my Mentoring offering and have found one-on-one mentoring very rewarding. I find it to be a great learning experience as well as a good opportunity to pass on at least some of what I’ve learned along the way during 25 years in the IT industry. I’m keen to do more in this area, so please let me know if you or your colleagues might be interested in working together. I also launched a new offering, my Second Pair of Eyes service, but have yet to see any traction for it – again, I’d appreciate any leads to kick start this offering.

Testing-related events

I didn’t attend any virtual or in-person testing conferences during 2023, but I did three virtual presentations in the first few months of the year.

First up was an Association for Software Testing (AST) webinar as part of their “Steel Yourselves” series. The idea behind this series is to make the case for a testing idea/concept/approach that you strongly disagree with and I was tasked with defending the need for a testing phase in my session, “Shift Nowhere: A Testing Phase FTW”! I blogged about this experience and you can watch my stab at this difficult task in the second half of the following YouTube video:

I took part in my first “Ask Me Anything” thanks to a webinar by The Test Tribe in which I fielded questions about Exploratory Testing for an hour! This was something new for me and I found it quite challenging, but also enjoyable. I blogged about doing this AMA and a recording of the webinar is on YouTube:

I was pleased to be invited to speak for the Sydney Testers meetup group and I presented a brand new talk, “Lessons Learned in Software Testing” (a deliberate play on the awesome book with the same title) in which I offered a few (potentially contrarian!) lessons I’ve taken away from my long stint in the software testing industry. Thanks to Paul Maxwell-Walters for the invite and it’s great to see Sydney Testers continuing as a large testing community in Australia.

I blogged about my experience of giving this presentation and it was also recorded in full (my talk starts at 32 minutes into the following YouTube video):

I was once again invited to act as a peer advisor for one of Michael Bolton’s virtual “RST Explored” classes running in the Australian timezone. I enjoyed acting in this role back in 2021 and my 2022 experience was great too. I still feel that RST has incredible value and it’s so much more accessible in the virtual format (representing amazing value for money, in my opinion).

Testing books

I made solid progress on the free AST e-book, Navigating the World as a Context-Driven Tester. This book provides responses to common questions and statements about testing from a context-driven perspective, with its content being crowdsourced from the membership of the AST and the broader testing community. I added a further 7 responses in 2023 (bringing the total to 23) and it was good to see a number of new contributors through the year. I will continue to ask for contributions about once a month in 2024. The book is available from the AST’s GitHub.

I failed to publish an updated version of my book An Exploration of Testers during 2023, but hope to do so in 2024. I remain open to additional contributions to this book, so please contact me if you’re interested in telling your story via the answers to the questions posed in the book – and remember that all proceeds from sales of this e-book go to the Association for Software Testing’s excellent Grants program (with another donation from recent sales coming early in 2024).

Reading

My strong reading habit continued during 2023, thanks to the great service from Geelong Regional Libraries. For the first time in many years, I added some fiction into the mix and thoroughly enjoyed doing so. Of the 30-odd books I read this year, the most impactful was From Strength to Strength by Arthur Brooks, while the most engaging fiction came from Hugh Howey’s “Silo” series.

My reading is detailed below (with links to my tweets or blog posts on each of them):

Non-fiction

Fiction

Volunteering for the UK Vegan Society

I’ve continued to volunteer with the UK’s Vegan Society both as a proofreader and also contributing to their web research efforts. I didn’t tackle many proofreading jobs this year, focusing more on a number of web projects.

I came up with recommendations for changes to the website’s “About Us” page after reviewing many other sites to define what a modern layout and content should look like for the page. I also undertook the mammoth task of reviewing the “Statistics” pages to identify older stats that should be removed as well as researching newer ones to replace them.

The Society is in the process of building a completely new website and I’ve been heavily involved in testing it. It’s been fun to get my hands dirty with some real testing again and it’ll be great to see the new site going live soon!

Outside of my proofreading and web research work, I’ve also written a couple of blog posts, the first was on vegan Christmas desserts and the second on aquafaba! The audience and style are completely different when writing for the Vegan Society so, as a writer, I’m enjoying this challenge and hoping to pen some more blogs for them in 2024.

Working with The Vegan Society is really enjoyable and they handle volunteers very kindly. It was lovely to receive a “thank you” gift of a sponsorship of “Brucey the goose” at Good Heart Animal Sanctuary in recognition of my efforts this year.

In closing

I remain grateful for the attention & support from the readers of my blog and also my followers on other platforms. I wish you all a Happy New Year and I hope you enjoy my posts and other contributions to the testing community to come through 2024.

While I have no confirmed public appearances in 2024 yet, I’m sure I’ll be out and about somewhere either virtually or in-person so I’ll “see you” around…

(Featured image for this post by Jake Weirick on Unsplash)

Going “From Strength to Strength”

1 Reply

I recently read “From Strength to Strength” by Arthur C. Brooks and rarely has a book talked to me as directly as this important effort.

I realized after reading this book that I’ve been a striver for as long as I can remember, pushing myself to achieve and enjoy professional success. I was the star student of mathematics at university and then successfully completed my PhD in mathematics in a little over three years.

My professional career saw me climbing the corporate ladder, ultimately becoming one of six Director-level folks leading up the software development arm of a US$300m+ software business. When my 21-year stint at Quest came to an abrupt and unexpected end, I took some time to reset and ultimately started my own consultancy to continue on with my professional life in testing.

An important idea in the book is the difference between fluid and crystallized intelligence. In essence, fluid intelligence is your ability to process new information, learn and solve problems, while crystallized intelligence is the ability to use a stock of knowledge learned in the past.

This diagram illustrates how these types of intelligence generally change as we age:

Chart of fluid and crystallized intelligence as they vary with age

Brooks makes this observation, which struck a very strong chord with me:

“If you are experiencing decline in fluid intelligence – and if you are my age, you are – it doesn’t mean you are washed up. It means it is time to jump off the fluid intelligence curve and onto the crystallized intelligence curve. Those who fight against time are trying to bend the old curve instead of getting onto the new one. But it is almost impossible to bend, which is why people are so frustrated, and usually unsuccessful.”

I’ve witnessed older members of the testing community in Melbourne struggling with this as they try to fight against time and stay relevant in ways that really don’t make sense. I’m realizing I’ve naturally been shifting my focus to crystallized intelligence, out of necessity but also it’s where I get more fulfilment.

Brooks makes the point that:

“when you are young, you have raw smarts; when you are old, you have wisdom. When you are young, you can generate lots of facts; when you are old, you know what they mean and how to use them.”

He notes the inevitably of decline but also the hope that shifting perspective can bring with it new and exciting opportunities later in life:

“Almost without fail, you will notice the decline in the fluid intelligence portion. However, there always exists the ability to redesign your career less on innovation and more on instruction as the years pass, thus playing to your strengths with age.”

I recognize that I can no longer keep up with the younger folks in many ways, I have less and less interest in the latest tools and tech, and being on the critical path in day-to-day project work doesn’t bring me joy. But I’m finding my niche in helping to guide folks with my accrued wisdom:

“So what do the young hotshots need? Old people on product teams, old people in marketing, and old people in the C-suite. They need not just whiz-bang ideas but actual wisdom that only comes with years in the school of hard knocks.”

Brooks notes:

“Practiced properly, old people have an edge over younger people, because they have more experience at life and relationships”

and he also quotes the philosopher Marcus Tullius Cicero who:

“…believed three things about older age. First, that it should be dedicated to service, not goofing off. Second, our greatest gift later in life is wisdom, in which learning and thought create a worldview that can enrich others. Third, our natural ability at this point is counsel: mentoring, advising and teaching others, in a way that does not amass worldly rewards of money, power or prestige.”

I’m certainly finding a lot of enjoyment and fulfilment from my mentoring activities, there’s no frustration there and it’s probably a sign of already spending some or most of my time exploiting crystallized intelligence:

“The decline in your fluid intelligence is a sign that it is time not to rage, which just doubles down on your unsatisfying attachments and leads to frustration. Rather, it is a time to scale up your crystallized intelligence, use your wisdom, and share it with others.”

I think my mindfulness practice over the last few years has helped here too (and is also mentioned by Brooks in his book) and I’ve become much more comfortable showing vulnerability. I have no issue with saying “I don’t know” and feel no need for pretence – and have found in doing so that people generally seem to see me as more credible, rather than less:

“To share your weakness without caring what others think, is a kind of superpower”

The cool work of Chip Conley and his Modern Elder Academy gets a well-deserved shout out here too.

This is an important book and especially so for those of us later in our careers as we look for new meaning and ways to find satisfaction from sharing, teaching and so on. It’s a great read and a handy nudge to follow a different path – not better or worse, but one befitting our wisdom and also likely to be more fulfilling. As Brooks notes:

“Get old sharing the things you believe are most important. Excellence is always its own reward, and this is how you can be most excellent as you age”

I’d love to help more people so please check out the mentoring services I offer through my consultancy business, Dr Lee Consulting.

ER: Presenting at the Sydney Testers meetup (12th April 2023)

1 Reply

The Sydney Testers meetup has been Australia’s largest testing meetup for many years and I was more than happy to help when organizer, Paul Maxwell-Walters, was looking for speakers.

I already had a conference talk “in the can” due to being unable to present at the Melbourne Testing Talks conference in 2022, so my preparation only consisted of some minor updates to the slide deck and a practice run to nail down the slide transitions and timing.

The meetup took place on the evening of 12th April and I would be second up (presenting virtually over Zoom), following Ashley Graf‘s half-hour talk on “50 questions to faster onboarding (as a QA)”. A decent crowd formed during the first 30-45 minutes of the session and I took the virtual stage at just after 6.30pm.

My talk was titled “Lessons Learned in Software Testing” and I shared six lessons I’ve learned during my twenty-odd years in the testing industry. I guess some of my opinions and lessons are a little contrarian, but I’m OK with that as much of what I see presented as consensus around testing (especially on platforms like LinkedIn) doesn’t reflect my lived experience in this industry. If you want to know the six lessons that I shared, you’ll need to watch all 45 minutes of my presentation!

Thanks to Paul for the opportunity to present to the Sydney Testers audience and also for the interesting questions during the Q&A afterwards.

A recording of both talks from this meetup (as well as the Q&A) is available on YouTube (my talk starts at 32 minutes into this recording):

2022 in review

Leave a reply

It feels like much less than a year since I was penning my review of 2021, but the calendar doesn’t lie so it really is time to take the opportunity to review my 2022.

I published just 10 blog posts this year, so didn’t quite meet my personal target cadence of a post every month. There were a few reasons for this, the main one being my unexpected re-entry into employment (more on that below). Perhaps due to my more limited output, my blog traffic dropped by about 40% compared to 2021. I continue to be grateful for the amplification of my blog posts via their regular inclusion in lists such as 5Blogs and Software Testing Weekly.

March was the biggest month for my blog by far this year, thanks to a popular post about a video detailing how testers should fake experience to secure roles. I note in writing this blog post now that the video in question has been removed from YouTube, but no doubt there are similar videos doing the rounds that encourage inexperienced testers to cheat and misrepresent themselves – to the detriment of both themselves and the reputation of our industry.

I again published a critique of an industry report in November (after publishing similar critiques in 2020 and 2021) and this was my second most popular post of the year, so it’s good to see the considerable effort that goes into these critique-style posts being rewarded by good engagement.

I closed out the year with about 1,200 followers on Twitter, steady year on year, but maybe everyone will leave Twitter soon if the outrage many are expressing recently isn’t fake!

Work life

For the first few months of 2022, I continued doing a small amount of consulting work through my own business, Dr Lee Consulting. It was good to work directly with clients to help solve testing challenges and I was encouraged by their positive feedback.

Quite unexpectedly, an ex-colleague from my days at Quest persuaded me to interview at SSW, the consultancy he joined after Quest. A lunch with the CEO and some formalities quickly led to an offer to become SSW’s first Test Practice Lead (on a permanent part-time basis). I’ve now been with SSW for about seven months and it’s certainly been an interesting journey so far!

The environment is quite different from Quest. Firstly, SSW is a consultancy rather than a product company and I’ve come to realise how different the approach is in the consulting world compared to the product world. Secondly, SSW is a small Australian company compared to Quest being a large international one, so meetings are all standard working hours (and I certainly don’t miss the very early and very late meetings that so frequently formed part of my Quest working day!).

I have been warmly welcomed across SSW and I’m spreading the word on good testing internally, as well as working directly with some of SSW’s clients to improve their approaches to testing and quality management.

Testing-related events

As I announced mid-2021, I was excited to be part of the programme for the in-person Testing Talks 2021 (The Reunion) conference in Melbourne, rescheduled for October 2022. Unfortunately, I had to give up my spot on the programme due to my COVID vaccination status – though, surprise surprise, all such restrictions had been removed by the time the event actually took place. But I did attend the conference and it was awesome to see so many people in the one place for a testing event, after the hiatus thanks to the pandemic and the incredibly harsh restrictions that resulted for Melbourne. (I blogged about my experience of attending Testing Talks 2022.)

In terms of virtual events, I was fortunate to be invited to act as a peer advisor for one of Michael Bolton’s virtual RST classes running in the Australian timezone. This was an awesome three-day experience and I enjoyed interacting with the students as well as sharpening my understanding of some of the RST concepts from Michael’s current version of the class.

Two very enjoyable virtual events came courtesy of the Association for Software Testing (AST) and their Lean Coffees. I participated in the May and September events suited to my timezone and they were enlightening and fun, as well as offering a great way to engage with other testers in an informal online setting.

I had an enjoyable conversation with James Bach too, forming part of his “Testing Voices” series on the Rapid Software Testing YouTube channel:

Although I’ve interacted with James online and also in person several times (especially during his visits to Melbourne), this was our most in-depth conversation to date and it was fun to talk about my journey into testing, my love of mathematics and my approach to testing. I appreciate James’s continued passion for testing and, in particular, his desire to move the craft forward.

Testing books

I didn’t publish an updated version of my book An Exploration of Testers during 2022, but may do in 2023.  I’m always open to additional contributions to this book, so please contact me if you’re interested in telling your story via the answers to the questions posed in the book!

I made good progress on the free AST e-book, Navigating the World as a Context-Driven Tester though. This book provides responses to common questions and statements about testing from a context-driven perspective, with its content being crowdsourced from the membership of the AST and the broader testing community. I added a further 10 responses in 2022, bringing the total to 16. I will continue to ask for contributions about once a month in 2023. The book is available from the AST’s GitHub.

Podcasting

Paul Seaman, Toby Thompson and I kicked off The 3 Amigos of Testing podcast in 2021 and produced three episodes in that first year, but we failed to reconvene to produce more content in 2022. There were a number of reasons for this, but we did get together to work up our next episode recently, so expect our next podcast instalment to drop in early 2023!

Volunteering for the UK Vegan Society

I’ve continued to volunteer with the UK’s Vegan Society both as a proofreader and also contributing to their web research efforts. I’ve learned a lot about SEO as a result of the web-related tasks and I undertook an interesting research project on membership/join pages to help the Society to improve its pages around joining with the aim of increasing new memberships.

I really enjoy working with The Vegan Society, increasing my contribution to and engagement with the vegan community worldwide. It was particularly rewarding and humbling to be awarded “Volunteer of the Season” and be featured in the Society’s member magazine, The Vegan, towards the end of the year.

Photo of Lee with Lola in his arms, overlooking the beach, Corio Bay and the You Yangs Text is a Q&A about Lee's volunteer work with the UK Vegan Society

In closing

As always, I’m grateful for the attention of my readers here and also followers on other platforms. I wish you all a Happy New Year and I hope you enjoy my posts and other contributions to the testing community to come through 2023 – the first public opportunity to engage with me in 2023 will be the AST’s Steel Yourselves webinar on January 30, when I’ll be arguing the case for a testing phase, I hope to “see you” there!

A tester’s critique of “The 2021 State of Software Quality: The View from Enterprise Leaders & Followers”

5 Replies

I really should know better, but I decided to watch a webinar titled The 2021 State of Software Quality: The View from Enterprise Leaders & Followers from MicroFocus and Enterprise Management Associates, Inc. The promo spiel for the webinar read as follows:

The rapid rise of the digital economy became twice as important after layering on a worldwide pandemic. With every company having to become a software company, enterprise application development speed, volume, cost, quality, and risk are key determinants that define who survives and who does not. The pressure on application development teams to build more software faster and cheaper often runs counter to the objectives of software quality and managing risk.

Join Steve Hendrick, Research Director at Enterprise Management Associates, to hear key findings from a recent worldwide survey about software quality. This webinar will look at the characteristics and differences between software quality leaders and followers. Key to this discussion of software quality is the impact that people, process, and products are having on enterprise software quality. Completing this view into software quality will be a discussion of best and worst practices and their differences across three levels of software quality leadership.

While the opening gambit of this promo literally makes no sense – “The rapid rise of the digital economy became twice as important after layering on a worldwide pandemic” – the webinar sounded like it at least held some promise in terms of identifying differences between those “leading” in software quality and those “following”.

The survey data presented in this webinar was formed from 316 responses by Directors, VPs and C-level executives of larger enterprises (2000 employees or more). The presenter noted specifically that the mean enterprise size in the survey was over 11,000 employees and that this was a good thing, since larger enterprises have a “more complex take on DevOps”. This focus on garnering responses from people far away from the actual work of developing software in very large enterprises immediately makes me suspicious of the value of the responses for practitioners.

Unusually for surveys and reports of this type, though, the webinar started in earnest with a slide titled “What is Software Quality”:

While the three broad software quality attributes seem to me to represent some dimensions of quality, they don’t answer the question of what the survey means when it refers to “software quality”. If this was the definition given in the survey to guide participants, then it feels like their responses are likely skewed to thinking solely about these three dimensions and not the many more that are familiar to those of us with a broader perspective aligned with, for example, Jerry Weinberg’s definition of quality as “Value to some person”.

The next slide was particularly important as it introduced the segmentation of respondents into Outliers, Laggards, Mainstreamers and Leaders based on their self-assessment of the quality of their products:

This “leadership segmentation” is the foundation for the analysis throughout the rest of the webinar, yet it is completely based on self-assessment! Note that over half (55%) self-assess their quality as 8/10, 9/10 or even 10/10, while only 11% rate themselves as 5/10 or below. This looks like a classic example of cognitive bias and illusory superiority. This poor basis for the segmentation used so heavily in the analysis which follows is troubling.

Moving on, imagine being faced with answering this question: “How does your enterprise balance the contribution to software quality that is made by people, policy, processes, and products (development and DevOps tools)?” You might need to read that again. The survey responses came back as follows:

Call me cynical but this almost impossible to answer question looks like it resulted in most people just giving equal weight to all of the five choices, so ending up with just about 20% in each category.

It was soon time to look to “agile methodologies” for clues as to how “adopting” agile relates to quality leadership segmentation:

It was noted here that the “leaders” (again, remember this is respondents self-assessing themselves as quality leaders) were most likely to represent enterprises in which “Nearly all teams are using agile methods”. A reminder that correlation does not imply causation feels in order at this point.

The revelations kept coming, let’s look at the “phases” in which enterprises are “measuring quality”:

The presenter made a big deal here about the “leaders” showing much higher scores for measuring quality in the requirements and testing management “phases” than the “mainstreamers” and “laggards”. Of course, this provided the perfect opportunity to propagate the “cost of change” curve nonsense, with the presenter claiming it is “many times more expensive to resolve defects found in production than found during development”. He also sagely suggested that the leaders’ focus on requirements management and testing was part of their “secret sauce”.

When the surveyed enterprises were asked about their “software quality journey over the last two years”, the results looked like this:

The conclusion here was that “leaders” are establishing centres of excellence for software quality. There was a question about this during the short Q&A at the end of the deck presentation, asking what such a function actually does, to which the presenter said a CoE is “A good way to jumpstart an enterprise thinking about quality, it elevates the importance of quality in the enterprise” and “raises visibility of the fact that software quality is important”. An interesting but overlooked part of the data on this slide in my opinion is that about 20% of enterprises (even the “leaders”) said that their “focus on agile and DevOps has not had any impact on software quality”. I assume this data didn’t fit the narrative for this highly DevOps-focused webinar.

Attention then turned to tooling, firstly looking at development tools:

I find it interesting that all of these different types of development tooling are considered “DevOps tools” and it’s surprising that only around half of the “laggards” even claim to use source code management tools (it’s not clear why “mainstreamers” were left off this slide) and only just over half of the “leaders” are using continuous integration tools. These statistics seem contrary to the idea that even the leaders are really mature in their use of tooling around DevOps. (It’s also worth noting that there is also considerable wiggle room in the wording of this question, “regularly used or will be used”.) Deployment, rather than development, tooling was also analyzed but I didn’t spot anything interesting there, apart from the very fine-grained breakdown of tooling types (resulting in an incredible 19 different categories).

The presenter then examined why software quality was improving:

Notice that the slide is titled “Why has your software quality been improving since 2019?” while the actual survey question was “Why has your approach to software quality improved since the beginning of 2019?” Improvements in approach may or may not result in quality improvements. Some of the choices for response to this question don’t really answer the question, but clearly the idea was to suggest that adding more DevOps process and tooling leads to quality improvements while the data suggests otherwise (more around business drivers).

Moving from the “why” to the “how” came next (again with the same subtle difference between the slide title and the survey question):

There are again business/customer drivers behind most of these responses, but increased automation and use of tooling also show up highly. A standout is the “leaders” highlighting that “our multifunctional teams have learned how to work more effectively together” was a way to improve quality.

Some realizations/revelations about quality followed:

There were at least signs here of enterprises accepting that improving quality takes significant effort, not just from additional testing and tooling, but also from management and the business. The presenter focused on the idea of “shifting left” and there was a question on this during the Q&A too, asking “how important is shift left?” to which the presenter said it was “very important to leaders, it’s a best practice and it makes intuitive sense”. But he also noted that there was an additional finding in the deeper data around this that enterprises found it to be a “challenge in piling more responsibility on developers, made it harder for developers to get their job done, it alienates them and gets them bogged down with activities that are not coding” and that enterprises were sensitive to these concerns. From that response, it doesn’t sound to me like the “leaders” have really grasped the concept of “shift left” as I understand it and are still not viewing some types of testing as being part of developers’ responsibilities. The final entry on this slide also stood out to me (but was not highlighted by the presenter), with 17% of the “leaders” saying that “software quality is a problem if it is too high”, interesting!

Presentations like this usually end up talking about best practices and this webinar was no different:

The presenter focused on the high rating given by the “leaders” “adoption of quality standards (such as ISO)” but overlooked what I took as one of the few positives from any of the data in the webinar, namely that adopting “a more comprehensive approach to software testing” was a practice generally seen as something worth continuing to do.

The deck wrapped up with a summary of the “Best Practices of Software Quality Leaders”:

These don’t strike me as actually being best practices, rather statements and dubious conclusions drawn from the survey data. Point 4 on this slide – “Embracing agile and improving your DevOps practice will improve your software quality” – was highlighted (of course) but is seriously problematic. Remember the self-assessed “leaders” claimed that their software quality was increasing due to expanding their “DevOps processes and toolchain”, but correlation does not imply causation as this point on this final slide implies. This apparent causality was reinforced by the presenter’s answer to a question during the Q&A also, when asked “what is one thing we can do to improve quality?”. He said his preference is to understand the impact that software quality has on the business, but his pragmatic answer is to “take stock of your DevOps practice and look for ways to improve it, since maturing your DevOps practice improves quality.”

There were so many issues for me with the methodology behind the data presented in this webinar. The self-assessment of software quality produced by these enterprises makes the foundation for all of the conclusions drawn from the survey data very shaky in my opinion. The same enterprises who probably over-rated themselves on quality are also likely to have over-rated themselves in other areas (which appears to be the case throughout). There is also evidence of mistakenly taking correlation to imply causation, e.g. suggesting that adding more DevOps process and tooling improves quality. (Even claiming correlation is dubious given the self-assessment problem underneath all the data.)

There’s really not much to take away from the results of this survey for me in helping to understand what differences in approach, process, practice, tooling, etc. might lead to higher quality outcomes. I’m not at all surprised or disappointed in feeling this way, as my expectations of such fluffy marketing-led surveys are very low (based on experiencing of critiquing a number of them over the last few years). What does disappoint me is not the “state of software quality” supposedly evidenced by such surveys, but rather the state of the quality of dialogue and critical thinking around testing and quality in our industry.

The webinar can be viewed from https://content.microfocus.com/optimize-devops-tb/2021-software-quality (note that registration is required).

A year has gone…

2 Replies

Almost unbelievably, it’s now been a year since I left my long stint at Quest Software. It’s been a very different year for me than any of the previous 25-or-so spent in full-time employment in the IT industry. The continuing impact of COVID-19 on day-to-day life in my part of the world has also made for an unusual 12 months in many ways.

While I haven’t missed working at Quest as much as I expected, I’ve missed the people I had the chance to work with for so long in Melbourne and I’ve also missed my opportunities to spend time with the teams in China that I’d built up such a strong relationship with over the last few years (and who, sadly, have all since departed Quest as well as their operations there were closed down this year).

I’ve deliberately stayed fairly engaged with the testing community during this time, including giving a talk at at meetup, publishing my first testing book, launching my own testing consultancy business, and blogging regularly (including a ten-part blog series answering the most common search engine questions around testing).

Starting to work with my first clients in a consulting capacity is an interesting experience with a lot of learning opportunities. I plan to blog on some of my lessons learned from these early engagements later in the year.

Another fun and testing-related project kicked off in May, working with my good friends from the industry, Paul Seaman and Toby Thompson, to start The 3 Amigos of Testing podcast. We’ve always caught up regularly to chat about testing and life in general over a cold one or two, and this new podcast has given us plenty of opportunities to talk testing again, albeit virtually. A new episode of this podcast should drop very soon after this blog post.

On more personal notes, I’ve certainly been finding more time for myself since ending full-time employment. There are some non-negotiables, such as daily one-hour (or more) walks and meditation practice, and I’ve also been prioritizing bike riding and yoga practice. I’ve been reading a lot too – more than a book a week – on a wide variety of different topics. These valuable times away from technology are foundational in helping me to live with much more ease than in the past.

I’ve continued to do volunteer work with The Vegan Society (UK). I started off performing proofreading tasks and have also now joined their web volunteers’ team where I’ve been leading research projects on how to reduce the carbon footprint of the Society’s website and also to improve its accessibility. These web research projects have given me the welcome opportunity to learn about areas that I was not very familiar with before, the “green website” work being particularly interesting and it has inspired me to pursue other opportunities in this area (watch this space!). A massive proofreading task led to the recent publication of the awesome Planting Value in the Food System reports, with some deep research and great ideas for transitioning UK farming away from animal-based agriculture.

Looking to the rest of 2021, the only firm commitment I have in the testing space – outside of consulting work – is an in-person conference talk at Testing Talks 2021 in Melbourne. I’ll be continuing with my considerable volunteering commitment with the Vegan Society and I have a big Status Quo project in the works too! With little to no prospect of long-distance travel in Australia or overseas in this timeframe, we will enjoy short breaks locally between lockdowns and also press on with various renovation projects on our little beach house.

(Given the title of this blog, I can’t waste this opportunity to include a link to one of my favourite Status Quo songs, “A Year” – this powerful ballad morphs into a heavier piece towards the end, providing some light amongst the heaviness of its parent album, “Piledriver”. Enjoy!)