Test Pappy

I want to take the time to look back on 20 years of success and failure. This blog will be long, unstructured, ugly and boring. I write this post mostly for me. My advice is: Don’t read it!

T-Systems – the early years

I started in IT in March 2000, with joining debis Systemhaus, which was soon after bought by Telekom and merged into T-Systems. I was mostly doing Reporting, Data Warehouse and eBilling projects. End of June 2003, while my first Java project came to an end, I was interviewed for the role of an operational tester for one of our two big Test Factory projects. July 1st 2003 was officially my first day as a tester. On that day I could have already realized that software and testing is a lot about people and that testers are living with a special pressure. The ~50-60 testers of the project that we were back then had a facilitated de-escalation workshop that day; as I learned, the mood and temper in the team was horribly tense and aggressive. So I spent my first day witnessing how the team tried to deal with anger and disappointment. This event should mark the start into what has today become a 20 year long career as a tester.

The project I joined was the back then famous project “Toll Collect”, the introduction of a distance based charge for trucks using German’s Autobahn network. The project was already running for about 1.5 years and should go live September 1st 2003. The pressure was immense. Late in July or very early in August 2003 it was decided to cancel the project start due to a list of problems. It was already after a few weeks when I learned how important testing can be for the success of a project. When I joined we had big trouble keeping the system running under load for 13h, which was the initially planned auditor test scenario for the go-live. Remember that: 13h.

The start of the project was delayed until Jan 1st 2005. I had some time to get used to “my” subsystem, and one ability became apparent that I realized only about a decade later. Systems Thinking. Understanding things, patterns, behaviour, finding names, having a more structured approach only years later, is a common pattern still true to this day in my career and you will read similar things some more often. When combined with common sense, you know that thing that is not that common anymore, it’s an even more powerful tool.

Good Systems Thinking helped me to quickly understand

Good Systems Thinking helped me to quickly understand the whole Toll Collect system, consisting of 15-20 more or less complex subsystems. The system got more stable, problems were fixed, solutions were found. While we were preparing the project start during late summer and autumn of 2004, I got, in addition to my role as sub-team lead (of 3 testers and 2 sub-systems), the role of load test coordinator, supporting the designated test manager. That autumn we had three, each 2-weeks long (much more realistic than the original 13h load tests), and auditor accompanied load tests. Especially the third one I will always remember and tell people about it if they want to hear about it or not.
While the first two tests were to show that the system was able to work under expected load for the standard and fallback scenarios, the third was to show how resilient the system was. An auditor was sitting with a super admin in a room, and from time to time told them to corrupt some system, database, network connection, you name it. The goal of the test was to see how quickly would a ticket pop up, is the failure visible in the monitoring, would the system failover properly, continue running on the other node, would operations people know what to do, how quickly would backlogs be processed, and so on. You might have already guessed it. What in 2011 became popular by Netflix under the name of Chaos Engineering, was performed here 7 years earlier as a standard test scenario in project Toll Collect. And yes, keeping the system under load for 14 x 23h (due to simulator restrictions), was not an issue at all. 13-14 months earlier we were not able to keep it alive for 13h.

In the next 3 years until end of 2007 I became deputy team lead of two test teams, learned all about several other subsystems, and even learned my basics of the SAP system. While project “Toll Collect” was the laughter of the nation end of 2003 and all year 2004, after the successful and pain-free start in 2005, the project was running quietly and smooth. Two major releases a year got properly tested, verified and rolled out without issues in production.
In the meantime I made my ISEB Foundation certificate and my iSTQB Test Manager certificate, learning not much more about what I kind of did for several years on a daily basis. The whole project was heavily iSTQB driven, so I had no choice than to get indoctrinated from the very beginning of my career. It was so strict, that it was even not allowed to raise a bug against a subsystem, where no according test case was written in advance of the release start. Exploratory Testing was frowned upon, while it was actually nearly all that I did all year. I hated writing test cases upfront. How would I know several months in advance, what I’d discover while observing the system under load myself.

In 2006 I managed to “test” a new sub-system out of the system again within two releases. I didn’t like the idea of the new sub-system and found it was completely wrong and useless. In addition the initial dev team of “my” subsystem handed over “my” subsystem to the team that should implement also this new system. They had no idea about the intricate database design, didn’t listen to what I was trying to tell them, and produced this new piece of waste. Against the first release I raised 85 bugs and prevented it from going live with the planned release. That was more than a third of all bugs that 70+ testers raised against all 15-20 subsystems in the same time. Two releases later the system was history again. I was not able to prevent it from wasting a lot of time and money or even reaching production for the time of one release. But I’m still proud to say, that it was never used in production. It was the first time that I actively influenced the architecture of a system.

As a tester it’s often not your turn to bathe in the light of success.

Another anecdote about the lack of understanding of the new dev team followed in 2007. I was not responsible for that system anymore, when a production bug occurred. One record caused some issues. When the Test Factory was informed, the project lead came running to me, gave me the production database password and told me to invest an hour or two into the analysis. Two hours later I was able to tell them what happened, why it happened, and what to do to correct the record accordingly. It took two weeks for the dev team to analyze the record, come to a nearly correct conclusion about what happened, and to come to a wrong conclusion what to do about it. But they sold it as their victory, while including my project lead and me, in addition only two of the responsible people on internal customer knew that I solved the problem already two hours after it occurred. As a tester it’s often not your turn to bathe in the light of success.

T-System – Advancing into Test Management

In 2008 I advanced in my career and got nominated as the leading test manager for the international road charging project of T-Systems. Together with a bunch of my former colleagues of the German system, we tried to help develop a more light-weight system and sell it to some of the European neighbors that tried to introduce a similar system for their road charging initiatives as Germany. It was a good time. We tried a few light-weight approaches with an external dev team working in 2-week sprints. The rusty, slow oil tanker of our iSTQB heavy approach was not even closely able to keep pace with this. We tried to adjust, but the indoctrination of years of long-cycle waterfall approaches slowed us down. Nonetheless, it was a good year.

In 2009 the German project had some personnel problems, and I was brought back to the Toll Collect project as Release Test Manager. The job was preparing a release for 3-4 months, coordinating and reporting the release tests for about 4-5 months, accompany the rollout for a month, cleaning up the project for another 1-2 months. Then keep it a bit slower, support the next release a bit, and slowly start again with the whole shebang for the release after that. We were usually a team of four, always pairing up, one lead, one support and back-up.

In the next four years, I was the responsible Release Test Manager for the first and long time only release that did not make it to production in time. So I was also responsible from testing side for the only double-release rollout so far. That release was stressful, full of red reporting slides, troubleshooting phone calls and what not. And during the unplanned extension of the release my daughter was born, which didn’t make it any easier.
Furthermore I trained a handful new release test managers, coordinated large portions of the first hardware exchange, I was the responsible release test manager for introducing road charging on the first 1200km of federal highways and I was the oracle of ancient process wisdom.
Whenever someone wanted to know why we were doing something the way we did, I could tell them. I knew all the processes, and more important, I knew all short-cuts to speed things up.
In the four years as test manager I hated our time-consuming test approach to the bones. Now I was in a role where I could shift things around, react to short-term customer requests, bypass teams when I didn’t see the value. Teams were not often happy about that. I did a lot of things my way. If that was good or not, I still don’t know. I was respected with both internal and external customers and colleagues, and people came to me when they wanted to get shit done.
In 2012, my last year in the project, someone from the external customer called me, because they wanted to know something very specific about what used to be “my” subsystem in 2003-2006. I was renown in the whole project to be the person with the second-most knowledge about this system, behind the person on Toll Collect side who re-designed the sub-system from 2007-2012, who happened to be on vacation that week.

Nine and a half successful years as a tester came to an end. It was time for me to move on. In September of 2012 one of our team leads shared a newsletter from EuroStar conferences about a series of free webinars on testing topics.
In over nine years, I got the ISEB Foundation 2004, the iSTQB Test Manager 2005, ITIL v2 Foundation in 2005, a seminar on risk-based test management in 2006, and participation in the course of one of the other advanced level iSTQB courses in 2008, as they were given in-house, where I never took the exam. And in 2012 I got a really cool seminar on project management with a lot of basics. Maybe a bit late already, but you know… better late than never. Anyway, I didn’t know about the testing community, testing conferences, webinars, books (except the one I got for the advanced level, co-authored by a colleague), blog posts, Twitter and what not.

I had to realize after this first webinar ever, that I didn’t do one thing in 9 years. I didn’t ask many questions, I didn’t try to understand basics, I tried to fake it until I make it. I saw asking (stupid) questions as weakness between all these seasoned testers, developers, project managers and leads. Oh, what a waste of time. I was for a long time one of the youngest people in the team, and one of very few that did not have a university degree (at one point we even had 13 out of 70 people with a Ph.D.). 9 years I accepted the processes defined by people who were no longer in the project, I accepted terms, and tried to learn how to use them in the right context, rather than understand what they actually meant.
I knew a lot of things, but it took me forever to learn them, I rarely questioned things, because someone more intelligent than me has introduced them for a good reason, I thought.
In that week started my awakening as a tester. 9 years I advanced in my career, in rank, but mostly in a timely manner. I have common sense and was a good systems thinker, which helped me most in all aspects. I was able to continue doing what people did before me, filling out the ever same Excel sheets, writing and adapting the ever same offers, creating the ever same PowerPoint slides. Thanks to my systems thinking skills I could easily handle complex situations and adapt ancient processes quickly and accordingly. At least for me.
In 9 years I didn’t do anything new or innovative. There was no reason for me. I tried to do what I thought was best for the project, all very context-driven (a term I also learned only a bit later about).

Urban Science – becoming a QA Lead – or not

In 2013 I finally left T-Systems and started a job in a mid-sized US company called Urban Science, active in the automobile industry’s dealership network area, that bought a small company in Munich a few years earlier, which’s product fit in their portfolio. And since coordinating and managing the testers from the US was too cumbersome, they were looking for someone to be the “QA Lead” for the Munich office.
A weird chapter in my career began, that I both don’t want to miss and also would have liked to skip. I should have realized that something’s not right, when I asked to meet the team after the interview, and they came up with excuses. Of course, I didn’t ask any questions back, because I never really learned that. And I didn’t spend much time thinking further about it. Two months after signing the contract and still a few months ahead of me starting, when the boss boss from the US was in Munich, they asked me to meet him and the team. And then it became a bit more apparent why they needed me. They didn’t want to deal with one of the testers in the Munich office themselves. Well, too late.

The biggest defeat in my career to date.

One of my saddest moment in my career was the time, when I was not able to come to terms with that colleague, find a way to professionally deal with them, help them to become a better tester and more accepted in the team. But besides the lack of experience in leadership, I also lack a psychological degree, and I was not able to handle the situation accordingly. So my boss in the US and someone from HR took the short route and made them leave in the middle of my attempt to improve the situation, which is to this day still one of my biggest defeats in my career.

On a positive note, during my time there, I read loads of blog posts, magazines, a few books, and I started to actively participate in the Testing Community on Twitter. I caught up with everything that I missed over the past 10 years.
I even started to be innovative. In parallel to some people in the South-East Pacific region I started to develop something that went later viral as Visual Test Model. Using a mind-map to document the system under test and guide your exploratory testing.
I learned that Exploratory Testing was a thing, that it could be structured and highly useful. And that it actually was what I was doing for a long time. I even created Work Item Types in Microsoft Team Foundation Server to document it and use it more effectively. I applied Session Based Test Management very successful in the introduction of a new product.
In 2015 I started designing my first test framework to finally start with automating something that was never designed to be automatically testable. I came up with innovative ideas myself, I read a lot, I discussed with people in the interwebs, I adapted a lot of industry standards.
My disgruntlement with the old school approach that I was using successfully for 9 years, even got me to participate actively in some activities that I later regretted deeply. I was an active iSTQB basher and ISO29119 proponent. While I realized at some point that this iSTQB bashing was not my battle and that I should never have participated in it in the first place, I’m still against ISO29119, though could have been a bit more constructive in the process.

While trying to learn everything I could, trying out new things in the test approach, surviving long weeks, I also had to fight the long entrenched opinion in the company, that bugs found in production are a fault of the QAs, not testing good enough.
I had to fight weird processes when it came to promotion for one of my team members, thanks to the US colleagues again, which later ended in them leaving disappointed. Which was a shame, because they were really good.

And it was impossible for me to find any good testers. I had about 40-50 interviews for a “manual” tester and a test automation engineer position. I’m not good at doing interviews and the company did not attract much testing talent, so it seemed. I was also only given budget for an entry level position.
In the end I found one candidate for the manual job, whom I was never able to push into the right direction. My other team member, who was on my team for 3 years, was not able to understand IT basics, while having a degree in CS, and the products we tested were often too complicated for them. I tried “Leading by Example”, which was a fault in hindsight. During my time there I had to work with 10 different testers, except two, one not even assigned to the same products as me, all were lacking basics. And I was, and probably still am unable to explain basics from that low level. I think I’m doing okay when the other person brings some foundation. I was at least able to help two people advance in their skills. The others, no chance. 😦

Leading by example only works, when the group you lead is in sight of you. When they are so far behind that they can’t see where you are, they have no chance to follow. I separate the blame for that between them and me. Me because I was not able to have them catch up enough, them for not being able to tell me, that they need more adjusted guidance.

My biggest success -personell wise – in 3 years and 8 months was having an intern for a week. She was the daughter of a colleague and a very smart lady. After half a day of introduction to what software testing is and how our product basically works, she was doing an amazing job in the next 4 days. She was easily out-testing the other two colleagues working permanently with me on the products.
Anyway, I realized in this job, that I was not lead material of any kind. The early defeat of having to let go a colleague, not being able to train juniors, not being able to request budget for more (senior) people, not being able to get one colleague promoted earlier than they lastly was, and other misconducts. Also, I changed line of reporting three times in three years. Nobody wanted me and my team. I can’t blame them for that.

Welcome to micro-management hell! The consequence of trying to do everything yourself.

One of my low-lights was being micro-managed by my boss. I constantly had around 100 tasks in my backlog, I was switching context one to two dozen times a day. I was never able to even finish something by more than 70%. I was lucky when I reached that much. Most tasks were cancelled before I could start them. My team was not experienced enough to help me with anything, it was disappointing. Every Monday morning in my 1:1 my boss would select the tasks for the week. I ignored them, as I had long adopted the approach: who yells the loudest gets served next. Next Monday I had then to justify, why I didn’t make progress on any of the selected tasks. Fun times.

I really liked the products I was working on though. I quickly understood the products, thanks to systems thinking again, understood the complexity of the dozens of configuration variables that often interfered with each other. I understood the majority of the 14 setups for 12 different clients on 10 different versions of the same product. The complexity of legacy data in the system was tricky but doable, at least for me. My team was always struggling.
It took me way too long though to understand the basic nature of the product, not how the product worked, and how to successfully re-structure the test set in a more efficient way. But as the design of the products was very customer-driven, there was not much of a basic design and architecture understanding anywhere in the company. The developers were organized in module silos within team silos, all working on a monolith, throwing that over the fence to the QA silo to see if it actually works.
One of my favorite anecdotes for that was one bug fix, where the developer came to me, telling me, they think they fixed it, but they don’t know how to get to this screen, so they can’t test it themselves.
Also the ever ongoing fight to finally introduce unit tests, so that we don’t always suffer under basic blocking issues all the time. I don’t think that any unit test was even written before the day I left there. We were still on the level: “Try to see that the application starts, before handing over to QA.” Not that it was consequently followed. Playing ping-pong was our sad daily business.

I really enjoyed impersonating a Business Architect from time to time to help design solutions.

Working closely with the Business Architects, impersonating one from time to time, helped me doing something that became a pattern. I influenced and designed solutions, I partially took the decisions for architecture and design. I was actively involved in the product development. I really enjoyed this.
While I was busy doing that, having calls and meetings with angry clients, I missed to take care of myself as well, and ended up in the hospital one nice December evening. From that year on, I evaluated every year (reviews and employee satisfaction surveys) compared to that event, which probably gave away who was giving that feedback, but I assume they knew anyway, as I was the only one who wrote looong texts.

Failure and Success were always very close in that company. I had a lot of colleagues I liked, a few I couldn’t stand, and I was working in an international company, speaking English all year long, which I really enjoyed. I tried hard to succeed. And as you might have already realized, I’m often slow in recognizing things. So I tried for 3.5 years to still make this assignment a success and move the company forward, before finally giving up. Giving up is probably not the right term. I learned about this approach from Sally Goble in a talk she gave at Pipeline Conf. She talked about removing the safety net from the developers by dissolving the QA department; which was the perfect solution for my context. I presented this approach to my boss and my boss boss. While my boss was afraid, probably because he nurtured the QA blaming culture for many years, the boss boss liked the approach. My suggestion to move me to the BAs, where I anyway spent most my time and the other two leftover QAs into the dev silos as embedded testers, was not accepted by my boss. As I saw this as the only way to improve the understanding for quality and responsibility in the company, I did the next best thing. I quit. That day is another weird story, but I’ll skip that.
Later I found out, that my approach worked, and the devs now own more of the quality and responsibility of their modules. One of my higher-ranked US colleagues had to take over for the transition, but in the end gave up as well, for the better I’d say. This company seems to do rather well without testers.

During that time another big thing in my career happened. I went to my first testing conferences, met people I knew from Twitter in real live, gave my first conference talk and my first workshop. I started blogging. I became invested in the Testing Community.
I had this theory that in 2016 I could have gone to any tester meetup anywhere in the world and know at least one person, except in Germany.

QualityMinds – Consulting, but different

In September 2016 a new chapter of my career began. I joined QualityMinds, a young and active consulting company, when I joined, mostly active in the software testing area. QualityMinds gave me the opportunity to get into consulting without the need to travel all week. And they kept their promise. Except for two workshops, my only travels in nearly five years were to testing conferences. I like my family and my home. Being on the road all week for some consulting gig, working long hours, sleeping in hotels, with the hope to have a short Friday, that is not for me.

During my time at QualityMinds a few important things happened for my career.

• At my first client, I experienced the first real Agile project and probably the only one to this point. I don’t mean having stand-ups and two week sprints. I mean continuously experimenting and improving and meaning it.
  I was the first experienced tester on that project, and never again have I seen a team that lived the “everybody can test” mentality more than they.
  It was also the project where I implemented together with an intern and a newbie a UI testing framework, as I designed it in my previous job.
• While I felt most of the time basically useless at my second assignment, I learned a few things. Continuous Compliance can be reality with a few basic rules to follow. That a proper BDD framework is a lot of discipline and work, but can be fun and can save also a lot of time. And I learned that also as a senior tester you can spend hours each day doing simple and fundamental tasks like analyzing nightly tests, stabilizing tests, balancing test suites, keeping dev containers up-to-date, maintaining reporting, which might not be all that important to you, but it all was important for the larger team. When I left this assignment after 18 months I soon after learned that I left a hole in the project, as I seem to have fulfilled a lot of tasks that were important for the basic stability of the project that was filled by more than a handful of people to take over what I haven’t seen as that important.
• On my third assignment I felt at home from week 1 on. I mean, the interview process for that project was me in the room with two senior people of the project. And the interview began with “So, we are here to sell this project to you, so that you decide to join us.” I have to add, that this project was in a sister company of project two, and people know each other for ages there. So someone must have said a good word about me.
  Onboarding was a bliss. I quickly understood all the business processes, the tech stack was familiar, and the automated test set was in a good condition. I was quickly able to participate in discussions, and from time to time even influence product decisions (which is rare for an external tester). Mostly I learned to maintain, stabilize and develop the automation framework and to bring it to the next level. I was able to teach a few testers around me a few tricks. And I learned the code base in a way, that I was able to fix bugs, implement small tasks and help in any way I could to Accelerate the Achievement of Shippable Quality.
  After 18 months, I had to have a mandatory break, and when returning to the project, I was put in a small, agile team for implementing demo capabilities with production stability. My tasks now included by default also development tickets, which was a great honor for me.
  Honestly, I always thought that when I leave QualityMinds at some point in time, I would simply join this company. That was home. And the way they treated their employees including the externals was amazing. I spent in total 3 and a half years in that building.
• In my last year I also had an internal assignment, joining my own colleagues for a change, at least for some time, and helping develop a project from scratch. My architecture knowledge helped me to drive decisions, establish a few basics, and overall actively participate in the project on all layers.
  I implemented a new kind of test framework from scratch, based on my experiences with the last three frameworks I worked with.
  And in general, working in a consulting company with the people who receive their paycheck from the same company as you, has a special feeling, especially after being on solo assignments for 4+ years. I enjoyed this project to the last day.

In those nearly 5 years not all was happy sunshine. Due to probably mostly self-induced pressure, I soon after joining stopped learning. Like completely. I learned on the job, but not a single bit outside. Which is especially ridiculous as QualityMinds specialized in Learning more and more over the years.
I spent at least 40h a week at the client, we all had a few hours every week for team tasks, as we were very self-organized, and every now and then there was some time spend for the wider company. I was not able to find a balance, where my learning found a place. And then there was something else.

On my first day working for QualityMinds, when taking the train to the new work, I received a call from Kris Corbus; Rosie agreed that we could organize the first TestBash in Germany. This was a big opportunity, and after speaking with my CEO at QualityMinds, I agreed. Because Michael happily promised me all the support I need. And I got that, from the whole company.
13 months later, Kris and I, with a lot of help from Vera, Marcel, and Daniel, welcomed around 200 testers to the first edition of TestBash Germany. It was a success, at least from a TestBash perspective.

Personally I had to realize in hindsight, that on Saturday, the Open Space day, I had the beginning of what was a burnout. Not realizing this, not seeking professional help, trying to power through, not having a single sick day, continue to work 100% on-site at the client, ended up in around 18-24 months of mild to medium burnout, with depression, fatigue, weariness, and no energy for basically anything but the most basic functions. An approach I can highly NOT recommend. Sometimes you have to accept the fact, that something’s not okay, and that you need help.

To end this chapter on a more positive note, I also landed two talking slots again at conferences. I repeated my talk from 2016 in a worse version, where I only realized during the Q&A what potential the talk could have had. And in 2019 I gave my talk on “Testing Machine Learning Algorithms 101”, which I’m still a bit proud of.
I also made the personal decision to stop talking at conferences. I don’t have many interesting things to talk about, and others are usually better at that, and most of all, because the conference stages are already full of middle-aged, white Western European cis-men, no need to stress that contingent, when me not applying raises the chance for someone else who should really be heard and who brings diversity to the stage.

You might remember from a few paragraphs earlier, that I was sure, that when the day comes where I leave QualityMinds, that it will be for this one company that I worked for 2 years in the end. Well, some things quickly change from one day to the other. I got a ping from someone who was looking for Quality Engineers (whatever that is) for this medium-sized med-tech company in Berlin, working 100% remotely. Dan Ashby told me about the product, some challenges they have, and most of all, who else was working already at Ada. And I was sold!

Ada Health – and what does a Quality Engineer do?

Exactly 10% of my career, or better, two years ago I joined Ada as a Quality Engineer. What that role means, I still haven’t found out. And I couldn’t care less. The good thing is, that all QEs at Ada interpret their role individually. Everybody brings to the table what they are good at. And if you have a rough idea of who else works at Ada in the QE department, you know they are Good, with a capital G.
The nice people at Ada let me help on many levels. I am embedded in a wonderful team, with amazing colleagues. From within the team I was able to implement an end-to-end test framework for integration tests, that should serve more teams than just mine, because there was nothing appropriate in place when I joined.

I am able to support with process optimization all around releases and compliance. I inherited an awful script collection to create release documentation. In the meantime we rolled that tool out to nearly all teams at Ada and save those teams a whole lot of time during every release.

I’m doing about 2/3 of what I do in my day not because somebody asked me to, but because I think that I can help the company when I engage in these aspects. It took 18 years of good and bad experiences to reach this point in my career, where I think that these are all tasks that I can help with delivering a better product and faster and easier. Accelerate the Achievement of Shippable Quality.

Due to a necessary cost cut initiative over half a year ago, we lost a great bunch of people and had to re-organize. We slowly recover from that cut and find back on track. We got merged with another team that has lost their QE in the cut. So I’m now embedded in a team with four different modules, more than any other team here. And I’m involved in a couple of other projects on the side. We just got a new Engineering Manager through the team migration, which I’m looking forward to work with, as we lost one of the best EMs I ever had to the reorg. Our product manager who switched to us some time ago, is now finally onboarded and able to drive topics also for “our old” applications.
In the rough time not only I tried my best, the whole team stepped up in the past 8 months to fill the voids and keep the team running. I can’t be more proud to be a part of this team. We were an absolute dream team before the cut, and it got proven after the cut, just how awesome this team is working together. I hope we can continue this journey in the new structures, as more changes are currently happening within our team.

And once I realized that I can’t change the company alone overnight, I embrace every baby step in the right direction. There are personal set-backs, a lot of stand still from my perspective, but from time to time baby steps. There is a lot of work still to do. And people stay willing to change. But it has to be slow and careful. So slow and careful it is. And as long as this company is willing to accept me among their peers, I will stay and from deep down in the hierarchy try to push the whole place slowly and steadily in the right direction. The right direction from my perspective, of course.

And this is my boring story up until today.

Retrospection – the bitter insights of 20 years

20 years as a tester and 3 additional years as developer (apprentice) made me to what I am today as a professional. I have found my specialization in generalization (a few years ago I called it “the broken comb“). I know a little about a lot. I can help and support in many areas. I have the foundation to quickly learn new topics, I have the ability to quickly dive deeper into topics I know a bit about already. I spoke about this in my first talk at TestBash Brighton 2016. I’m half good in recognizing Mount Dunning-Kruger, and I still suck when it comes to Impostor Syndrome.
This is also very frustrating. Every skill I have, even those that I am probably good at, the chances are high that somebody around me is better than me. For years I have looked for a topic that I can specialize in, to maybe become a renowned expert for something. Besides being grumpy nearly all the time, I didn’t find anything. Every time I try to dive deeper into one topic, I get distracted (thank you ADHD) by something different that’s also interesting. There are so many interesting topics out there, that I can’t focus on just one or two. And as I stopped learning anyway a few years ago, why bother. It took me nearly 20 years to accept this fact, that there is always someone better than me. I have just so many different skills that it is impossible to be excellent at one of them. So I accept this fact now and be thankful for the experts around me and the chances where I can use my skills.

As generalist or generalizing specialist there is a huge trap for your confidence.

The broader your knowledge is the easier it is to find someone who is better than you.
There are probably other generalists who know more than you.
And that’s okay! Your variety is your strength! pic.twitter.com/JJQYhgyxX2

— Patrick Prill @testpappy.bsky.social (@TestPappy) May 12, 2023

As I told you already earlier, I’m often rather slow in recognizing and understanding things. For 20 years I’m keeping myself too busy, to take a step back from time to time and reflect. Slow down to speed up is so true, and I tend to not follow that advice. I also tend to not ask for advice early, I want to get shit done by myself. I don’t sharpen my axe, because there are so many trees to be felled.
Last year I was asked by a wise man, why I am putting in so many hours week after week. And it was one of these rare moments where an explicit question made me step back and ponder on that fact. The answer I came up with is a complex, yet frustrating one:
I’m doing this to compensate the feeling that everybody around me is way smarter than I am. As a university drop-out, though still one of the, I think, three or five best vocational training students of my year in the greater Munich area, I was the following years mostly surrounded by some very intelligent people. Since finding the testing community this number multiplied big time. In my current company it feels even more like nearly everyone they hired is an absolute ace in their area. This sense of inferiority I compensate with working extra hard and extra hours. And I know that some people say it’s not possible, but I can work effectively and concentrated for 10-11h most days.
Also I tend to attract a lot of work overall. My task list is always well filled by a variety of topics on many levels. I see so many things that need optimization, and while I just said, that I’m surrounded by smart people by the dozens, I rather do the job myself, so I know that the results fit my understanding of the system and how I like them to be. I can’t delegate, so I end up with even more tasks on my desk. I just made a list of major and medium long-term topics on my desk: 14!

Speaking of complexity. As I wrote very early in this blog post, that in hindsight I found out that one of my more natural strengths ever since is systems thinking to a certain degree that seems to be better than average. I’m not great at it, but probably better than others. During my intense phase of learning from 2012-2017 I also came across the Cynefin Sensemaking Framework. I have probably not understood most of it, but the basic idea is highly fascinating. And I take from it what is useful for me. Cynefin is separating the parts of systems into domains of Obvious, Complicated, Complex and Chaos. What I learned over the years is that I felt at home in understanding complex systems. Instead of understanding first in what kind of situation I’m in, as Cynefin suggests it, I start treating everything as complex. Maybe to avoid surprises, to avoid the cliff of complacency, or because 20 years of experience told me, that systems are rarely obvious or simple, and that even complicated is an exception, especially when people are involved. Maybe that’s also the reason why giving a simple explanation for something is especially hard for me. I always try to incorporate the dependencies and complexities, to avoid that people think that it’s simple. Because often it is not.

When thinking about what I do and what I can do best, I would say laying and improving foundations. Foundations are an extremely complex topic to get right and to get harmonized across the board.
Most topics I have on my plate and that I am somehow attracted to are:

• detecting issues in existing processes and find missing processes
• help setting things up, that should be there
• boot strapping things that others can adopt or build upon
• doing simple tasks that have to be done
• taking care of basic tasks, the every day things that are not appealing to many
• automating the basics, so less people have to worry about it and can focus their energy on other things

A lot of this is I prefer doing, because I have seen different times. I started in testing in a full-blown ITIL project. You might think of ITIL what you want, but basically any IT project that wants to be successful and maintainable will follow processes that are all somehow described in the ITIL world for ages. The way they are dealt with will differ greatly between hiring a gang of service managers and trying to manage it on the side. But not having these basic processes is not an option. Even the smallest of projects needs to deal with all of them.
In 2019/2020 during my time in QualityMinds, I received my most useful training of the past 23 years. The iSAQB Architecture Foundation. This course was astonishing for me. While I was the only non-developer in the round, I was also the only one who didn’t actually learn anything new. And yes, it still was the most useful training ever. Most topics were confirmations of what I have already seen and experienced in other projects. Partially topics I have long forgotten (another phenomenon in my brain). The best thing about this course was the condensed amount of different topics that form the scaffolding and foundations of a good IT project. If I would be able to stay up-to-date and learn a bit more about several technical topics, IT architect would probably a good fit for me.
And as many people around me jump onto the shiny new shit trains, the basics are often forgotten. Without good foundations and good scaffolding, the shiny new shit won’t stay up for long. And somebody has to do that job. So why not me. And when you see me on social media rooting for people talking about the basics, and being skeptic about shiny new shit, maybe you understand a bit better now why that is.

Now another weird insight of the past 20 years. I absolutely suck when it comes to explaining others the basics of software testing. I never was good at that. And I never learned or wanted to become better either. Software testing is something I just do based on systems thinking, common sense and my accumulated knowledge, analyzing potential risks. I’m not doing this in a structured manner, if I don’t have to. I don’t think that I’m a bad tester, actually I’m rather successful with the way I work for most parts of the last 20 years, but I am not a good software tester or teacher of software testing per se. I can’t teach others how I test, because it’s highly personal and complex.
I’m probably a slightly above average test automator. And I’m good in testing processes and projects though. I help teams optimize their flow. I help to test and improve new and old processes. And I rather like to test ideas and make decisions early in the process, improve architecture and design of the application. I’m better at many other things that I see as related to being a software tester or Quality Engineer, while they will probably not popup on a role description and are often not expected from people around me. (some very disappointing stories could be told about this)

Accelerate the Achievement of Shippable Quality and Removing unnecessary friction

Accelerate the Achievement of Shippable Quality. I have mentioned that a few times in this blog post. This motto was introduced by Alan Page and Brent Jensen in the AB Testing podcast somewhere in the 50s episodes, if I remember correctly. Those led to the Modern Testing principles introduced around episode 70. This podcast had probably the biggest influence on me over the past 7 years. Alan and Brent find words for the things that I do, without realizing that I’m doing them. The podcast is often challenging and expressing things in a rather extreme position. But I like that, as it has always challenged my status quo. The no. 1 guiding motto for me was and stays Accelerate the Achievement of Shippable Quality. This is what I’m trying to do for 20 years to this date. Most of the time without even realizing that this is my motivation.

Once again…

Quality is the absence of unnecessary friction https://t.co/myI9vfilyW

— sc (@StooCrock) June 23, 2023

Guiding motto no. 2 is the statement that “improving quality is removing unnecessary friction”, that my friend Stu Crocker started making about two years ago. While the topic of quality (the term itself) is one that I come back to rather often in the past 8 years, that statement was one of my few Eureka moments. Thank you Stu for our long chats during the pandemic.

The last 20 years I tried to change projects in a way that I could do my actual work better and easier.

With every new project came the same class of problems to tackle. Every time. Mostly basics that are missing in the testing process or simply understanding how testing should be done. The need to stabilize things, making deliveries faster and more reliable. Raising the awareness of what Quality actually means. And often closing gaps in software development fundamentals that are necessary as foundation for all the other topics. Sometimes it feels like Groundhog Day.

What are you proud of about your career? This could be something that happened a while back or just yesterday.

— Ministry of Testing (@ministryoftest) June 24, 2023

Last week Ministry of Testing was asking, what are you proud of about your career? And I have to say that besides some of the achievements you read about above, I’m most proud of not quitting testing. Testing can often be very frustrating. You have to deal with software, systems or processes that don’t work. You have to be the one bringing bad news often. You have to deal with people who are disappointed. You are usually last in line, you are often forgotten, others tell you what to do or how to do your job, you have to fight for your place in the team and in the company. Testing is often seen as stupid work that everyone could do, so hiring people, teaching them for a few days and they can do the work for cheap money. Or a bit more trained people just have to write a good set of test cases, and then you can outsource the testing to even cheaper labour. Testers are scape goats. Testers are not seen as peers of any other group in a company.
While all these points are not true everywhere, I have experienced them all, and I still hear stories from other testers going through one or the other of these issues just recently. Mental health is a big topic in the Testing Community for a very good reason.
I’m proud to be a tester, and while I’m terrible at self marketing, or easily convincing others of better approaches, I like my job, I like the variety it brings, the possibilities. Yes, I hate to go through the same tough and cumbersome onboarding time over and over again. But in the end it’s usually worth it. I’m not a tester like anyone else, most testers I know are not. We add value to product and projects on many levels, everyone a bit different and with their own style, and I’m proud to be one of this crazy crowd.

And there is one more thing I’m proud of. Over the last 20 years I was able to find several good friends in the testing community all across the globe. People I would not have met otherwise. Several wonderful people from all over Great Britain, especially in the South and North of Wales, the East and Middle- to Northern-ish parts of England, Columbus, OH, Utrecht, Riga, Munich, Melbourne, Ghent, Bangalore, Berlin, Charlotte, NC and some other areas of North America, Lisboa, and where not. And I am even allowed to call one of the most outstanding people in this community a good friend, who happens to live in our neighboring town. And there are definitely a few more that I have forgotten to list with their center of living. I’m grateful for all of you, I hope you all know who your are, and how much you are appreciated. Your friendship is not taken for granted.

Enough ranting, enough depressing self evaluation, enough boring stories from the trenches. Time to start into the next 20 years. Whatever this journey might lead me to, I bet it will be to help improve foundations to create better solutions and ship them faster.

And sorry for the long post, if you made it to this point, here is a potato.

TL;DR: Implementing Test Automation (from a test case and framework perspective) are a great place to do exploration on several layers of the SUT. If you are just there to automate a test case, you miss so many chances to improve the system.

This tweet from Maaike Brinkhof inspired me initially to come up with this post:

Well fuck it I'm just gonna say it. I don't follow "a process" during testing, I just use common sense and see where that gets me.

— Maaike Brinkhof (@Maaikees) June 2, 2021

That is addressing a topic that crossed my desk quite some times lately. When preparing a talk for our developer CoP at QualityMinds, a colleague asking me for advice how to structure their test approach, or one of my teams at my last full-time client engagement, where in the planning session for every story the same three tasks were created for QA. 1. Test Design, 2. Test Automation, 3. Manual Execution + ET.

As you might have already read about it, my understanding of “testing” is probably a bit different from others. What I want to describe today is a bit of a story how my actual testing looks like, when I am in a test automation context. Over the past couch*cough years I learned to enjoy the power that test automation provides for my exploratory testing on basically all layers of the application.

Disclaimer: I didn’t work in a context where I have to automate predefined test scenarios in some given tool for quite some time now. For the last 8-9 years, I worked embedded (more or less) in the development teams, mostly also responsible for maintaining/creating/extending the underlying custom test framework.

Test Automation provides for me the opportunity to investigate more than just the functionality, because I have to deal with the application on a rather technical level anyway. This way I can make suggestions to the system architecture, look at implementation, do a code review, clean up code (you probably know those colleagues that always forget to put a “final” on their variables causing lots of useless Warnings in the SonarQube) and understand better how the software works. Blackbox testing is okay, but I like my boxes light grey-ish. This can save me a lot of time, by just looking at if-statements and understanding how different paths through the functions look like.

Unit and integration tests were mostly part of the devs’ work, but when I preferred that level, I also added more tests on that level. But most of the time I implement tests on service and UI level.

I start with an end to end spike for the basic test case flow for the new functionality. This helps me in creating the foundation of my mental model of the SUT. And I also see first potential shortcomings of the test framework. Things like missing endpoints, request builders, elements in page objects or similar UI test designs and so on. First issues with testability might appear here, if according endpoints are missing, access to some aspects is not given, and whatever you can come up with in the current context that would make testing your application easier. So either go to the devs to let them improve their code or do it yourself and let them know what you did. (If you do it yourself, the second part is important! That belongs to communication and learning!)

This is also the moment when I most often look for implementation details, if they are consistent, logical, complete and usable. That goes for endpoint names, DTOs for request and response, REST method, UI components and all that. <rant>Especially these dreadful UI components, when for the same application the third or fourth different table structure is implemented, which makes it impossible to keep a consistent and generalized implementation for your UI test infrastructure, because you need to take care of every freaking special solution individually.</rant>

Once the first spike is implemented we have a great foundation for a standard test case. Now we can come up with the most common scenario and implement it. The happy path, if you want to call it that way. I try to put in all necessary checks for the SUT, that all fields are mapped and so on.

At that point I also come up with the first special scenarios I want to check, if they are not anyway already mentioned somewhere in the story description or test tasks. So I continue building up my test case and try some variations here and there, and compare the results with what I expect, what the requirements state (not necessarily the same thing), and how data passes through the code.

I tend to run my tests in debug mode, so that I can see what different variables hold. Additional request parameters, additional response fields, unnecessary duplication of information. That often gives me also more insights in the architecture and design. Why are we handing out internal IDs on that interface? Is that information really necessary in this business case? Why does it hold information X twice in different elements of the DTO? Can we probably remove one of them?

I also like to debug the application when running my test case. Do we pass through all layers? Why is that method bypassing the permission check layer? Do we need special permissions? Ah, time to check if the permission concept makes sense! This step is converting from one DTO to another? Couldn’t we then just take the other DTO in the first place? Persistence layer, hooray! Let’s check for null checks and according not null columns in the database. Did the developers forget something? I might not be able to pass null on that parameter via the UI, but probably through the API directly?

I found more scenarios, all similar but not the same. Can we simply add a parameter table to the test and let run one implementation multiple times? What would the difference be? Do I really need to add test cases for all of them? What would the value be?

<rant>Recently I had an assignment to analyze some implementations for a customer. And there was this one(!!!) method that took care of 26 different variants of an entity. There wasn’t even a unit or integration test for it. They left it for QA to check it in the UI end-to-end-tests or manually! 26 scenarios! That is a point, where I as a tester go to the devs and ask if we could re-design that whole thing. Is that out of my competency? I don’t think so? I uncovered a risk for the code base, and I want to mitigate that risk. And mitigating it by writing 26 different test scenarios is not the way of choice! So stand up and kick some dev’s butt to clean up that mess! </rant>

I send in request A, and get back response B. Can I simply evaluate response B that the action C I wanted to trigger happened or did the endpoint just enrich response B with the information from request A and didn’t wait for action C to actually do something? Trust me, I have seen this not only once! I also have seen test cases where the author checked the request for the values they set in the request, because they mixed it up with the response?

Back to action C! How can I properly evaluate the outcome of action C? In the past years I had several projects where you always found proxy properties of the result that you could check. This is a bit like sensors in a particle accelerator. You won’t see or measure the actual particle that you wanted to detect, but the expected result of an interaction with other particles. This often happens in software testing, too, when it’s not possible to observe all entities from the level of your test. Request A triggers action C, but you don’t trust response B. You rather check for result D that action C will have caused if everything worked properly. This actually requires a lot of system thinking and understanding.

Then comes the part where I “just” want to try out things, that some call exploratory testing, some call it ad-hoc testing, I call it also testing, as it’s just another, important part of the whole thing where I try to gain trust in the implementation. Anyways, so I take some test scenario and play around with it. Adjust input variables, add things, leave out things, change users, or whatever comes to mind. You know this probably as “Ha, I wonder what happens, when…” moments in your testing. I might even end up with some worth-to-keep scenarios, but not necessarily.

Earlier this year I also had the context that I was adding test automation on unit and service layers for a customer-facing API. So basically in the service layer tests I was doing the same thing that the customers would do when integrating this API. I was the first customer! And thanks to some basic domain knowledge I could explore the APIs and analyze what is missing, what is too much, what I don’t care about, etc. I uncovered lots of issues with consistency, internal IDs, unnecessary information, mapping issues, and more, because I was investigating from a customer perspective and not just blindly accepting the pre-defined format! This was exploratory testing at it’s best for my perspective in that context!

When I implement new automated test cases, I also always test for usability and readability of the tests. So when implementing a scenario and for example the test set-up is too complicated to understand or even create, then I tend to find simpler ways, implementing helpers, generalize implementations to improve those aspects of the test framework for the new extended context it has to work in.

As some of the last steps of implementing automation I go through the test cases and throw out anything that is not necessary and clean up the test documentation parts. I don’t want to do and check more than necessary for the functionality under test, and I want others to understand it as good as possible. Which I have to say is often not that easy to achieve, because I tend to implement rather tricky and complex scenarios to cover many aspects. As I mentioned before, I’m a systems thinker and systems tend to become rather complicated or even complex rather quickly and I reflect that in my test cases! Poor colleagues!

Some might call this a process. Well, if you go down to terminology level, everything we do basically is a process, but it doesn’t necessarily follow a common process model. Because when we refer to “a process” in everyday language, like Maaike in the tweet I mentioned on top, we usually mean “a process model”. And this is why I totally agree with her. And some people who ride around on the term “process” don’t simply understand the point she wanted to make!
Context and exploration are so relevant and driving forces for me, that it’s impossible for me to describe a common process model of what I do, and that common sense (I know, not that common anymore) and my experience help me most in my daily work and not using some theoretical test design approaches. Test Automation, Automation in Testing, Exploratory Testing and Systems Thinking in general all go hand-in-hand for me in my daily work. I don’t want to split them and I don’t want three different sub-tasks on the JIRA board to track it!

I’m just not one of these types that read a requirement, analyze it, design a bunch of test cases for it, implement and/or execute them, and happily report on the outcome. Of course I come up with test ideas, when I read the requirement. And if I see tricky aspects, I mention them in the refinement or planning, that they can already be covered by the devs or rejected as unnecessary. When I actually get my hands on the piece of code, I want to see it, feel it, explore it. And when I decide that the solution dev came up with is okay and actually what I expected it to be, then I’m okay to stop further testing right there.

When I started writing this article, some weeks ago, I was made aware of that Maaret Pyhäjärvi also wrote about the intertwining of automation and exploration. You can find for example this excellent article for Quality Matters (1/2020). And there’s probably more on Maaret’s list of publications. And probably other people also wrote great posts about this topic. If you know any, please let me know in the comments.
But I wanted to write this post anyway, to help myself understand my “non-process” better, and because some people on LinkedIn and Twitter asked for it. And probably it adds something for someone.