Test Pappy

In 2016 I gave my first talk at a testing conference – TestBash Brighton. The topic was about the power of ignorance. One model that I tried to convey back then was the rain drop model. Based on the concentric circles that rain drops produce on the water surface. Each rain drop stands for a particular topic or system. The growing circle demonstrates the growth of knowledge. And it visualizes wonderfully how certain circles start to intersect.

I am able to see this model in action. My daughter is well into her teenage years now. For at least six or seven years she loved the audio books of German author Marc-Uwe Kling about the adventures of living with a communist kangaroo. In the beginning it was purely because of the funny voices and some simple jokes. The books go rather deep in many aspects around politics, culture, economics, history and so many more topics. It’s four books with many small chapters, that help more or less with the overarching plot.

My daughter learned in school more about history, economics, politics, etc. Some topics we explained to her, if she wanted or not. Every time we listen to a few chapters again you see the Eureka! moments in her eyes. She makes more and more of the connections of complex topics. And thanks to the kangaroo she is able to memorize some of these topics better.

Imagine all these circles as systems. And the more you learn about each system, your knowledge grows. You learn about more than one system all the time. So have several growing knowledge circles. And from time to time these circles start to intersect. That is the moment when two system become one. You finally see the interconnection. And then you cannot unsee it anymore.

These Eureka! moments when you spot another connection of two systems that you thought were not connected. And then you see more complex dependencies. When a system that started to intersect on “one side” with another system, that also intersects with something that was formerly another distinct system. So not only does system A have a dependency with system B. Due to system B having a dependency with system C, there might be dependencies between A and C as well. And so the knowledge and understanding of the world grows.

It’s these moments when you hear or read something, where bits and pieces of one system are mentioned in the context of a completely different system. And then you connect the dots. And with the new knowledge you start explaining things that you could not before.

When you read about certain historic events and now current events make more sense. When you understand some chemical or physical process, and suddenly certain applications make sense. Maybe you look into a machine to understand what it does, and now the process it supports makes more sense. When you look at a piece of code, and you finally understand why that bug happens in certain situations.

Practice your systems thinking, connect the dots, understand the intertwinedness of the world. The world is complex and many things are more connected than we might initially think. Try to understand these connections, and the world becomes a place you might better understand. Gaining that knowledge doesn’t mean the world becomes a better place. And your frustration with humans will grow. But at least you know why.

Stay curious!

Unpopular opinion: even in Agile we just work in many small waterfalls or V-models, in parallel, all the time. Scrum and Kanban in action feels more like my ADHD brain works. Doing all the things, all the time, and in parallel.

Don’t get me wrong. My brain is fine with working that way. But do you know what is merely impossible? Drawing a model that represents that complexity in a way that also neuro-typical people understand where potential problems might be.

I already hear the Scrum masters and Agile coaches facepalming hard how wrong this is. But this is my reality for over 10 years now. The 15 years before that I worked in waterfall/V-model. And my assumption – you know, I’m good at those – is that it is true for others as well.

The models look so sweet and clean. Circles, cycles, the DevOps ∞, and what not. Why do we choose these hamster wheel representations? I don’t want to start over with everything every two weeks.

We are working on features, epics, stories, tasks, you name it. We don’t have one of each. We have multiple of each. They are not in the same state. Tickets are being drafted, refined, estimated, planned, put in progress, are in code review, being tested, delivered, deployed, and a dozen other things that I forgot. All the time. In parallel we have these reoccurring meetings. Plannings, refinements, reviews, daily stand-ups, and some more. Every x weeks we increase the number of our sprint name. Sometimes we do, sometimes we do not also deliver a bunch of work. We don’t deliver for one task, story, epic or feature. We mostly deliver for parts of many. That doesn’t mean they are all complete by then.

The power of Agile is fast feedback cycles. Now that you get frequent and fast feedback, you need to bring the reaction to the feedback back into your work management.

Then there are these pesky dependencies. Task A can only be finished when Task B is ready. Task B relies on Epic C, but that is waiting for something from Feature D. And Feature D is waiting for production feedback from Task A. Have I mentioned timelines already? Oh, people want to get things delivered by certain dates.

All this means that we constantly manage dozens and dozens parallel waterfalls, that are in different stages of the waterfall, on many different levels. As a tester you have to keep up with a whole bunch of tickets that your developers are working on. You have to jump in before and after and be involved with all of them to provide fast feedback. Early feedback enables to save time later.

Depending on the complexity of your delivery process you add also the stress of delivering every now and then. What used to be once or twice a year is now every other week or every week. If you deliver daily I’d assume that your delivery process is as smooth and lean as it could be. All others have to deal with delivery stress and overhead 20-50 times a year now.

Sure, the impact on production is smaller when you mess up. Or is it? It depends, I would say.

And then someone comes and shows you a simple circular model how Agile works. That someone better runs for cover.

I recently thought a lot about simplicity and complexity. And I came to the conclusion, that simplicity is often only possible when you shift complexity. The complexity doesn’t go away. Setting boundaries reduces complexity, but someone has to set these boundaries. Sometimes the shift is from many to few, sometimes from few to many. Let me explain my thought.

Let’s start with a wood working example. When you want to join two pieces of wood at a corner of any kind of box (most things in carpentry are variations of boxes), you have a ton of options. One of the options with the most split opinions in the wood working community is the Festool Domino. The split opinion is most probably based on the high price. It is a machine to create so called loose mortise and tenon joints. Something you can also cut by hand. Cutting mortise and tenon by hand is a tedious process and prone to failure. On the other hand, cutting them by hand also leaves lot of options for variations and adjustments. Through tenons, split tenons, and I don’t even know the names of all of them. The Domino cuts one variant. It does it precise and repeatable. The machine is not a trivial piece. It is an elegant yet complex solution. You can understand that, when you compare the attempts to build one yourself, or look at the different options that lately become available to achieve similar results on a budget price.

It is easy to use for a wood worker. To make joints in a simple way, you use something more complex. Or you use simpler tools, like a saw and chisels and you have to learn and practice making proper joinery. And the same is true for most of the machines that mankind invented. To make something simpler, you need to create and/or use something more complex.

Let’s look at another aspect in life. Going grocery shopping. I’m comparing Aldi with any big super market. It used to be even extremer, but Aldi increased their available options lately. Let’s choose the example of plain flour or pickled cucumbers. Aldi used to have one brand for plain wheat flour. There was one brand and two or three options for pickled cucumbers. When I go to the supermarket around the corner, I have the option between 6-8 brands for plain flour, and a shelf with 5-6 levels and about 10 meters long with different brands and types of pickled cucumbers.

Buying flour and pickles at Aldi is simple. I take this pack of flour and my favorite kind of the three pickled cucumbers. Done. At the big store, if I don’t have a clear favorite, I have to compare prices and brands and types, and what not. This takes time.

The complexity shifted to the person responsible for buying the products for Aldi. Which manufacturer is good, which product do most people want, who can deliver it for the best price. At the big super market, it feels like they buy just everything that is available on the market. Of course there is complexity connected here as well, getting the best prices, understanding how much of which brand is needed etc. That’s how each boundary is set. Aldi sets a narrow boundary, which makes for a simple choice. Buy this product or none. At the supermarket the boundary is much wider with lots more choices.

Take operating systems of the late 90s and early 00s. You had Linux systems where you needed to compile your own kernel to activate exactly what you needed and you got the best performance for your needs. (I know you can still do that.) Setting up a Linux system in the 90s was a pain. On the other hand, Windows. One size fits all, it kind of works, you have limited options to configure the system. You have to operate within stricter boundaries. Someone at Microsoft made the choices for you. With the amount of combinations of available hardware elements, it’s still closer to a wonder that this thing actually worked most of the time.
The same with iOS and Android. Especially at the beginning. iOS gave you a few options and basically it worked. Android gave you lots of freedom and options to set. That overwhelmed lots of people. These days it feels a lot closer. iOS widened their boundaries, while Android implemented ways to narrow them down. Both providing more or less back doors to move the boundaries.

Processes can set strict boundaries or very loose ones. We would call one with strict boundaries a complex process and one with less boundaries a simpler process. Yet the opposite might be the case, depending on the perspective. A simpler process means that you leave the option how to achieve a process step or get to the next process step to the process user. And here it depends on the needs, the skills, the motivations behind the process. What do I want to achieve and who is available.

A bureaucratic process at the city council. Strict rules to follow. Nobody likes these processes anyway, nobody has enough knowledge to understand the what and why. So it is easier to explain it step by step to allow less wiggle room and less room for errors in the sense that a good result can be achieved. Coming back to yesterday’s post, not even Shu (“Follow the rules”) is required to follow this process.

A process with less boundaries and more freedom means that the process users have to have more knowledge and skills and understand why and what is needed to come to the best result given the circumstances. These users need to have reached Shu to deliver good results. While the process looks simpler – less steps and rules – it shifted the complexity to the user of the process to know all the details.

While I could give you some work related examples, I stay at something more people will understand. Cooking! A recipe is nothing more than a process description. There are recipes or recipe books which explain you every step, every detail, and link basic techniques to other chapters, so that you can follow a precise recipe and get a good result. Writing a recipe like that is a lot of development work and very complex, but simple to follow. And then there are recipes like those from grandmas and mothers, which is a title and a list of ingredients. Of course with no measurements. That is something where you need to have reached at least the state of Ha (“Break the rules”) to get any edible output. Or a phone call to grandma – if possible – to ask for help.

The less boundaries you provide the more complexity shifts to someone else in the system to make the right things. The more boundaries and rules you set the simpler it is to follow, yet people will miss the freedom. It all comes down to the Goldilock Zone where things are just right. And they never will be right for all. There’s always someone who wants a different type of pickled cucumber, or knows better how to perform process step A.

From my point of view 2025 is set up to be another shitty year – on a large / global scale. But that doesn’t mean it will be a shitty year on a level that I have some control over.

I can get better one day at a time, one step at a time. One of my favorite wood workers always says, just be better today than you were yesterday. Another wood worker’s motto is to just do it! Trust me, in German it’s not an ad slogan.

When you combine those two, it means don’t over-plan it, rely on your experience and learn. Learn from failure, learn from success. I stay in wood working terms. When you hand-cut a dovetail you will have lots of failure. Some that you can repair, some that you can not. You will also have success. Many emphasize the aspect to learn from failure. Yes, true, in failures there is a lot of potential to learn and improve. But also in success there is potential to learn. Small habits, structures, routines that you did right. What was it that you did right that lead to success. Just because you did them right this time, doesn’t mean you will make them right the next time.

One of my former team leads was big into Japanese martial arts. And Shu-Ha-Ri was her motto. Shu: Learn the rules. Follow the rules, so that you can get to the result. The rules protect you. Ha: Remove the rules. You have achieved a natural flow. You understood why the rules are there. It is time to bend and break the rules. As you have achieved the Shu, you can do so in a safe and controlled way. Ri: Be the rule. You can build new rules for others to follow. You have reached the next level.

Whatever you do, try to be better than yesterday. Just a bit. You can not become perfect over night. It’s all about small changes, small improvements. One tiny step at a time. And you don’t need to reach Ri for everything that you do. Reaching a solid state of Shu is often more valuable.

Look at IT examples. A program is not perfect from the start. Because at the beginning, there is nothing. A program gets better one change at a time. Once it does what it needs to do, you can make it even better.

In nature, the stone gets hollowed out one drop at a time. A species improves generation by generation. Some things go faster, some take time. People don’t have time. Everything needs to happen now. Embrace time, embrace advancement, embrace being better today than you were yesterday. You don’t need to compete with others to be better as them in everything that you do. Be better than yourself, is already a lot to achieve.

I am not setting any new year’s resolutions. I know I have no discipline to follow them. So I just continue to try to get better. One blog post at a time, one test case at a time, one code change at a time, one cut into the wood at a time, one stitch at a time, one meal at a time, one bike maintenance at a time.

I try to be more in the present and focus on one thing at a time. I’m usually jumping between 3+ topics at the same time. And as we know, multi-tasking is the best way to get nothing done. Next week I will write about embracing the process.

Have a great 2025! Stay healthy! And become better, one step at a time.

You can build a lot of crazy things with the right foundation. You can also build a lot of crazy things without the right foundation, only then they won’t last long. But it’s so much easier.

This is true for constructions like buildings or bridges, friendships, processes, skills, money and so many more. With a solid foundation you have something to rely on, to fall back on. Something that holds the weight.

A solid foundation enables to achieve fantastic things. Without the foundation some of these achievements might be possible as well, but there is risk attached to it. Build a house on sandy grounds. It can work. Or it will bend at some point, and maybe even collapse.
You can do some interesting and risky money investments and earn millions and millions. Or you loose everything and go bankrupt. With a solid foundation you would invest only as much as you can spare to keep your standard of living.
You can build great software. But at some point it will be too big to be maintained, delivered, operated and extended by more than one or two people. With the right guidelines and processes in place, you have a solid foundation upon which to expand.

The foundation has to fit what’s coming on top. You can over-size the foundation for a certain additional cost. And you can under-size the foundation and adding a certain amount of risk for failure.

Real world example from about two decades ago. There was a car manufacturer who built a new van. They re-used components of the platform from their largest mini-van. e.g. motors and transmissions. They realized soon that the percentage of broken transmissions was higher than usual for their models. The root causes for this increase was, that the transmission was not suitable for total weights that exceeded the limit of the mini-van. The total weight of the van and transport capacity was just a certain amount higher than the one for the mini-van. Putting a larger vehicle on a too small platform was obviously not a good idea.

Back to our beloved IT. You may think that you don’t need a solid foundation from the beginning. But the same is true for all these different types of topics, no matter if real or “virtual” world. Once the thing you build is over a certain point in the starting phase, it will be extremely hard and expensive to add more to the foundation in hindsight. In IT I mostly think about a solid process foundation. I remember my ITIL v3 foundation training back in 2003 or 2004. Change Management, Bug Management, Release Management, Requirements Management, Configuration Management, and so on. Also the role of an architect, providing guidelines and rules for the solution. Of course you don’t need a service manager for each of these topics or a dedicated architect. But that doesn’t make these process needs go away. And I can guarantee you, that someone on your team will take over these roles, if they want or not. Someone will make a decision at some point on these. If they know about the difficulties of each topic or not.

Topics like ITIL are not taught anymore in the Agile age. The role of architects is discussed in different directions. And the same is true for so many other roles and processes. Projects are quickly at a point where you are beyond just hacking away on a prototype. You need some consistency in your project, a solid foundation that you can rely on.

Imagine your project is going on for 2 years, 5 years, 10 years. You realize that you need to change something fundamental. Even though we don’t talk about bricks and mortar on a concrete platform, moving some of these projects will become a real pain and requires a lot of effort.

There are not many people who step up and in to fill these roles, when there is no dedicated person assigned. Too many people I have met are not even aware of the explicit need for these roles. These tasks are not working with the shiny new shit that everybody celebrates. This is defining processes, rules, guidelines, that no one wants to follow. Change is hard, and people are allergic to change. Working on project foundations is a task that is not bringing you friends and fame. Freely translating a German saying: you can not win a flower pot with these.

Working on these foundations needs knowledge and experience, a crystal bowl, stamina, and a lot of patience and endurance. And I would say, a good solid foundation in systems thinking is extremely useful as well.

Take a moment and think about your project, your product or your company. How are you doing things like Release Management, Config Management, Change Management or Requirements Management? Is there some person or even a team? Are there processes in place? Are all people and teams following the same approach? Is the approach suitable for the situation you are in? Does it scale?

When a system works, we – humans – too often ensure that we bring it to the brink of failure.

Many systems around us are working at the edge of failure. Any issue of a certain severity makes it fail. Reliability seems to be an afterthought, robustness just too expensive. We want to make everything better, more efficient, more effective, just more. We always want more. We accept failures to happen. We extend systems until we don’t understand them anymore. We seem to be incapable of accepting a simple solution as is.

<rant>Look at ecosystems. Humans intervene with them all the time. And then they wonder why the system collapses. Either we don’t care or we don’t understand it.</rant>

There are many ways how a system may fail.

Adding to the capabilities – forgetting the core problem to solve.

I’m an MS Word user for 30+ years. The last time I used Word, I needed most probably less than 0.1% of the functionality. Maybe even less.

Programs and apps get so convoluted, just to add more and more functionality. Some applications became so popular because they were simple, solved an immediate problem at hand, and were good at it. But then comes the effect of we need more. Of course, if something is good enough, we cannot leave it at that. We need to add more to make it even better. Just to make it worse.

Rarely do I know of a product that got better by adding more and more capabilities. Not every product needs to be a Swiss Army Knife. If the product solves something very good and in a very efficient way, there is no need to make it do more. Is it inventors, business people, egos, I don’t know. When someone had success with a solution for whatever reason, they want more. There are not many inventors who made one thing with huge impact, and then created the next brilliant thing. In software it’s even worse. When you have one good application, the owners start putting more and more functionality in. Because only more is more.

Look at cars. The majority of people who want a car need it to move themselves, their families or friends and a bunch of goods from A to B. In the first world we build cars that are so luxurious and overpowered, that only the top 1% of the population can actually afford them, even though 5% or so buy them. Are we building cars for the wrong reasons these days? What happened to the Volkswagen Beetle?

Reaching capacity limits

Systems that cannot handle the load anymore. It starts with a simple solution, maybe you thought ahead for a bit. Over time more and more was added. The system needs to do more, or there are more actors in the system, and finally the architecture of the system starts to fail. The system has reached its capacity limit.

There are examples in software applications, hardware infrastructure, or systems like your local public transportation network. Look at the suburban trains in Munich. The system is designed to bring people from the suburbs into downtown. As Munich has a distinct center they designed the rails in a way that lead from West to East through the middle of town. And every suburban line is attached to each end. Yes, they all meet at both ends and go through the middle, on one track each. You can imagine how fragile this system is all the time. We have long reached the capacity limit of trains going through the tunnel. Oh, yes, the central part with one track in each direction is underground.

Optimizing for failure

Trying to make things lean (like processes or people) up until the point that too much was removed from the system. When key elements of a system are removed that took care of the stability of the solution, the system fails. Teams, groups, companies, but also many other types of systems are continuously optimized.

We start with a system to solve a certain purpose. In the beginning we might not know what exactly is needed, so we add a bit extra to be on the safe side to build a good solution. At a certain point in time the system is ready to start solving the problem at hand. And now comes the point, where we start with optimizing. We take elements away, that were too much. We bit by bit reduce the nodes of the system. When you do it slowly and carefully, you can react to malfunctions of the system and reintroduce the element back. When you do it too quickly, it will be harder to understand what just happened and why it starts failing. You have removed a service that was doing one function more than you thought it was, and now the system doesn’t work anymore. As you don’t know that it was that service, it will take a bit to understand.

Or you start removing people from the teams, redistributing them to other projects. People are not clearly defined resources though, even if they are often treated like that. There is knowledge, enthusiasm, engagement, relations, and many more that defines a person. You cannot just remove a person without thinking that it has only the impact that the key role or job of the person is no longer done. That is a bit short-sighted.

Don’t forget resilience and robustness, if you have the chance

Eco-systems often entail a certain resilience and ability to react to change. Humans can get away with a lot of change until the system collapses more or less from one day to the other. But things don’t just happen. Remember the web of causation?

When you are part of a system and you can design the system, keep resilience and robustness in mind. You have only so much influence on your system. You never know what will happen. The most pesky pests of all: users! Users! The world could be so nice without them. Users are creative, stupid, intelligent, malicious, arrogant, and so many other things. If you don’t want your system to fail, keep that in mind.

Prepare for tomorrow. If the system is not part of a very well defined larger system, things will change. Know what your limits are or implement monitoring that show you early when you reach them. Design the system in a way that makes it easy to maintain and extend. To take an IT example. Writing down 100 lines of code to implement a certain functionality is easy. But implementing it in a way that you can still understand tomorrow what it does, or that it is easy to adjust or remove is not that easy. Making a system scale, failover tolerant, recoverable, these are the hard parts.

One last rant!

Because it just came up in the latest episode of the Cabrera Labs Podcast.

The whole is more than the sum of its parts.

No! It is not! If the whole is more than the sum of its parts, you just have not understood or found all of the parts, relations and perspectives.

In an article from August 2019 my friend Damian Synadinos taught me about the concept of “differentia”. The distinguishing characteristics of a thing that makes it possible to differentiate it from another thing.

In the DSRP-method for systems thinking, D stands for Distinction. It’s all about what a node of the system is and what it is not. Sometimes that’s as easy as, this tree is this tree and not all the other trees around it. Sounds stupid, but can be that simple. Not always it’s that simple. Where do you draw the border between a thing and not-a-thing. And is the border part of the thing or not?

The distinctive element of a node in the system can be very helpful. Let’s go back into the world of IT and look at a service that handles log-ins. This service is responsible for authenticating users. In one case it is not responsible for storing the user’s real name. In another case it is. So it can be helpful in the description of a service what its responsibilities are. It can also be helpful to distinguish what the service is not responsible for. This creates a clearer expectation towards the service. It handles the login, but not the storage of further user data. This defines the borders of the element better.

Maybe you have to dissect the node into smaller parts to find the differentia. Take two types of trees for example. On first sight they might be similar. But then you look at the parts, like the bark, or the leaves, or the fruits. Maybe the bark looks the same, maybe the fruits look the same. But the leaves have distinguishable elements. That might not help in winter, when the tree lost all its leaves. Then you need to keep searching for further distinguishable elements.

The differentia can also help to describe your job. The job description can be a list of tasks that your role has to do. Are there tasks your role will not do? Are there tasks your role could do but doesn’t have to? Where does your role start and end? That a tester is not responsible for finance and controlling might be obvious. But actually, I had that task as a tester once. For a while I was taking care of my team’s controlling. Was that in my job description? Probably not. But was it excluded in my job description? No. Would it make sense to create a job description with that amount of information what your job is not? Not at all. But there might be important details that should be listed, that you don’t have to do. One of the most important elements of the job description for me was: “Don’t need to travel, if you don’t want to.”

When a small child sees a cat, it knows it’s a cat. Cats all look roughly similar. The differentiae are not that huge. Though the same happens for dogs. A small child knows that it is a dog, no matter if it’s a Chihuahua or a German Shepard. The differentiae are quite enormous. But the dog typical elements seem to be clear as well. So there is something about dogs that makes it clear that it is a dog and not something else.

So, it is important to describe what it is, but also to a useful degree what it is not. Next time you describe an element of a system. Think about what it is not. And it will become much clearer what it actually is.