Welcome to my Theory Thursday newsletter where I dig into various different theories and models relating to management, leadership and bits which sit on the edge of software/product development.
This is a set of sub-articles that live alongside my main newsletter which I publish every Saturday morning.
If you aren’t already subscribed, please consider signing up - it’s completely free and helps me grow my audience.
Today’s article is a little different to my other Theory Thursday posts, mainly because it’s more technically focused than simple management theory.
This post was born from some conversations I had with some fellow friends in the industry, particularly around a company we knew of that was struggling with some crippling technical debt and now facing some financial issues. Money was put into the wrong places, so instead of building out a stronger engineering team that focused on moving away from the technical debt that once allowed the product to get to market fast, the team remained small and unnecessary middle management was brought in with little effect.
Before we get stuck into the detail, let’s define technical debt to make sure we’re on the same page.
What is ‘technical debt’?
A simplistic view of technical debt is “it’s the result of prioritising speedy delivery over perfect code”. When I say “perfect” I mean well-engineered, thought out code that is easy to read, easy to change (extensible), has good test coverage, and can be shipped frequently.
You could also look at technical debt in terms of cutting corners in order to save time or money today, knowing that you’ll have to go back and fix it later on. However, there are a few more levels to technical debt than just this simplistic view.
If we break it down further, we can look at intentional vs unintentional technical debt.
Intentional Technical Debt
Out of the two options, intentional technical debt is the more measured approach to taking on less-than-ideal builds to a system or product. They are often calculated decisions which factor in a number of things like:
Time constraints ⏰
Needing to meet a deadline, therefore limits need to be put in place
Budget constraints 💰
Money might prevent full builds, therefore some corners might need to be cut
Skill constraints 🛠️
The current skill of the team might not allow for a certain approach e.g. only GCP capabilities over AWS
Feature constraints 🔧
Using a JSON file as an initial data source instead of hooking into an actual database engine
The decision to move fast and take a hit in some of the areas identified in the above list can provide some advantages:
Quicker to market
Choosing to cut certain things out of the equation can result in [initially] faster deployments.
Prevents overengineering
As much as we want to deny it, engineers are prone to over-engineering solutions. We love making things super robust, and in some cases we reach a point of diminishing returns.
Flexibility
Intentionally choosing to take on some tech debt can allow for greater flexibility. E.g. having an API initially read data from a JSON file instead of a CMS means you don’t need to tie yourself into a CMS platform, and nor do you have to think about the complex integrations.
Faster feedback
Instead of investing a lot of time in the “perfect” solution, you can develop simpler solutions with intentional tech debt. You can then leverage feedback from users to hone the solution and make it more robust as you go.
Unintentional Technical Debt
Where intentional tech debt is down to a decision made by a software team, unintentional tech debt is the opposite.
What are the causes of unintentional tech debt?
There are a multitude of potential causes for unintentional tech debt, so let’s have a look at the most common:
Lack of knowledge
Inexperienced engineers may lack the expertise needed to implement the most optimal solutions or architectures.
e.g. A database schema may end up poorly normalised due to lack of understanding about good database design
Breakdown in communication
This can result in misalignment between stakeholders and engineering teams, resulting in the wrong things being built. This can create a culture of “quick fixes” which end up tainting the codebase and making it harder to understand and maintain.
e.g. Implementing a feature based on poor / no set requirements, only to find it doesn’t actually solve the problem.
Short-term focus
This is often a result of pressure to deliver work quickly with inadequate time for doing things the right way. Engineers will find themselves hacking solutions together, resulting in poor quality implementations and skipping things like TDD or code reviews.
e.g. A distinct lack of unit tests due to rushed development, resulting in a brittle, untestable codebase.
Lack of standards
This results in a free-for-all environment whereby nobody adheres to any set standards or approaches to engineering. The disparity, especially between teams, can lead to inconsistent and even conflicting approaches.
e.g. One team uses one framework whilst another uses something else (incompatible), making integrations challenging.
Tech limitations
This might be due to using old, outdated frameworks or libraries that result in critical limitations. It can also happen with the early adoption of immature technologies which quickly hit a wall due to limiting features.
e.g. Building a critical feature using a third-party API that suddenly changes its terms or performance.
This list is by no means exhausted, but it gives you a good flavour of the types of reasons why things can go downhill rapidly without proper thought.
What is the impact of unintentional tech debt?
Again, this list is by no means exhaustive, but there are a few tell-tale signs that it is causing pain:
Frequent bugs or incidents
Customers are always reporting issues and downtime is high
Code smells
There’s a large use of anti-patterns to good design and architecture like excessive duplication, large monolithic components/classes/modules.
Slow development cycle
It takes a long time to actually get anything from idea, through development and released out into production.
High onboarding time
Developer setup is painfully long and complicated - often a sign that things need to be simplified.
Dependency management pains
Outdated libraries or fragile frameworks can signal bad tech debt.
The list goes on…
Having all of this in place not only decreases developer productivity, but it can massively increase maintenance costs, create system instability, stifle innovation and ultimately result in frustration from stakeholders and customers alike.
Paying it back
So you’ve identified that your system has tech debt, whether it be intentional or not - don’t panic!
Identifying tech debt is the first major hurdle that you need to overcome, and that’s a massive step in the right direction. The next important thing is to prioritise and tackle the debt so that it is paid back and starts pushing those side effects further away.
Do these things:
Regularly review codebases to identify areas of debt, whether that is code smells, outdated libraries, broken tests, lack of pipelines etc.
Prioritise and categorise the debt based on the impact and urgency (e.g. critical bugs vs minor inefficiencies)
Refactor your codebase as part of regular development, following the boy scout rule. In other words, pay the debt back whilst your in the process of adding new features.
Establish best practice by implementing coding standards, CI/CD pipelines and test-driven development.
Invest in training to avoid costly mistakes, learning from failure as you go.
Allocate time to fix the debt, especially the chunkier tasks that couldn’t be done whilst working on another smaller ticket
Tech debt - it’s everywhere
So to wrap up, let’s just be honest with ourselves and be clear that there will always be some form of tech debt in our systems. There will always be an element of refactoring that could improve the codebase a little more, some libraries that could be updated or some dead code that could be removed to make it easier to understand.
Just be mindful of the implications that arise when you ignore tech debt, and if you need to, intentionally take on tech debt if you want to move a little faster - but for the love of God, pay it back!