This could be one of the extra chapters of The Programmer’s Black Book. In too many cases, I see the following work dynamic (and I have also fallen into it many times, although less now): we work on a project and our main obsession is to add functionality, new features, now this, now that, and so on for months when we believe we are moving forward.
Since that is probably the most attractive aspect for software developers, that is what we do, and we enjoy watching the application grow with new features, to our delight and satisfaction. For the time being, the developers are happy, and so are the business and client.
Everything “seems” to be running smoothly, like those relationships that appear to be the perfect couple but then in private they throw things at each other’s heads; in other words, something is rotten somewhere that will eventually explode at the least expected moment.
Going back to the software project, you may not have realized what is being created, and you may even see the situation as normal, because that is what we are paid for, right? To make something work and provide value to someone by adding more functionality and implementing requirements.
But at the same time, without realizing it, you are planting the seed of chaos in your project, especially if it has a life of many years and must handle countless changes, or if you have no idea what kind of changes will be required in the medium term, which is usually the case.
Does any business know where its growth will come from in a few years? Very few can even have an idea. That is: if the application you build is going to support that business, we have no idea how it will evolve.
Time passes, the application or system grows out of control, and we find ourselves living in constant chaos. The uncontrolled organic growth of the system has created an unmanageable monster.
This chaos can come in different forms: a system that becomes increasingly fragile (you touch one thing and another breaks), functionality added too quickly without respecting how things are done in that project (i.e., without following the design, if one exists clearly), the need to spend too much time making any change (time is money, a cost the project must pay), and, as if by magic, there comes a moment when adding, putting in, or removing something in the project becomes a real nightmare, and we then miss the moment when we began with so much enthusiasm, when the baby had not yet turned into that rebellious and unbearable teenager…
Over time, you end up maintaining a fragile, highly coupled system with a design that is more monolithic than flexible.
This means: the uncontrolled organic growth of a project leads to a spaghetti project that is costly to maintain and evolve. The result? Frustrated developers and managers who eventually have to defend to the leadership the idea of starting everything over from scratch, but “better.” Sometimes they even defend a new “version” when the reality is that they are throwing everything away and rebuilding the system from scratch.
The question I want to raise here is why this unsustainable situation occurs, a situation I have experienced while working with some of my clients and which I admit is not pleasant at all.
The dynamic is well known: add more and more functionality almost uncontrollably because time is pressing and the business needs all that functionality now, increasing the snowball until a moment comes when it crushes us.
The way to avoid it is also well known.
This is a universal principle in software: the cost of acting at the end (when it is already too late) is much greater than acting repeatedly throughout the life of the project to avoid the previous situation.
But acting… on what exactly?
This is what I call performing “technical pauses,” especially, I insist, in projects or products that must have a life of several years.
There is no other recipe: part of our working time must be focused on improving the design and architecture of the solution.
I am not talking about making small day-to-day improvements by cleaning this and that, refactoring here and there. I am talking about setting aside time specifically to analyze the solution as a whole and find and synthesize improvements that allow it to grow in a better and more orderly way.
I estimate that this activity should take about a quarter of the available time when working on a long-lived project.
That may seem like a lot of time, but my experience shows that this apparent step back allows us to move forward much faster afterward, achieving a maintainable and higher-quality solution (and at lower cost).
I have seen it: if you keep adding functionality continuously without pausing to reflect on whether the design and architecture are keeping up with that functional growth, sooner or later you will have a hot potato on which someone will have to answer.
Set aside one out of every four or five sprints or work cycles to consolidate the functionality already in place (technical pause), improving any aspect you consider appropriate, refactoring the design even further, improving the architecture and the procedures for configuration management and deployment. It may seem like that effort is an expense (since it adds nothing new to the system), but time will prove that this effort will drastically reduce problems in the future and allow more features to be added later with less difficulty.
Just as the 80/20 Pareto rule says that 80% of results come from 20% of effort, in software I affirm that 25% of working time should be devoted to improving the current system to enable its future growth, at least in projects that must last years of development with a large number of changes.
A step back (in appearance) that will allow us to move several steps forward with greater certainty, security, and quality.