This is one of the aspects I like the least among the different activities within my responsibilities.
Estimating the effort required to generate a project is an art in itself, and in software, I would say it is even worse. Below I am going to list some of the reasons and uncertainties why I prefer to work on projects with a very long time horizon (which may last for years) or products that, in any case, have a longer life cycle.
In this other post, some time ago I commented on the advantages and disadvantages of working on products or projects, understanding by projects those commissions that begin on one date, end on another, and are quoted with a fixed budget.
One of my responsibilities is estimating the hours needed to develop a new project from scratch or create a new evolution for one already in production. More often than I would like, you find yourself in the situation of needing a crystal ball to make the best possible estimate, because the client expects you to set the work they want to commission at a fixed price and all you can estimate is approximately the hours required for its completion: if you get it right, great; if you quote fewer hours than are ultimately needed, then money is lost on the project or profitability drops; if it ends in fewer hours than were finally used, profitability increases.
I often discuss the matter with my developer colleagues who will bear the weight of the work, so we can contrast opinions.
I know there are estimation techniques that, in any case, try to minimize the worst-case scenario: that the project takes much longer than expected to finish, thus incurring extra costs not included in the quote and economic deviations (i.e., losing money, basically).
It is not about your experience developing or managing projects, even if you know the technology very well or the business universe related to them, or even if you know in detail the capabilities of the developers who will be most involved.
Inevitably, there are always uncertainties beyond your control and aspects against which you can do little: the client is not clear about certain functionalities they need to develop (they define them as they begin to see something working, which is also understandable), the client does not want to make too much effort to agree on a specification document (and you, since you want them to hire you for the project, are not too heavy-handed either when it comes to refining details, so you end up a little between a rock and a hard place), like when you buy a box of materials of any kind, the client wants a fixed price and a development time, unless you have a lot of discipline and a very helpful hand to achieve it, and you try to minimize meetings as much as possible (where many hours are spent), and if the project has dependencies on third parties, they will also condition the effort needed, even though at the beginning, when you are quoting, you have no idea how much.
In short: uncertainty + uncertainty + uncertainty…
That is, many times you are going in a certain sense blindly, which is paradoxical when the client wants a fixed price and you are dealing with an uncertainty of around 30% in time (to say something).
Or even worse, and something typical in large companies: that estimate is not made by anyone related to the technical team, but by someone from another department with insufficient experience.
And it can get even worse because for the client you are not the only candidate provider: my experience shows (and I hope it is not the same for everyone) that in the end, variables such as commitment, trust, and quality are inevitably subordinated to the final cost, or the same thing, they choose you based on price/hour as the only selection variable.
We know productivity depends on many factors, not just the talent or experience of each developer. Especially in small companies, it is common for a developer to have to deal with more than one project at the same time or, at any moment, have to put out a fire, adding wasted time due to changing contexts and finishing tasks for one project or another (time impossible to estimate).
On the other hand, you also feel the added pressure that the client should be highly satisfied so they come back to you or your company later, something that is sometimes difficult to manage with difficult clients.
If you had not realized it until now, there are good and bad clients (and you cannot always choose them, I fear): the first align more with what is specified and make a strong effort to let you work as well as possible because they see you as part of their team (they are committed to the project), while the second expect you to read their minds and take the liberty of changing specifications at any time, causing the team to lose hours modifying what was already done (unquoted time), or they do not understand that when a project is delivered, your part ends there unless there is an agreement for its maintenance. These latter are only involved in the project, and you, as its responsible, sometimes have to act as an umbrella to prevent the client from changing too many things in meetings with a certain level of tension.
You may clearly identify with the scenario above: they ask you to quote and build a building, but they do not tell you how many floors it has; they only tell you the size of the windows afterwards and we will see whether it has a garage or not, and yes, the delivery date is non-negotiable.
I am sorry if this is stressing you as you read it, but all of the above is part of the very common dynamics of many companies whose business model consists solely of contracting projects, at a fixed price, with a start and end date, and who, when trying to manage all that uncertainty, end up with stressed developers, rushing to get things done that require much more dedication and causing things in production to fail at any moment.
It is true that there is always some indeterminacy when specifying; that is why the agile approach is a wonderful tool for reducing that uncertainty, but the issue is that almost always you end up underestimating the real effort for one reason or another.
No matter how well you apply good design practices, preparing your software project for change with a good architecture and excellent code quality, you will not be able to amortize all that uncertainty easily.
Given that, it is perfectly understandable that projects are sometimes delivered with poor design and a lot of technical debt, due to the lack of sufficient margin to choose the projects you have to develop or manage, since the business dictates and you have to make it profitable to cover expenses, salaries, etc.
For all of this, when I come across a project with serious deficiencies, I never blame the developers who built it, but rather the conditions under which they did it. And, surprise, there is almost always a work dynamic behind it where everything has to be done before yesterday, salaries are reduced, or projects are underquoted and poorly specified.
To avoid this unpleasant dynamic as much as possible, I always try to identify the coordinating person on the client side and involve them as much as possible so they validate what is being progressed in small iterations (one to two weeks), but experience also shows that projects are taken on with a lot of enthusiasm at first and little by little, that recurring validation effort and that coordinator end up seeing it as just another heavy task (because they have other things to attend to in their own organization, of course).
What, then, is your role as a project manager, as in my case?
There is no doubt in my mind: I am neither the boss and I do not like hierarchies, nor do I like ordering like a general sending soldiers as cannon fodder. I am clear that my role assumes full responsibility for the good development of the project and its delivery with sufficient quality, and my main mission is to favor the conditions under which developers work, which also implies that the client does their part (which is not always easy).
As we know, creative work requires certain conditions for it to be carried out, and these are the responsibility of all members of a company to achieve, as well as the client.
And then? Is it possible to escape this previous dynamic of working on projects that are mostly underquoted (like under pressure)? How then can the profitability of each project be ensured?
There is no doubt in my mind: for this reason, for many years I have preferred the development of products that will evolve constantly for many years, or startups testing an idea and able to pivot development as many times as needed, or very long-term projects (like Hub de Libros). In those contexts you have enough room to create a quality project with little or no technical debt, with improvement iterations and orderly project growth.
As a software developer, I would a thousand times rather be involved in a product (which will be commercialized to tens, hundreds, or thousands of clients) than in projects quoted by the hour.
Handling uncertainty as I have described above and trying to get right the development effort is, for me, the worst aspect of my profession as a professional software developer.
Of course, there are tricks to reduce the negative impact of all of the above: sometimes it is worth attracting a new client and managing small margins so they get to know the quality of your work (hoping to make more contracts with the same client later and thus increase margins), use proprietary frameworks on which to base your projects so they can be done with good quality and in less time, put more effort into the ad-hoc projects you do for clients so they can be reused in some way in other projects, etc.