Causes of Late Software Delivery and Overblown IT Budgets

Exceeding the budget for a project is a much more common problem than you might imagine. And not only do young startups face it, but large companies as well. So why, having such advanced business analysis and project management tools, can we still observe such cases? Of course, we do not take into account completely unique and unpredictable factors. In most cases, despite advanced tools and approaches, budget overruns often occur due to trivial reasons. We'll also tell you how to deal with such reasons so your project won't exceed your budget a few times. Enjoy the reading!

Budget influencers

However, before we talk about the causes of budget overruns, let's first discuss who forms and spends it. After all, the causes of budget overspending are often directly the mistakes of specific project participants with a certain area of responsibility, level of competence, and type of interest.

Owners/Sponsors

It all starts with the owners or sponsors, who are strategically responsible and provide the resources for the project. They often look at the project from an economic perspective, and their interest is primarily in making sure that the project pays for the money spent on it, as well as strengthening the company's market position.

At the same time, they may not often be interested in or may lack competence in the technical aspects of development. Accordingly, the problems of budget estimation which they may cause mainly consist in the desire to achieve the maximum commercial results with the minimum budget and underestimation of the real technical complexity of the development. More specifically, we will stop later on all kinds of problems with a similar basis and ways to solve them. Now let's consider in the same way the others involved in the project and affecting its budget.

Project team

The project team is usually an extension of the owner's team or of the sponsors of the parent company. They are mainly responsible for the detailed and consistent development of the project structure and cost. However, they may not be the direct implementers of the project, so their technical evaluation may not be exhaustive either. The problems they can cause are mainly similar to those of owners or sponsors. Still, they can also cause problems related to a sub-optimal project design, inadequate allocation of resources, and choice of consultants and contractors.

Consultants

Project consultants are third parties from outside companies who may have more expertise in the areas the project aims at and provide their expertise. Their responsibility is to fill gaps in technical, commercial, or other competencies and to provide the most accurate assessment of the necessary means and requirements of the project for its ultimate success. However, they, too, can cause problems that make budget overruns. Most of these problems are based on the fact that these outsiders may not be sufficiently immersed in all of the work processes of the client company and, therefore, may not accurately correlate the real needs and capabilities of the company with the means and goals for the project development.

Contractors

Contractors are the direct performers in project development, and their responsibility is to most accurately estimate the amount and complexity of all upcoming and possible types of work, which accordingly affects the budgeting of the project. Often, contractors can rightly take on the role of consultants, too, since direct experience not only in the industry itself but also in project development can be the most accurate indicator of the funds needed.

However, this is assuming a really great deal of both technical, management, and commercial experience, as well as a high level of openness and decency on the part of the contractors. Otherwise, they may overestimate their technical or management skills, experience in developing relevant projects, understanding of the industry, and so on. It can lead to problems such as failure to meet previously planned deadlines, the emergence of a large number of unplanned works, and many others.

As said before, these are the main persons who influence the formation and allocation of the project budget and can also be a reason for budget overruns. However, depending on the company, the industry, and the project itself, some parties may play multiple roles at the same time. For example, the project team may serve as both consultants and contractors. Or vice versa, other people may be involved in addition to the main ones, such as the owner's operations staff and regulatory authorities. Still, we will not take them into account in this article because these are too individual cases.

Now that we have understood who is responsible let's take a closer look at why do software projects always go over budget? And which ways do we have to prevent it?

Optimistic estimates

This is the most common and broadest cause of budget overruns, so don't be surprised that subsequent ones may be similar to it in some aspects. Also, everyone involved in the project may be responsible for this cause in one way or another. Optimistic estimates are when those estimating the budget expect that the work will be done most consistently and predictably and that no difficulties will arise in the process that will require separate time and effort to resolve.

This may come mostly from sponsors or owners, either because they believe in the project or because they lack the necessary expertise to anticipate all the possible difficulties, or simply because they believe that every minute of the work must yield a clean result. While in practice, it turns out that faith may be groundless, difficulties are inevitable, and 80% of productive working time is quite a good result.

The project team can also cause it. The higher a project member is to top management, the more distant they perceive the work processes and the more productive the project results they are trying to present. This can lead to wishful thinking or even deliberate lying. In this case, the entire structure of the project suffers, and consequently, the initial estimate of the budget becomes far from reality in the end.

And contractors can also be the cause of optimistic estimates, which they give to the customer to conclude a contract with them. Often they can set a price for their services that does not cover the actual cost of the work, which puts over-budgeting based on the project. Take a closer look at some of the causes of budget overruns associated with optimistic estimates and see how they can be avoided.

Insufficient time for research

Young teams often neglect research because it requires significant resources, which may be limited. Larger companies may neglect research, even though they have the resources for it, because they may be overconfident in their evaluation decisions based on extensive previous experience, which may not necessarily be fully applicable to a new project. In such cases, the project does not have sufficient initial data.

This can be expressed in the fact that the company starts the project without realizing how competitive it is, how it can win or lose to existing solutions, on the basis of what technology is better to implement it, what type and level of specialists are required for this and so on. Without full-fledged research, it is impossible to formulate the right initial requirements for the product, team, solutions, and practices.

How to avoid budget overruns due to a lack of research?

It is necessary not to save on it in order to initially understand whether it is worth launching such a project at all and, if so, what requirements it should meet.
If your company has sufficient competence, assign it to an internal department and allocate a sufficient amount of time.
If not, then spend on consultants, who may cost a few thousand dollars, but can save you tens or hundreds of thousands of dollars.

Inaccurate MVP definition

MVP, or minimum viable product, is a key result of the launch of any project because it allows the business to realize its interests while testing and improving its development strategy.

But if minimum viable product objectives are not clearly defined and implemented, then the minimum viable product itself will require a larger final budget than the initial one. After its release will not let you understand how best to form a new budget to develop a full-fledged product.

How to avoid inaccurate MVP definition?

Therefore, it is necessary to pay separate attention to MVP, particularly its budget formation and allocation, instead of comprehensive budget formation for the development and maintenance of a full-fledged product.
Thus, it is possible to reduce MVP functions to the priority minimum and, based on its release results, to allocate a separate budget for its divisions or iterations.

Incomplete scope definition

This can be a frequent mistake made by top management when they want to apply a solution to a particular problem. However, they may not have an idea of exactly how it should be technically implemented and what set of technical solutions is needed for it. Therefore the top management or the project team, which gives the development to a contractor, may often make the tasks too general and allocate less budget than necessary.

How to avoid the incomplete scope definition before the beginning of development?

It is necessary to negotiate the list of tasks, breaking them into smaller tasks with more understandable and predictable budget costs.
Then summarize them in the total development budget.

Underestimating project complexity

When tasks are most accurately defined and broken down into the smallest subtasks, you may get the impression that if the essence and number of tasks are as clear as possible, their execution will also be simple. But the complexity of performing even the smallest tasks, especially their aggregate, is a separate parameter, even when they are most accurately divided and defined.

How to avoid underestimating the complexity of the tasks?

It is necessary to understand that while the exact definition of the tasks is the responsibility of the project team with the participation of the performers, the assessment of the task complexity, on the contrary, is to a large extent, the responsibility of the performer.
The evaluation of the performer in this matter should be primarily considered when forming the budget.

Inaccurate planning

When you have decided on the priorities, number, and complexity of all the upcoming and possible tasks, the necessary budget is still unclear until it is translated into a precise execution plan. If the plan is not precise enough and does not consider all the associated tasks and processes, it is impossible to understand how much time to perform the tasks and how much the corresponding work will cost.

How to avoid budget overruns due to poor planning?

It is necessary for the plan to take into account all previously announced aspects of the tasks and all the features of the processes of their implementation.
Clearly draw the duration of all the upcoming and possible types of work.

Treating estimates as solid numbers

Okay, let's say you estimate the number, complexity, and duration of tasks as accurately as possible, but this could be the basis of another reason for further budget overruns, which is to treat the estimate as solid numbers.

You also need to understand that even the most accurate estimate is not final because there is no additional work only with very rare exceptions. And if you strictly adhere to the original estimates, not allowing the possibility of going beyond, you can either freeze the project because of its impossibility to continue without solving the difficulties encountered. Or ignore them and put the technical defects in the very foundation of the project, which later will appear and most likely at the most inopportune moment, requiring additional resources not only for their forced solution but also to compensate for the consequences. Simply speaking, this mistake is a direct path to technical debt.

How do you avoid treating estimates as solid numbers?

This has a fairly straightforward solution. Just stop doing it.
You need to understand that any project's development is always more complicated than it may initially seem, even with the most accurate estimates.
Extra work is an inevitability that must be treated accordingly.

150% of the estimated budget

Not being able to cover the final budget is also often a problem, mostly for young companies, because they have not yet gained real experience in developing complex projects with changing budgets and initially do not have a large reserve of deferred funds. This may be the case with large companies, though, when they cannot go over a certain budget for some reason, such as fear of losing tax benefits or other arrangements.

Of course, no one is saying that exceeding the budget by two or more times is something you should always be prepared for and take as a norm. Such an increase clearly indicates some mistakes in the formation of the budget. However, a reserve of half the amount of the original budget is normal practice for most large complex projects. And the lack of such a possibility can often be a cause of freezing a project, the unfreezing of which is undesirable because it will also require separate funds.

The conclusion here is simple.

First, you should follow all the practices discussed in this article to determine the project budget as accurately as possible.
Second, even so, provide yourself the opportunity to stretch the budget by half a time.

Poor operation

Well, having understood the reasons that lie at the beginning, now look at the possible causes of budget overruns, which lie in the development process. As in any reason, each party can play a role here to a greater or lesser extent, but it is still more relevant to the project team or contractors than to the owners and consultants. This is because instead of strategic, tactical factors and subject areas play a big role here. First, let's look at the rather fragmentary and trivial causes and then move on to the more complex ones.

Human error

It looks pretty trivial, but that's why the often forgotten reason for budget overruns is human error. It is really easy to forget about it when you have made all the right steps to estimate the cost of project development, made a quality plan that takes into account all the upcoming and possible work, formed a team of real experts, and so on. Of course, in this case, even strange to think that something can go wrong for reasons that do not have a decent explanation.

But it is important to remember that we work with people, and sometimes even the smartest and most experienced professionals can make a completely trivial mistake simply because they are human. On the way to work, they could have broken down their car, lost their wallet, slipped and fallen in a puddle, or all together. So they could come to work in a bad mood, lose concentration for a minute, and do something wrong at a critical moment.

How do you avoid human errors in overrunning the budget?

Each other's work should be open to review, and review should be done regularly.
This can include both trivial things like discussing solutions before they are implemented and more specific things like testing the plan, budget, or product.

Forgotten taxes

Another not very common but possible cause of budget overruns may be that the company simply forgot about taxes. That is, in addition to salaries for consultants and contractors, the company may prescribe in terms of cooperation the payment of their taxes but not take this into account in the budget. This is undesirable not only because you suddenly have to pay large debts at one moment but also to pay fines for non-payment of taxes, which may be higher than the taxes themselves.

Avoiding overpayment of taxes is easy enough

You just need to remember the terms of the contract and comply with them clearly, paying taxes on time, thus not creating financial and legal problems.

Poor control scope changes & creep

Now let's move on to more complex causes, or rather the lack of control over changes to the extensions to the project. As we said before, you need to understand in advance that the initial and final budget will necessarily be different, so you need to be flexible about changes in the project. But changes in a project can begin to multiply uncontrollably, requiring multiples of each month's budget increase.

This can often be because the owners/sponsors want to add more and more changes to the product, making it better and better, in their opinion. But you have to understand that adding changes doesn't always improve the product but always requires an expanded budget. Also, adding changes to the project can come from the project team or contractors as a reaction to difficulties and attempts to solve them, which also may not always be justified, but are always costly.

How to avoid budget overruns due to a lack of change control?

Any changes should not be immediately implemented only at the request of the project team or even the owners.
They require prior consideration of the current state of the project, its strategy, architecture, and infrastructure.
If changes are deemed really necessary and not contradictory to the current state of the project, then they must be approved and added to the queue for execution with the appropriate priority relative to earlier tasks.

Poor project leadership

A directly related cause to the previous and subsequent ones may be the lack of a good project leader. Yes, it may sound a bit pretentious, but it is really important because a good leader sees the picture from several angles at once and can convey that vision to each party.

Without a good leader, the project does not stagnate; on the contrary, it begins to fall apart because each side promotes its vision and makes decisions primarily based on it. In such a case, of course, the product becomes either extremely imperfect from a technical point of view or uncompetitive on the market. As a result of all this, the final product will require either more expensive support or more expensive promotion.

How to avoid the causes of cost overruns due to lack of leadership?

You need a leader who understands technical fundamentals, commercial goals, and humanitarian points well.
He or she needs to take all this into account and make appropriate decisions that satisfy each party's vision or bring a unified vision to all.

Poor communication

A frequent cause of budget overruns can also be poor communication. This can be expressed in a large number of ways, such as misunderstanding the priority and relevance of tasks, procrastination and accumulation of difficulties, low motivation and weak initiative of project participants, and so on. Of course, solving all of this can require a very large budget overrun, and the further it goes, the bigger it gets.

How to avoid budget overruns due to poor communication?

It is necessary to make the most active use of all communication tools, such as Slack, Jira, and so on.
It is also important to actively use practices that increase the effectiveness of communication and support motivation and initiative.
And in general, it is necessary to create a culture of open and transparent development, in which each participant can propose a solution or share their vision, which will definitely be reviewed.

Speaking of which, we have a great article on how to manage a remote team the best way. There we share our best-proven practices that make development most stable, transparent, and efficient.

Poor cross-review task requirements

Another reason that follows from the previous one may not be obvious. The point is that there may be excellent communication between direct members, but not with indirect members. For example, the front-end and back-end development departments may not have the needed communication level, which means that project development solutions may not be the best in one case or another or their combination. When it turns out that the front-end solution is not as good as the back-end solution, a separate work time is required to dive into the context, adoption, and development of the new solution.

How do you avoid budget overruns due to poor cross-review task requirements?

It is necessary to establish a high level of communication between all project departments.
It is important that each department is at least in the general context of the work of other project departments.
Decisions that depend on the work of several departments should be discussed together and constantly monitored by each other.

Poor product maintenance

Well, now let's move on from the causes of budget overruns to the active development of its causes in maintaining the project. Here, too, there will be both trivial and rather complex factors, and the solutions for them can be fairly obvious and more subtle.

Maintenance costs

It is essential to initially understand that product development is endless. And it is necessary to start maintaining it as soon as it is launched, not when it starts making a profit. This means that project maintenance must also be budgeted for right away. Otherwise, even a successful launch can suffer a subsequent collapse due to the inability to pay for the initial fixes and improvements.

Also, cloud infrastructure to support CI/CD and automated testing can increase the cost depending on changes in the size of the project, the need for additional cloud space when the number of users increases, etc.   

How to avoid budget overruns due to project support?

First of all, you need to budget enough money to pay for changes before the first profit.
You also need to budget for project maintenance infrastructure for several times more than its cost from the initially expected user traffic.

Damage or malfunction costs

It is also important to understand that any successful product can be exposed to a variety of threats. Damage can be intentional or accidental, ranging from cyber-attacks to power surges or fires on servers.

Theft of user data, corporate data leakage, and even loss of the project's code base can occur. Fixing such things can require huge budget overruns, which can even lead to the company's bankruptcy and the project's destruction.

How to avoid budget overruns due to damage or malfunction?

First of all, you should pay much attention and by no means cut funds to testing the product.
Testing should be the most profound and diverse, regularly improved, and run automatically.
It is also necessary to allocate sufficient funds for a reliable backup of the code base and databases, which should also be provided with the proper level of security.

Environmental factors

It is also important to keep in mind that budget overruns can be caused by ignoring the fact that the market does not stand still and can change very quickly and dramatically as a result of global world events.

These can be political or economic crises, natural disasters, and other types of global changes. All that can affect currency exchange rates, the cost of services for the solutions you use, changes in demand for entire industries, and so on, which can greatly affect the profitability of your solution, even if you have done everything you can do properly.

How do you avoid overspending on your budget from environmental factors?

First and foremost, you need to actively monitor changes in the world and how they affect the pricing of the resources you use and the demand for the products you develop.
You need to be able to adapt and have decent alternative resources and service providers for different scenarios.
And, of course, you need to allocate additional funds to pay for resources and services that you cannot do without if their cost increases.

Summary

95% of problems can be fixed with proper communication and development processes. Since 2004, we’ve worked on dozens of highly-loaded IT projects and have come up with practical ways to mitigate risks, adequately plan project scopes, and build a corporate culture of rock stars. Although Mad Devs has a long journey ahead of us, we’ve started documenting our knowledge to share with companies that face similar challenges.

In this article, we’ve summarized a very generic list of common reasons why your software projects are late and exceed initial budgets. We hope you’ve found them helpful.