10 Years of Research in Technical Debt and an Agenda for the Future

Technical debt is a widely recognized software engineering concern that refers to the tradeoff between the short-term benefits of rapid delivery and the long-term value of developing a software system that is easy to evolve, modify, repair, and sustain. Technical debt in a code base comprises those structural elements that, if not reworked, contribute to accumulation of increased operation and maintenance costs. In this blog post, we examine the evolution of the field of technical debt and identify open research questions that will drive future developments.

Researchers and organizations in government and industry recognize that if you do not actively manage technical debt, it will manage you. The U.S. Department of Defense has affirmed the importance of technical debt by specifically citing it in the National Defense Authorization Act of 2022. Section 836 of this act authorizes the U.S. secretary of defense to initiate a study of technical debt in software-intensive systems that includes analyses and actionable, specific guidance and recommendations for statutory or regulatory modifications.

Our work in technical debt is predicated on the principle that if you don’t make something visible, you won’t manage it. Avoiding the negative outcomes associated with accumulating technical debt requires organizations to embrace technical debt management as a core software engineering practice. Just using the phrase “technical debt” is not an engineering practice. We have long recommended that technical debt be added to an issues backlog to analyze how and where architecture requirements might be failing and that it be managed as an integral part of product-development activities.

Although there are several metrics in use today that aim to quantify technical debt from different perspectives—and there is no one-size-fits-all metric—we are gratified by the influence that our work on technical debt metrics has had on the field of software engineering. The International Conference on Software Architecture (ICSA) recently awarded us with the Most Influential Paper Award for a paper we wrote in 2012, In Search of a Metric for Architectural Technical Debt, along with our colleagues Philippe Kruchten and Marco Gonzalez-Rojas of the electrical and computer engineering department at the University of British Columbia. In the paper, we detailed how technical debt is rooted in software architecture and described how an architecture-focused and measurement-based approach to developing a metric based on propagation cost helps strategically manage technical debt. We demonstrated this approach by applying it to a system-development effort that was then underway.

The technical debt concept has proven to be a useful construct for communicating the tradeoff between rapid delivery and long-term system sustainability. Its influence is evident in Agile at scale and DevSecOps practices, as well as in government acquisition practices, such as the Adaptive Acquisition Framework and the Practical Software and Systems Measurement Continuous Iterative Development (Agile) Measurement Framework.

Unfortunately, our use of the term architectural technical debt in the title of our paper has been taken by some to imply that there is a taxonomy of various types of technical debt, of which architectural technical debt is just one. We therefore hear and read of documentation technical debt, security technical debt, and so on. It is important to clarify that this interpretation was not our intention. Our view is that technical debt is rooted in architecture- and design-related issues and the rework that these issues cause. When not managed, technical debt increases the likelihood of security issues and defects that further increase the accumulating consequences.

The essence of our approach—both when we wrote the paper and now—is this: The value of the delivered features and the impact of cost that will be incurred must be taken into account in decision-making related to delivering a product. Making technical debt visible provides the necessary information for making informed decisions about managing the potential impact of rework over time. In the interest of making debt visible, our paper presented a dependency-analysis framework for measuring architecture rework as a proxy for technical debt.

Our colleague and co-author Philippe Kruchten posed a useful way of characterizing elements in a software-development project throughout the development lifecycle. These elements are depicted in Figure 1.

Positive-value aspects are shown on the top row of Figure 1. The green represents the features and functionality that the project must deliver for its stakeholders, which is visible on the release schedule and drives the organization, the sales, the success, and the customer satisfaction. The yellow represents the architectural infrastructure and design constructs that are essential for the longevity, structure, and behavior of the system and for achieving its intended quality attributes. Features are visible to the user and the architecture is invisible to the world outside of the development team, though both add positive value.

Similarly, we have the negative-value aspects shown in the bottom row of Figure 1. The red represents defects that are visible to the user and must be managed as quickly as possible to avoid jeopardizing the system. The black represents technical debt that is an element invisible to the outside world resulting from not getting the full architecture requirements right. Technical debt accumulates as the product is built and key decisions are deferred or poor work is done.

The dependency framework we described in that paper takes into account how architecture enables functionality and how developers can deliver functionality while also attending to the architecture in the short term and the long term. In the paper, we compared two paths, shown in Figure 2.

We refer to Path 1, shown in blue in Figure 2, as the deliver soon path. It starts by delivering value but increasingly incurs rework (shown in the lighter shade of blue). We call Path 2, shown in red, the architecture-savvy or reduce rework and enable compatibility path. This path initially incurs a higher cost, but the cost of the rework is more consistent as the necessary architecture is in place to support subsequent iterations.

The point of creating a path comparison is to make the tradeoffs visible and explicit. Path 1 represents the case of when the system is new and its business value is being explored and discovered. In such circumstances, it is paramount to deliver some functionality and to test it in the field. As the use of the system proves valuable, the system begins to grow. In the example we describe in the paper, following the initial strategy created uncontrolled implementation costs, requiring a significant re-architecting effort, which was suboptimal for the long-term sustainment of the system. Performing such analysis on potential rework as the system grows can enable timely decisions about when to start re-architecting the system or when to pay back the interest on the borrowed time of the earlier design decisions. Our approach offered considerations at the architecture level for making rework explicit at each release.

Our paper also demonstrated how software architects can utilize a change-propagation metric to calculate and justify the cost of taking on technical debt. The quantification of rework is not trivial, however, especially when it comes to architecture. While no one metric alone can help developers optimize development cost over time, structural architecture analysis and related practices provide the perspective needed to make the cost of implementing the selected options explicit.

Our paper influenced numerous research methods and software tools in the 10 years since its publication, many of which have been validated in industrial case studies with software-development companies. This type of follow-up work takes into account how architecture enables functionality and how to achieve the balance between delivering functionality and paying attention to architecture in both the short and long term. Examples of this work include the following:

As time has passed, the body of work on understanding and managing technical debt and architecture evolution has grown. At the SEI, we plan to investigate how empirical data and analysis can be used to improve iterative and incremental architecture practices to manage technical debt.

A significant amount of research remains to create a technical debt quantification framework that is not only repeatable and reliable, but also cognizant of domain-specific system challenges. We encourage researchers in both software architecture and metrics to more deeply explore the relationship between the structure and behavior of architecture patterns and how metrics behave.

In the future, it will be important to clearly establish the validity of emerging methodologies for quantifying technical debt. Not all current metrics for characterizing and quantifying technical debt are proven and not all are trivial to use. While tools are increasing in sophistication, they tend to focus on the problem of architecture conformance rather than on the architectural aspects of quantifying rework.

Key open research questions in the field of technical debt research include the following:

• How do we quantify rework using different metrics to guide how and when to refactor systems to resolve technical debt?
• How do we refactor systems, especially correctly incorporating relevant metrics and tool support?
• How can we relate rework quantification to operational practices?
• How do we capture the empirical data that is coming from industry and government, analyze them, and use them to inform the proper use of iterative, incremental architecture practices and how to improve these practices along with managing technical debt.

We would welcome the opportunity to work directly with organizations interested in collaborating with us in this work. Please contact us at [email protected].

We also encourage researchers to consider working on seeking answers to the questions above and to become involved in the annual TechDebt Conference. This conference connects leading software engineering researchers and practitioners to discuss approaches for managing various types of technical debt, to share experiences and ideas, and to identify key challenges for industry and academia. It provides an opportunity to vet ideas among vendors, industry developers, and researchers in technical debt management.