Skip to main content

Technical Debt is Real Financial Trouble

Do you shun the people at work who hassle you with statistics about your company's software cyclomatic complexity?  Are you tired of this issue, frequently raised by developers whom you need, but whom you privately think of as highly strung prima donnas?  Do the words "JUST GO AWAY" not seem to work with these people?
From http://www.comicvine.com/prophet-of-doom/29-66484/
You and I may find this counter-intuitive, but it turns out these geeky prophets of doom have a point.  "Technical debt" has actually been proven in the real world to cost you money both directly and indirectly.  Lots of money.  Moreover, you incur additional security risks when you turn a blind eye to unnecessary code complexity.  Your current laissez-faire attitude towards code quality could actually turn into an unplanned financial catastrophe at an inconvenient moment in the future.

The "golden age" of research on costs of cyclomatic complexity seems to have been between 1990 and 1992.  A lot of the original work has perhaps been tainted for the modern agile audience because it was done to justify the use of Computer Aided Software Engineering tools (CASE), discussion about which has gone out of vogue.  But studies since the 1990s have consistently had the same results, and if anything, the situation has only gotten worse as the world's code base has grown.

Quick Technical Debt FAQ:

What is Technical Debt?  The phrase was coined by Ward Cunningham in 1992, actually.
Shipping first time code is like going into debt. A little debt speeds development so long as it is paid back promptly with a rewrite... The danger occurs when the debt is not repaid. Every minute spent on not-quite-right code counts as interest on that debt. Entire engineering organizations can be brought to a stand-still under the debt load of an unconsolidated implementation, object-oriented or otherwise.
Right, so it's just coders being fussy.  It's a matter of opinion.  Uh, no.  It is measured in terms of things that actually should be fixed quickly rather than allowed to linger, for actual business reasons.  Examples of this are not aesthetic or subjective.  They are countable things like:
  • High number of known open defects logged in your test repository.
  • Lack of an automated test base, so you have to pay a lot for manual testing in order to know from one release to the next what you might have broken.
  • High cyclomatic complexity, which is a term developed by Thomas J. McCabe, Sr., and is similar to the Flesch-Kinkaid Readibility Test.  It measures independent paths through a software system, if you consider all "if" or "case" statements, and the like.  A high score indicates high complexity.
  • High amount of duplication of individual lines of code.  This means that to "fix" it if it's wrong, you have to go more places and fix it more than once.
  • Very long methods Just as you have to put more effort into reading a blog than a tweet, a programmer has to work harder to make sense of an individual function or method of 1000 lines than of one limited to 100 or so.
  • SEI Maintainability Index -- a mechanical combination of this type of factors to a score indicating overall difficulty of making changes to the codebase as needed.
Why would those things translate to an actual cost?  As your developers would aver, it's fairly intuitive that it costs more to maintain something complicated than something simple.  They seldom describe this impact in terms of money, however.  So let's look at the two main scenarios where the problem moves from "coder stress" to "CFO expenditure."  I did a casual survey of the available interweb research and found the following.  A rigorous review would surely find more:
  • Routine Software Maintenance:  The original referreed MIT research on the cost of complex software in an enterprise environment found that when you control for other variables, "high levels of software complexity account for approximately twenty-five percent of maintenance costs or more than seventeen percent of total life-cycle costs."  This study was published in 1990, when state of the art for commercial software was COBOL.  One can conjecture that in modern multi-component software architectures, opportunities have arisen for even more complexity, and even higher maintenance costs.
  • Security Risk:  A 2012 white paper published by the Coverty company documents software "mistakes" which cost billions of dollars, and for Bank of America, a stock plunge of 15 percent.
Oh, I get it, isn't this is just a classic long-term/short-term decision about capital spending?  Well, yes and no.  The "short term" doesn't last very long on a project.  As Ward Cunningham said in the quotation above, shortcuts are fine for any given release, if your project is small.  On a large project, where you don't release for several months at a time, you can easily accumulate technical debt even before you deploy to market.  Technical debt doesn't age well.  It is a "payday loan" type situation, not the kind they advertise on the radio with "NO MONEY DOWN, NO PAYMENTS UNTIL JUNE!"  The longer you wait to fix the complexity, the more expensive it gets to fix.  Complexity gets added to complexity.  Like financial debt, technical debt compounds.

Your policy should be to maintain a constant state of low complexity except in exceptional cases, and then only with a plan to go back and fix the anomalous complexity immediately after the market advantage of the premature release has been enjoyed.


In the diagram above, "A," "B," "C,"and "D" represent different spending levels to get the same software written on behalf of your company.

A:  represents the money you are budgeting for total cost of ownership of your software today, which calls for paying no attention whatsoever to the financial risks inherent in your increasingly complex code base.

B:  represents the money you could be budgeting and purposefully spending to to avoid expensive maintenance and security risks.

C: represents the money you are spending today.  Technical debt isn't hypothetical.  You are spending 17-25% more than you should be on BAU software support, and that is a conservative estimate.  That's every single day.

D: represents the money you think your developers want you to spend on making the code "elegant."  And maybe they do.  But while you are busy trying to do "A" instead of "D," you have lost track of the opportunity to go from "C" to "B."

It's not just you.  In 2011, Andy Kyte at Gartner estimated that 60 years of accumulated technical debt has created a world-wide ticking financial time bomb worth $500 billion in 2010, with the potential to rise to $1 trillion by 2015.  What's worse is that every day, companies are spending billions just to keep maintaining this mess.  This isn't a "hypothetical" -- it is a cost to your business right now.  Can you really afford this?  Get a jump on your competition by systematically finding and eliminating technical debt the way you would any other serious organizational risk.

Popular posts from this blog

A Corporate Agile 10-point Checklist

I'm pretty sure my few remaining friends in the "small, collocated team agile" community are going to desert me after this, but I actually have a checklist of 10 things to think about if you're a product owner at a big company thinking of trying out some agile today.  Some of these might even apply to you if you're in a smaller place.  So at the risk of inciting an anti-checklist riot (I'm sorry, Pez!), I am putting this out there in case it is helpful to someone else.

Here's what you should think about:

1.Your staffing pattern.  A full agile project requires that you have the full team engaged for the whole duration of the project at the right ratios.  So as you provision the project, check to see whether you can arrange this staffing pattern.  If not, you will encounter risks because of missing people.  Concretely it means that:
a.You need your user experience people (if applicable) and your analysts at the beginning of the project, as always, b…

The Agile Business Case

Many agile teams have never seen a business case, ever, and they may even be proud of it.

Our mantra is that we deliver "business value," not just "software," quicker, better, and faster, but if so, we certainly don't spend a lot of time reporting on value delivery, and in fact we may be scornful about "analysis paralysis."  As software developers, we consider ourselves to be doing quite well if we can deliver the software every two weeks (or continuously).  And this is particularly if we've enabled this frequent high-quality delivery through automated testing and automated build-and-release techniques.  We've reduced business risk by making results visible more often, and allowing the business to change direction more frequently.  We assert that along the way of course we're also delivering value.  But how would we prove it?

I've recently posited that we shouldn't even think of doing agile projects without capturing and recording s…

Requirements Traceability in Agile Software Development

One of the grim proving grounds for the would-be agile business analyst (henceforth "WBABA")  is the "traceability conversation."  Eventually, you will have to have one.  You may have seen one already.  If you haven't, you may want to half-avert your eyes as you read further.  It gets a little brutal.  But if you close them all the way, you can't read.
Dialogue:
WBABA:   ...so in summary, we complete analysis on each story card, and then we support the developers as they build it that same iteration!Corporate Standards Guy:  but how do you do traceability in agile?  You have to have traceability.  It's broadly recognized as an important factor in building rigorous software systems. These software systems permeate our society and we must entrust them with lives of everyday people on a daily basis. [The last two sentences are an actual quotation from the Center of Excellence for Software Traceability website!] WBABA: [cowed silence]Corporate Standards …