Technical Debt is Real Financial Trouble

Do you shun the people at work who hassle you with statistics about your company's software cyclomatic complexity?  Are you tired of this issue, frequently raised by developers whom you need, but whom you privately think of as highly strung prima donnas?  Do the words "JUST GO AWAY" not seem to work with these people?

From http://www.comicvine.com/prophet-of-doom/29-66484/

You and I may find this counter-intuitive, but it turns out these geeky prophets of doom have a point.  "Technical debt" has actually been proven in the real world to cost you money both directly and indirectly.  Lots of money.  Moreover, you incur additional security risks when you turn a blind eye to unnecessary code complexity.  Your current laissez-faire attitude towards code quality could actually turn into an unplanned financial catastrophe at an inconvenient moment in the future.

The "golden age" of research on costs of cyclomatic complexity seems to have been between 1990 and 1992.  A lot of the original work has perhaps been tainted for the modern agile audience because it was done to justify the use of Computer Aided Software Engineering tools (CASE), discussion about which has gone out of vogue.  But studies since the 1990s have consistently had the same results, and if anything, the situation has only gotten worse as the world's code base has grown.

Quick Technical Debt FAQ:

What is Technical Debt?  The phrase was coined by Ward Cunningham in 1992, actually.

Shipping first time code is like going into debt. A little debt speeds development so long as it is paid back promptly with a rewrite... The danger occurs when the debt is not repaid. Every minute spent on not-quite-right code counts as interest on that debt. Entire engineering organizations can be brought to a stand-still under the debt load of an unconsolidated implementation, object-oriented or otherwise.

Right, so it's just coders being fussy.  It's a matter of opinion.  Uh, no.  It is measured in terms of things that actually should be fixed quickly rather than allowed to linger, for actual business reasons.  Examples of this are not aesthetic or subjective.  They are countable things like:

• High number of known open defects logged in your test repository.
• Lack of an automated test base, so you have to pay a lot for manual testing in order to know from one release to the next what you might have broken.
• High cyclomatic complexity, which is a term developed by Thomas J. McCabe, Sr., and is similar to the Flesch-Kinkaid Readibility Test.  It measures independent paths through a software system, if you consider all "if" or "case" statements, and the like.  A high score indicates high complexity.
• High amount of duplication of individual lines of code.  This means that to "fix" it if it's wrong, you have to go more places and fix it more than once.
• Very long methods Just as you have to put more effort into reading a blog than a tweet, a programmer has to work harder to make sense of an individual function or method of 1000 lines than of one limited to 100 or so.
• SEI Maintainability Index -- a mechanical combination of this type of factors to a score indicating overall difficulty of making changes to the codebase as needed.

Why would those things translate to an actual cost?  As your developers would aver, it's fairly intuitive that it costs more to maintain something complicated than something simple.  They seldom describe this impact in terms of money, however.  So let's look at the two main scenarios where the problem moves from "coder stress" to "CFO expenditure."  I did a casual survey of the available interweb research and found the following.  A rigorous review would surely find more:

• Routine Software Maintenance:  The original referreed MIT research on the cost of complex software in an enterprise environment found that when you control for other variables, "high levels of software complexity account for approximately twenty-five percent of maintenance costs or more than seventeen percent of total life-cycle costs."  This study was published in 1990, when state of the art for commercial software was COBOL.  One can conjecture that in modern multi-component software architectures, opportunities have arisen for even more complexity, and even higher maintenance costs.
• Security Risk:  A 2012 white paper published by the Coverty company documents software "mistakes" which cost billions of dollars, and for Bank of America, a stock plunge of 15 percent.

Oh, I get it, isn't this is just a classic long-term/short-term decision about capital spending?  Well, yes and no.  The "short term" doesn't last very long on a project.  As Ward Cunningham said in the quotation above, shortcuts are fine for any given release, if your project is small.  On a large project, where you don't release for several months at a time, you can easily accumulate technical debt even before you deploy to market.  Technical debt doesn't age well.  It is a "payday loan" type situation, not the kind they advertise on the radio with "NO MONEY DOWN, NO PAYMENTS UNTIL JUNE!"  The longer you wait to fix the complexity, the more expensive it gets to fix.  Complexity gets added to complexity.  Like financial debt, technical debt compounds.

Your policy should be to maintain a constant state of low complexity except in exceptional cases, and then only with a plan to go back and fix the anomalous complexity immediately after the market advantage of the premature release has been enjoyed.

In the diagram above, "A," "B," "C,"and "D" represent different spending levels to get the same software written on behalf of your company.

A:  represents the money you are budgeting for total cost of ownership of your software today, which calls for paying no attention whatsoever to the financial risks inherent in your increasingly complex code base.

B:  represents the money you could be budgeting and purposefully spending to to avoid expensive maintenance and security risks.

C: represents the money you are spending today.  Technical debt isn't hypothetical.  You are spending 17-25% more than you should be on BAU software support, and that is a conservative estimate.  That's every single day.

D: represents the money you think your developers want you to spend on making the code "elegant."  And maybe they do.  But while you are busy trying to do "A" instead of "D," you have lost track of the opportunity to go from "C" to "B."

It's not just you.  In 2011, Andy Kyte at Gartner estimated that 60 years of accumulated technical debt has created a world-wide ticking financial time bomb worth $500 billion in 2010, with the potential to rise to $1 trillion by 2015.  What's worse is that every day, companies are spending billions just to keep maintaining this mess.  This isn't a "hypothetical" -- it is a cost to your business right now.  Can you really afford this?  Get a jump on your competition by systematically finding and eliminating technical debt the way you would any other serious organizational risk.