Skip to main content

Technical Debt is Real Financial Trouble

Do you shun the people at work who hassle you with statistics about your company's software cyclomatic complexity?  Are you tired of this issue, frequently raised by developers whom you need, but whom you privately think of as highly strung prima donnas?  Do the words "JUST GO AWAY" not seem to work with these people?
From http://www.comicvine.com/prophet-of-doom/29-66484/
You and I may find this counter-intuitive, but it turns out these geeky prophets of doom have a point.  "Technical debt" has actually been proven in the real world to cost you money both directly and indirectly.  Lots of money.  Moreover, you incur additional security risks when you turn a blind eye to unnecessary code complexity.  Your current laissez-faire attitude towards code quality could actually turn into an unplanned financial catastrophe at an inconvenient moment in the future.

The "golden age" of research on costs of cyclomatic complexity seems to have been between 1990 and 1992.  A lot of the original work has perhaps been tainted for the modern agile audience because it was done to justify the use of Computer Aided Software Engineering tools (CASE), discussion about which has gone out of vogue.  But studies since the 1990s have consistently had the same results, and if anything, the situation has only gotten worse as the world's code base has grown.

Quick Technical Debt FAQ:

What is Technical Debt?  The phrase was coined by Ward Cunningham in 1992, actually.
Shipping first time code is like going into debt. A little debt speeds development so long as it is paid back promptly with a rewrite... The danger occurs when the debt is not repaid. Every minute spent on not-quite-right code counts as interest on that debt. Entire engineering organizations can be brought to a stand-still under the debt load of an unconsolidated implementation, object-oriented or otherwise.
Right, so it's just coders being fussy.  It's a matter of opinion.  Uh, no.  It is measured in terms of things that actually should be fixed quickly rather than allowed to linger, for actual business reasons.  Examples of this are not aesthetic or subjective.  They are countable things like:
  • High number of known open defects logged in your test repository.
  • Lack of an automated test base, so you have to pay a lot for manual testing in order to know from one release to the next what you might have broken.
  • High cyclomatic complexity, which is a term developed by Thomas J. McCabe, Sr., and is similar to the Flesch-Kinkaid Readibility Test.  It measures independent paths through a software system, if you consider all "if" or "case" statements, and the like.  A high score indicates high complexity.
  • High amount of duplication of individual lines of code.  This means that to "fix" it if it's wrong, you have to go more places and fix it more than once.
  • Very long methods Just as you have to put more effort into reading a blog than a tweet, a programmer has to work harder to make sense of an individual function or method of 1000 lines than of one limited to 100 or so.
  • SEI Maintainability Index -- a mechanical combination of this type of factors to a score indicating overall difficulty of making changes to the codebase as needed.
Why would those things translate to an actual cost?  As your developers would aver, it's fairly intuitive that it costs more to maintain something complicated than something simple.  They seldom describe this impact in terms of money, however.  So let's look at the two main scenarios where the problem moves from "coder stress" to "CFO expenditure."  I did a casual survey of the available interweb research and found the following.  A rigorous review would surely find more:
  • Routine Software Maintenance:  The original referreed MIT research on the cost of complex software in an enterprise environment found that when you control for other variables, "high levels of software complexity account for approximately twenty-five percent of maintenance costs or more than seventeen percent of total life-cycle costs."  This study was published in 1990, when state of the art for commercial software was COBOL.  One can conjecture that in modern multi-component software architectures, opportunities have arisen for even more complexity, and even higher maintenance costs.
  • Security Risk:  A 2012 white paper published by the Coverty company documents software "mistakes" which cost billions of dollars, and for Bank of America, a stock plunge of 15 percent.
Oh, I get it, isn't this is just a classic long-term/short-term decision about capital spending?  Well, yes and no.  The "short term" doesn't last very long on a project.  As Ward Cunningham said in the quotation above, shortcuts are fine for any given release, if your project is small.  On a large project, where you don't release for several months at a time, you can easily accumulate technical debt even before you deploy to market.  Technical debt doesn't age well.  It is a "payday loan" type situation, not the kind they advertise on the radio with "NO MONEY DOWN, NO PAYMENTS UNTIL JUNE!"  The longer you wait to fix the complexity, the more expensive it gets to fix.  Complexity gets added to complexity.  Like financial debt, technical debt compounds.

Your policy should be to maintain a constant state of low complexity except in exceptional cases, and then only with a plan to go back and fix the anomalous complexity immediately after the market advantage of the premature release has been enjoyed.


In the diagram above, "A," "B," "C,"and "D" represent different spending levels to get the same software written on behalf of your company.

A:  represents the money you are budgeting for total cost of ownership of your software today, which calls for paying no attention whatsoever to the financial risks inherent in your increasingly complex code base.

B:  represents the money you could be budgeting and purposefully spending to to avoid expensive maintenance and security risks.

C: represents the money you are spending today.  Technical debt isn't hypothetical.  You are spending 17-25% more than you should be on BAU software support, and that is a conservative estimate.  That's every single day.

D: represents the money you think your developers want you to spend on making the code "elegant."  And maybe they do.  But while you are busy trying to do "A" instead of "D," you have lost track of the opportunity to go from "C" to "B."

It's not just you.  In 2011, Andy Kyte at Gartner estimated that 60 years of accumulated technical debt has created a world-wide ticking financial time bomb worth $500 billion in 2010, with the potential to rise to $1 trillion by 2015.  What's worse is that every day, companies are spending billions just to keep maintaining this mess.  This isn't a "hypothetical" -- it is a cost to your business right now.  Can you really afford this?  Get a jump on your competition by systematically finding and eliminating technical debt the way you would any other serious organizational risk.

Comments

  1. Thank you for another great article. Where else could anyone get that kind of information in such a perfect way of writing? I have a presentation next week, and I am on the look for such information.
    Software Company Lucknow

    ReplyDelete
  2. Glad to help, John! I was talking with a dev friend, and I realized that the devs out there know that tech debt is bad, and how to fix it, but not how to describe it in terms of money. Imagine if CFOs knew they could reduce BAU expenditures by 17% by increasing code quality!? Cheers! Good luck with your presentation!

    ReplyDelete
  3. Here's an awesome recent post by Ron Jeffries about technical debt as well: http://tech.groups.yahoo.com/group/scrumdevelopment/message/55626

    ReplyDelete

Post a Comment

Popular posts from this blog

How Do You Vote Someone Off of Your Agile Team?

One of the conundrums of agile conversion is that although you are ordered by management to "self-organize," you don't get to pick your own team.  You may not have pictured it this way, but your agile team members are going to be the same people you worked with before, when you were all doing waterfall!   I know I wasn't picturing it that way for my first agile team, so I thought I should warn you.  (I thought I was going to get between six and eight original Agile Manifesto signers.  That didn't happen.). Why "warn" you (as opposed to "reassure" you, say)?  Because the agile process is going to reveal every wart, mole, quirk, goiter, and flatulence issue on the team within a few hours.  In the old days, you could all be eccentric or even unpleasant in your own cube, communicating only by document, wiki, email, and, in extreme situations, by phone.  Now you are suddenly forced to interact in real time, perhaps in person, with written messag

A Corporate Agile 10-point Checklist

I'm pretty sure my few remaining friends in the "small, collocated team agile" community are going to desert me after this, but I actually have a checklist of 10 things to think about if you're a product owner at a big company thinking of trying out some agile today.  Some of these might even apply to you if you're in a smaller place.  So at the risk of inciting an anti-checklist riot (I'm sorry, Pez!), I am putting this out there in case it is helpful to someone else. From http://www.yogawithjohn.com/tag/yoga-class/ Here's what you should think about: 1.        Your staffing pattern.  A full agile project requires that you have the full team engaged for the whole duration of the project at the right ratios.  So as you provision the project, check to see whether you can arrange this staffing pattern.  If not, you will encounter risks because of missing people.  Concretely it means that: a.        You need your user experience people (if a

Your Agile Transformation Needs to Start with a Quiet Phase

From a really great blog post on agile adoption:  http://smoovejazz.wordpress.com/2011/02/16/an-agile-approach-for-adopting-agile-practices/ I've observed some different agile transformation patterns, and maybe you have too: Just Do Standups   (Shoot, then Aim):   some people feel that since you're "agile," you should just start doing stuff, like daily standups, and then build more of the the plan as you go.  Find a team and start doing some agile with them!  Start a revolution one practice at a time, one team at a time. Pros:   you're very busy from the start. Cons:   what exactly are you doing and why? KPI-Driven Change (Aim, then Shoot) : some people who have worked in large corporations for a while will tell you that to get the respect of the people, you need to start with a plan, support the plan with awesome printed and online collateral.  Then you "kick off," tell teams what to do, and measure them using "Key Productivity Indica