Be thoughtful when measuring Technical Debt with Sonar

Technical Debt (TD) is the gap between perfectly developing software and the reality (ship date, skills of the engineers, tools available, working environment). You get it, when you take shortcuts that fall short on good practices. As in finance, not all debt is bad. Take a debt to buy a house is mostly a good thing, buy something luxury with your credit card knowing your paycheck will not cover it, is stupid [1].
In finance, measuring debt is easy (principal of debt, interest). But in software development? There are code measurement tools out there like Sonar, which try to measure TD. They are pretty good in measuring the code quality (over time), test coverage, code duplication, debt hot-spots (classes, packages, sprints, etc). They also come up with a number: your TD is $10,000. But: is this a good measurement? When should we start paying back what?

I think, this measurement is as good as the lines of code (LoC) measurement for project complexity. It gives you an idea what’s going on. If your project has 50 million LoC, it is probably very complex. If your project has 5000 LoC it is probably not so complex. But in detail, this measurement falls short, because it depends strongly on developer skills and programming languages (in fact, copy&paste developers love it). You cannot really say that two 50000 LoC are equally complex just because they have the same amount of LoC.

Let’s come back to TD. If Sonar tells you, your debt is $50 million, than your project has very likely a bigger problem than a project with $5000 debt. What you cannot measure is the design/architecture rationale: did we choose the right pattern or style? Maybe not, but we implemented it just perfect. Did we choose the right technology for our problem? Maybe not, because we don’t know what’s out there to solve the problem best or we simply do not have the skills to use it right. After years we may have obsolescent libraries, frameworks or platforms, which should be replaced. Tools can hardly measure that. In these cases we have to trust the key technical people.

It also doesn’t tell us, what debt to pay back first or pay back TD at all. Unlike financial debt, you don’t need to pay back TD in any case. Strategic design teaches us, that there is no need to have a 100% perfect system. The parts which implement the main business cases may better be perfect. How we deal with other, less important parts of the system, is negotiable depending on our guesses how often they will be changed in the future (maybe never). Eventually the business rules our decisions if we want to do [2]

• debt repayment: refactor or replace the code/framework/platform which gives you a headache
• debt conversion: replace the current solution with a ‘good, but not perfect’ solution. The new solution has a lower interest rate. Do that if a perfect solution is too expensive to build
• just pay the interest: live with the code, because refactoring is more expensive than work with the not-quite-right code

Using Sonar to keep an eye on TD is a good idea, but it is just a nice little helper tool. Developers have to think about the implication of the measurement results and should talk among themselves and/or with the business people how to deal best with it. A measurement result cannot tell us, what we should repay, convert or just pay the interest. It also cannot tell us (yet) what refactorings are important and which one’s aren’t.

[1] Eric Allman, Managing Technical Debt, ACM Queue 2012

[2] Frank Buschmann, To Pay or Not to Pay Technical Debt, IEEE Software 2011