How To Build an Agile Executive Dashboard

You get a lot of woo-woo hand-waving when you ask a typical large-scale organizational change manager what they would put on the executive dashboard to show that their program is effective.

"That's going to vary a lot from one company to the next," they may hedge.  Or "it depends" is a popular answer, also provided in long form: "it depends on what your executives want to see."  Right.  Like, executives who like Baroque Art want to see more Rubens.  That's helpful.  Not.

A Rubens work from wikipedia.  You were expecting maybe an unclothed zaftig Venus?

The dashboard is often "delayed," sometimes delayed until after the change management is over!  Why?

One reason a change manager may have for hedging on the topic of the dashboard is that change is difficult.  A truthful dashboard is likely to show the standard change curve, reflecting the morale of the change-ees:

http://rule-of-thumb.net/2008/09/26/the-change-curve/

Change itself is difficult, prompting the Rule of Thumb blogger to compare organizational change to Elizabeth Kubler Ross's Five Stages of Grief.  Okay, fine, a respectful moment of silence.  But now get over yourself.  You can and should dashboard, and just make a pinky pact before you start that you won't lose heart until you're 3-6 months into the experience.  Plus "morale" should only be one of the dimensions you track.

Here's my suggestion.  In addition to tracking your feelings, track how things are going in the dimensions of your business which caused you to be willing to change in the first place.  Here's what your dashboard could look like, with no special tools beyond coaches who are paying attention, plus a spreadsheet.  (I don't have Excel on my laptop, so this is a Google spreadsheet, as it happens).  As is so often the case, "red" is bad, and "green" is good.  I think if you click on the chart you can see it bigger:

A dashboard from a program that had several projects deliver to production after 6 weeks.

The columns from left to right are:

1. Week number
2. Agility (as measured by you and your team)
3. Quality (ditto)
4. Speed to market (ditto)
5. Transparency (ditto)
6. Customer morale (likely measured by survey of some kind)
7. Team morale (ditto)
8. Speed to Risk Mitigation (how quickly does the team identify risks, escalate them, and have them mitigated by executives?  as measured by you and your team)

There are books written about how to measure each of these things.  "What is Agile," alone, is a matter for endless debate, if you insist on having the whole world agree.  Indeed, when you hear about agile metrics, some agile coaches never get beyond this one metric.  The coaching dashboard to show "progress of the agile transformation" ends up being columns like "how many teams have a backlog?" "how many teams do standups?"  "how many teams have automated testing," and so on.

Please--do your executives care?  No, they do not.  Normal executives do not want to be handing out t-shirts with "Hi, I'm AGILE!" on them after a team has done its eighth backlog grooming.  What executives actually care about is delivery.  Did you deliver software on time?  Was it early?  Was it earlier than before?  What was the quality?  What will be the total cost of ownership?  Your little agile fad means nothing if it doesn't give you back something that your executives value.

But Elena, you say, how are you going to measure these things then?  If it's not a number, it's "anecdotal," and that's not real to a numbers-oriented executive!

The answer is simple, and it can give you and your program a useful executive dashboard by tomorrow morning at Start of Business (SOB).  Two steps:

1.  Short term, aggregate facts.  Don't wait until you can get numeric measures.  For the first six months of your agile program, define agile in a way that makes sense in your context, and then collect facts about the running program.  Not measures--but also not mere anecdotes.  Find verifiable facts as you go and report them.

• If your customers said at the end of your planning game that this was way better than waterfall requirements, and agree you can quote them on it, that's a fact you record in your "fact spreadsheet," which is the detail that underlies the executive dashboard.  
• If you are able to deploy new features into production after 6 weeks, when normally it would have taken 18 months to roll those features into a giant release, that's a fact.
• After you have three or more substantive facts to support the hypothesis that things are better in a certain area, you mark the dashboard "Better" for that measure, and direct your executives to the fact spreadsheet for substantiation.  If you have three or more facts in the other direction, like the whole team spends a whole week arguing about the test strategy, then you can move down one level, from "Better" to "Same" or from "Same" to "Worse."  Again, the facts spreadsheet serves as the evidence, and it should accompany the dashboard as a tab or an attachment.

Excerpt from Executive Dashboard "supporting fact sheet"

2.  Longer term, introduce measures. As you roll your program forward, I recommend that your definition of each column becomes more rigorous, and the values in the column become more specific than "same," "worse," or "better."  You want to be able to show and measure how quickly the money your company spent on coaching, training, books, and brightly colored Sharpies paid itself back monetarily.

• Use your agile project management software to measure how many projects are keeping track of themselves on a virtual card wall.  
• Analyze your code base for better quality using tools like Sonar.
• Factually track how frequently you are now moving new software into production, or into a pre-production environment.
• Survey your customers--are they happier?  What percentage?
• Survey your team--how are things going there?  Using what numbers?  Maybe you can start measuring morale in terms of reduced numbers of people leaving the team, if you have enough data to prove it.

Measures are your friend.  But don't let the seeming perfection of numerically measured progress obviate your responsibility to report using a dashboard to your executives right from the first day you hit the ground.

Dashboarding is simple.  The choice to make the data under the dashboard more complex is a separate business decision which requires its own costs and justifications.  Don't wait, and if you're a company getting help with your transformation, don't settle for consultants who wave you away and say it's going to take years for you to see results, "once we have all the widgets up and running."  Keep your program results factually charted from the in a Big Visible Way right from the beginning.  Your executives will be grateful, and you will retain your job log another triumph!  Tally ho!