User Stories as SRBs (or, "Stories Aren't a Cargo Inventory")

If the Agile Founding Fathers had written a Manifesto about US NASA Space Shuttle trips to the International Space Station, it might have read, in part, "we have come to value the safe delivery of astronauts, space station parts, and experimental moss, gerbils, and other stuff to the space station over creating a dignified-looking Saturn 5 rocket."

http://www.popsci.com/files/imagecache/article_image_large/articles/Endeavor%20Docking%20525.jpg

As the years passed, if we had all found ourselves employed at NASA in the shuttle program rather than in the IT organizations in small businesses and big insurance companies, our artifacts, processes, people, and procedures might have played the following roles:

How we think user stories fit into software development, as described by the ISS analogy:

• The role of "the shuttle" would have been played by our overall defined software development lifecycle (SDLC) -- the process supports delivery of the payload.
• The role of "the cargo" (humans, gerbils, microbes, and all) would have been played by the working software--after all, that is what the team delivers.

Would there have been a role for agile user stories in the analogy?  Oh, shoot, wait, yes.

• The role of "the inventory of the cargo" would have been played by the user stories.  We would want to keep keep them small and testable.  And when each payload reached the ISS, we would have had a tester mark off that all of the stuff in the cargo hold was correctly offloaded into cubbyholes on the shuttle.

Because, you know what, if you're busy delivering things to a space station, routinely risking the lives of astronauts and school teachers, only a bean counter is interested in the cargo inventory.  (Plus, of course, the federal regulators who watch the bean counters). 

If this casting is the right one for the analogy, it's no wonder people don't understand the actual role user stories play in a software delivery project.  And yet, to all intents and purposes, it appears that "inventory of the payload" is exactly what we think we're doing when we write them.  (If you're new to agile, please refer to this excellent Kelly Waters summary for how to write a good user story, to catch up.) 

Absolutely no criticism of Kelly here--we should absolutely do everything he says to do--but if we think of our story writing as being analogous to building an inventory, we are likely to make some avoidable and big mistakes.  (Plus we will think our job is about as exciting as complying with the new directive about the TPS Report Cover Sheet in the Office Space movie.)   Are we with Scott Ambler when says, "user stories are very slim and high-level requirements artifacts?"

You know what?  No!  No, user stories are not inventories of deliverables.  And not just because we're holding a grudge that Ambler always says that in an agile project, "B.A.," stands for "Band Aid."  Stories are not "slim and very high-level requirements artifacts."  They are not even FAT high-level requirements artifacts, even when they are written down in more than six words to accommodate colleagues who aren't in the same room with us when we have conversations.  We've gotten the analogy all wrong.  Let's try this again.

How user stories ACTUALLY fit into software development, as described by the ISS analogy:

• The role of "the shuttle" is played by our overall defined software development lifecycle (SDLC) -- the process supports delivery of the payload.
• The role of "the cargo" (humans, gerbils, microbes, and all) is played by the working software, plus a well-oiled continuous delivery system, plus a well-architected regression test suite which is mostly automated, plus whatever written, video, or Viso documentation and training materials are needed to support the system and test suite at lowest possible total cost of ownership (TCO) for as long as the software is use--after all, that is what the team delivers.
• The role of "the solid rocket booster (SRB)" is played by the user stories. User stories are essential propulsive vehicles for an agile project which must be discarded quickly and efficiently once they have been used.

NASA explains SSSRBs as follows:

The Solid Rocket Boosters (SRBs) operate in parallel with the main engines for the first two minutes of flight to provide the additional thrust needed for the Orbiter to escape the gravitational pull of the Earth. At an altitude of approximately 45 km (24 nautical miles), the boosters separate from the orbiter/external tank, descend on parachutes, and land in the Atlantic Ocean (+ View Video: SRB Processing). They are recovered by ships, returned to land, and refurbished for reuse. The boosters also assist in guiding the entire vehicle during initial ascent. Thrust of both boosters is equal to 5,300,000 lbs. 

What does this have to do with user stories?  Everything.  Here's a state diagram showing the impact of the Space Shuttle and its SRBs on the astronauts:

Before:  astronauts pose in orange suits while located on earth.  Colorful, but revenue negative.

During Shuttle Trip:  shuttle conveys astronauts to the space station.  SSRBs fall away.

After:  astronauts are on the ISS working, doing experiments, and finding things out for humanity.

When NASA considers the trip, they look at "what did it cost to convey the astronauts to the ISS and back safely" and "what is the net impact to humanity for them having made the trip."  Somebody at NASA is probably interested in the secondary question of "did one of the gerbils get lost in transit," particularly if gerbils in space are federally regulated in some way, but that gerbil is unlikely to come up in a Senate hearing or a history book.

Now let's look at a software project.

Before:  the business spends a certain amount on its operations, and takes in a certain amount of income in a given time period. 

During the Project:  new software is created which transforms business operations

After:  the business is is making money more efficiently.  In fact, even after paying for the project, the business has higher revenue overall.

Just as solid state rocket boosters used to fall away from the shuttle after they directed it towards the ISS, our agile user stories are going to be discarded once they have been used to enable the business to move from one steady-state operating model to another.  In fact, they become obsolete even before the new system is delivered.  Once a story is complete, it is over.

Don't believe me?  Let's look at an agile story card: 

As Juanita, a data entry clerk, I need to ensure that accurate bills are sent to customers as quickly as possible, so that we collect money more quickly, and maintain higher cash reserves.

Before the project started, Juanita had to keep track of bills to be sent on an Etch-a-Sketch, because the company didn't want to invest in paper or pens.  Once the project is over, Juanita will be using a system which automatically feeds her a queue of bills to be typed, so she will be able to process 127 times more bills every hour.  How will the queue work?  The story has no idea.  The functioning of the queue is documented in the actual implemented code, and especially in the end to end automated regression test suites which we use to verify we didn't break it when we last upgraded.  The story makes an impact on permanent things, but it is not a permanent thing in itself.

Aha! you say, "but all of the stories together add up to a functional description of the system."  Um, nope.  A story in isolation is not helpful, nor is it authoritative in any way as a specification for the functioning of the system.  Nor is any given story guaranteed to be correct forever.

• As a system is going through development, stories may describe a partial implementation of a user function, with subsequent stories describing a fuller implementation.  
• Business owners of a system may change their mind, so story 176 may completely contradict what the team implemented back in Story 14.
• One story may describe the function of one third of one panel in a multi-page web presentation layer.  Without a larger context, the story is a miserable place to look for an operational understanding of the business, either in the "before" state, or in the "after" state.

In fact, if you're looking for a functional description of the system which is guaranteed to be correct and complete at any point in time, you are best off looking at the end-to-end regression test suite.  Fundamentally, stories are ephemeral project artifacts valid only until they have been played.  

Bottom Line.

The reason I bring this whole thing up is that if we think correctly about stories, we can avoid a lot of the pitfalls for which agile software development is famous:

1.  Where do I find the big picture, if my whole software inventory is divided into small stories?  

Answer:  your inventory is NOT in the stories!  

• Your inventory is expressed by an evolving big-picture architecture diagram your developers are maintaining.  
• Your inventory is expressed by an evolving set of end-to-end functional tests being developed by your developers and testers hand-in-hand with the emerging software.  
• Your inventory is expressed in the pages of the business operating manuals which quantify cost savings when a series of actions is automated.

Agile stories promote the gradual development of a number of different "big picture" items, whether architectural or functional, but the stories do not contain those items.  The stories are just a series of team actions which result in impacts to the evolving set of working software within a given operational environment.  

Agile allows the big picture to emerge as you go, as opposed to being fully documented before you start, but make no mistake--in agile, every delivery professional on the team is contributing to the development of a "big picture" which is never tracked within any single story. 

2.  You're being mean!  I feel like I want to write something down, but agile calls for "working software," not "comprehensive documentation."  Can't I just go ahead and put this stuff in my story?

Answer:  Nope.  Do not put that stuff in your story!  Are you under the impression that in the future, someone is going to go trolling through all the stories to get information?  Not going to happen, especially if your stories are buried in a Jira/Greenhopper system which nobody can access.  (Sorry, Atlassian, I love you, but you are a pain in the patootie).

Agile stories are designed as "anti-documentation."  User stories describe a temporary resting point for some area of your system and its automated tests, with the expectation that someone is going to move the bar and change that area of the system again in a week or a month.  The wise agilist keeps stories tiny not just because the Agile Founding Fathers were too macho to hire professional analysts, but because even modern and enlightened delivery people who are alert to issues around total cost of ownership follow a discipline which requires safe storage for any information you are going to need in the future.  Agilists feel big-picture information should be stored where it can be found by the right operational user of the system long after the project team has moved on to something else. 

3.  I'm going to set up my permanent test suite so that all tests trace back to the story which defined the need for those tests.  I'll load up the stories, and I'll see a 1:1 correspondence, just like I did with the FRD requirements in the olden days.

Answer:  please, stop.  You are hurting my head.  Don't even get me started about the olden days.

• If you have a test in your permanent regression test suite for every story, those tests will be riddled with contradictions--test 1 will attempt to confirm that some area of the system behaves exactly as specified in iteration 1, and that test will fail every single time in the future, once that area of the system is developed further.  
• Moreover, modern testers actually architect their tests to be built from reusable test case components.  Even if you avoided self-contradictory test scripts, are you seriously planning to build that much redundancy into the permanent regression test suite?  How will you ever know the impact of a requested change to this system five or ten years down the line?

Stories are little mini-bursts of energy directed at a working system, including its tests.  If you start to think of stories this way, you can stop trying to make them into tiny requirements documents, and focus on doing something useful.

Stirring Conclusion

Followers of Agile Lore will recall that the Agile Manifesto was signed by 17 men in 2001 at the Snowbird ski resort in Utah.  Here's a fun assertion from Wikipedia about the manufacturers of the US Space Shuttle Solid State Boosters:

The prime contractor for the SRBs and the manufacturer of the vital solid fuel rocket segments was the Thiokol Corporation of Brigham City, Utah.

Utah.  2001.  Mere coincidence?  I don't think so.