Scope vs. Effort

When estimating and executing a project, it is important to have a good view on effort and scope. In order to do so you need to have a metric that you can use to measure these. For effort it is obvious, man days, man hours, man weeks will be your unit. Often, the same units are used for scope as well. In Earned Value Analysis you have something called a budgeted man-day (or budgeted dollar) which is used to define both scope as estimated effort. Your actual man-day’s or dollars will then tell you how much you really had to spend in order to achieve a budgeted manday or dollar. In this setup, estimated effort and scope are treated as one. Personally I find that confusing, since the same unit of measurement is used for two entities. Another drawback is that you are expected to come up with pretty precise man-day estimates even for the smallest items. That means a lot of estimating to be done up front, and the relevance and accuracy of man-day estimates for small items a long time before they are implemented is questionable.

Of course this one of the reasons why the Agile community came up with the concept of story points. I must admit I needed a while before I learned to appreciate the story point. Initially I found it too abstract, and I had difficulty in relating it to budget. My reasoning went as follows: It is nice to have a project estimate in story points, but you can’t go to your project sponsor or CFO and tell him that this project is going to cost 750 story points. Senior stakeholders need a dollar figure in order for them to calculate a ROI on the project to see if it makes sense. So at the end of the day you still need to convert your story point estimate to man-days and dollars, and if that is the case, then why bother with story point estimating at all? It just seemed making estimating unnecessarily complex.  So up until a few years ago, I did all my estimates in man-days and use Earned Value Analysis to track progress.

But more recently I realized that the good thing about story points is that it can be used as a measure of scope, independent of effort. The two are related of course through the concept of velocity (story points per sprint), but they are separate entities that have their own measurement unit and can be estimated separately. This means that a team can have discussions on the contents of a user story or epic without thinking about the effort. They can decide on whether user story A is larger or smaller than user story B and as a result they assign more or less story points. There are several ways of doing this (planning poker and comparing with a baseline user story, or sorting the user stories by size and give the smallest one a value of 1 and then go up the Fibonacci series). Once all user stories and epics are estimated in story points at the beginning of the project, and many functional and technical issues have been addressed during the estimation process, putting a man day figure on the project as a whole becomes a lot easier. I explicitly consider the story point estimate as a measure of scope, because I can then also track it separately from effort. As experience learns, there are cheap story points and expensive story points. Over the whole project you will average out a certain velocity, but some user stories that have a low estimate in story points will turn out to be expensive in actual effort, and some large user stories will come in pretty cheap. That’s just a consequence of not knowing everything up front and unforeseen events that impact the actual effort. In other words, you will tend to overestimate some user stories and underestimate some others, and you will encounter team members falling sick or leaving or on the contrary maybe feeling more productive than usually. This means that the relationship between scope size and actual effort is a statistical one, not a deterministic one. On average you will achieve a certain velocity but for an individual user story you may either achieve that velocity, or go faster or slower. That will depend on the things you didn’t know when estimating the story and circumstances you encounter while you are implementing it, both of which you don’t have under control at the start of the project.

In Earned Value Analysis, this would come up as some budgeted man-days being cheaper in actual man-days than other budgeted man-days. Again, I just find that confusing. I prefer to work in story points nowadays. If we add a new user story, we only consider its size relative to the ones we already have on the backlog. If we are not entirely sure, we assign more points to incorporate the risk. We don’t bother with getting a precise man-day estimate. After adding a new story, our scope has increased with some points and our project burn-up chart will tell us if the total scope is still feasible within the allocated budget and schedule or that we have to remove something.

All fine and well, but as I said previously, CFO’s don’t care about story points when the time is there to assess projects on their potential ROI. So how do we go from story points to man-days and dollars? If you have an existing team with an established velocity this is straightforward. Just divide the amount of story points they estimated through their velocity and you end up with the amount of sprints. From there it is easy to get to man-days and then via their daily rates to dollars. If you have a newly composed team with still unknown velocity, you can approach this in several ways.

The first is to let the team establish  an initial value for the velocity. Let them take a few user stories again, of different sizes and try to estimate these in man-days. Then compare the results to the story points of those user stories. Continue with this until the man-day estimates for these stories are more or less in line with the story point estimates i.e. a 1 point story results in an X days estimate, a 5 point story results in more or less a 5X days estimate. That should give you an indication of story points per man-day. That you can translate to an initial velocity and total project man-days and dollars.

Another approach is from a resourcing perspective. You can ask the team the question, how much time would this entire project take us, and which team composition would we need for it, based on the information we learned estimating the initial user stories and epics? The result will be a man-day estimate for the entire project, and you can work your way back to an initial velocity.

Or you can let them plan the first sprint based on what they think is realistic and take that as your initial velocity. Again just divide the amount of story points through this initial velocity and from there go to man-days and dollars.