Why Projects Are Late (It’s in the probabilities)

The book Gödel, Escher, Bach: An Eternal Golden Braid (Douglas Hofstadter, 1979) offered up Hofstadter’s Law: “It always takes longer than you expect, even when you take into account Hofstadter’s Law.”

Back in the 19th century, Vierordt’s Law claimed, roughly, that the longer a task will take, the more likely it is to be underestimated.

Why are projects so often late? I heard a PMP instructor – a seasoned project manager and mentor to others – say that despite years of project management experience, a project that’s only a few weeks late instead of months late is still a rare thrill.

In my experience, two factors account for an awful lot of project lateness, but – strangely – almost nobody mentions one of them. One is that people underestimate. The other is in the math of probability.

Underestimating

The commonly cited factor for project delays is the strong tendency to underestimate.

Research from 2007 (“Bias in memory predicts bias in estimation of future task duration“) found that “underestimation is most likely when a task is well learned.” That seems counter-intuitive, because you’d think that the better you know a task, the better you’d be at estimating the time needed. The study, however, found that people tend to underestimate how long something took in the past, and that makes them underestimate how long it’ll take in the future.

An effect I’ve observed is that people often estimate task length as if nothing will go wrong. To me, this is like my GPS unit that never accounts for traffic lights in its time estimates. Estimating as if everything will happen without delay means your estimates are probably low.

Similarly, there’s what I call the “MapQuest effect.” (Or fill in your favorite mapping tool.) This means estimating the main activity while ignoring all the lesser activities that go with it. If I get driving directions from my suburban home to a particular downtown DC office building, MapQuest tells me it’ll take 30 minutes. If I want to be at a meeting that starts at 2:00, what’s my chance for success if I plan to leave home at 1:30? Non-existent. Even without traffic problems, I’ve left out any possibility of last-minute delays in leaving the house, and I’ve left out the time needed to find parking, walk to the office building, reach the reception desk, and get to the meeting space.

Another underestimating effect I’ve observed is that people feel pressured into promising more than they can deliver. Even though “fast” and “good” are often not the same thing, people often feel compelled to give short estimates to show they can handle a task quickly. It’s almost as if they’re admitting to a shortcoming if they don’t make their estimate as short as possible, so they go with the shortest possible estimate instead of something more realistic.

Probabilities

The other factor, which hardly anybody considers, is simple probability.

Many people figure that to estimate the overall project duration, you just add up the tasks that make it up (along the critical path), and there’s your estimate. Turns out, the probabilities are against you.

Here’s a thought experiment. First, what would you say your on-time percentage is? That is, what percentage of your tasks are completed within the time you estimated? I’m talking about the task level, not the overall project completion estimate – the pieces you’d estimate individually before you come up with the grand timeline.

I’d say that most people are no higher than 50% at finishing tasks within their estimates. Unfortunately, if your ability to estimate task length is weak, then your ability to estimate your on-time percentage is probably weak too. Darn.

But anyway, suppose you credit yourself with a 90% on-time rate. Nine times out of ten, you complete a task within the time you thought it would take.

For the second step of this thought experiment: How many tasks would you typically string together sequentially as part of an overall effort? This might be the number of tasks on the critical path of a project plan, or it might be your sequence of activities trying to get out the door before a vacation. Suppose you pick eight steps for this exercise.

So now you’ve got eight steps, and you’re 90% likely to be on time for each of those steps. Maybe you estimated two hours each for the first four, and four hours each for the last four. That’s a total of 24 hours. What’s the probability the whole effort will need 24 hours of effort?

In other words, what’s the probability of hitting 90% eight times in a row? You can calculate that by pasting this formula into a Google search box: 0.9^8. The answer is 43%. That’s worse than 50/50.

In other words, even if you’re pretty good at estimating each of the eight tasks, this project is likely to be late.

Adding up the task estimates to find the total estimate is a bad bet.

What’s the Fix?

There’s an old tongue-in-cheek rule of thumb that says you should double the number and increase the units (24 hours becoming 48 days, for example). But that’s not very realistic or helpful.

PERT Three-Point Estimate

A commonly offered fix is the PERT three-point estimation technique: (optimistic + 4*likely + pessimistic) / 6. You’re making three estimates: the optimistic estimate for when nothing goes wrong, the pessimistic estimate for when a lot goes wrong, and the estimate you consider most likely.

As an example, say you’ve originally estimated a task at 2 hours, assuming everything goes pretty smoothly. That’s now your optimistic guess. Maybe your pessimistic guess is that you could spend an extra hour and a half (3.5 hours total) if you find a surprise need for a trip to the store before you’re ready to go. Maybe your likely estimate is that you fudge in an extra half hour on top of your optimistic estimate for things like looking for lost keys and herding the cats (your passengers). The PERT estimate is therefore (2 + 4*2.5 + 3.5)/6, or about 2.6 hours.

At its best, the PERT estimate is an intelligent fudge factor to keep you from always using optimistic estimates. It gets you to think about what could go wrong, or what could delay you.

At its worst, the PERT estimate won’t be very good either. If you’re no good at making one estimate for a task, you might be no good at making three estimates for the task. The optimistic estimate might be hard enough to come up with, but it’s harder still to estimate what’s likely, and it’s hard to know how pessimistic you should be in your pessimistic estimate.

There’s still an element of probability in the PERT method. The task could still take longer than the PERT estimate suggested. The best we can hope for is that the PERT estimate is more realistic than our originally optimistic estimate. It’ll never be a guarantee.

Task Data

Another possible fix is to track your actual time, so you can use historical data in your future estimates. This could get you past the experience bias that makes you underestimate past tasks.

That’s great if you can collect enough accurate data, and if your future efforts are comparable to your past efforts.

What if entering all that data is too tedious to do in a consistent and timely manner? Someone who enters their task data long after the fact might not be very accurate.

What if it’s hard to say how much time you really spent on a given task? If you operate in an environment that features lots of interruptions and distractions, knowing that you started a task at 9am one day and finished at 5pm the next day leaves a lot of leeway in estimating how much time you spent on one particular effort.

In other words, data’s great if you’ve got it, but it might not be complete, accurate, or relevant for the task you’re estimating now.

The Probability Fudge

If you’ve got a decent idea of your on-time rate, you could fudge your estimates up by dividing by your on-time percentage.

If you’re on time 80% of the time, make your estimate, then divide it by 0.8. Your estimate of 24 hours becomes 24/0.8 = 30 hours. If you’re on time 50% of the time, divide your estimate by 0.5. For those who don’t breathe math, that’s the same as doubling it. If your on-time rate is 50%, your estimate of 24 hours becomes 48 hours.

This too is a fudge factor, but it might help steer you to more realistic estimates.

My Best Advice: Set Expectations, and Update Them

Given that no method is perfect at fixing squishy time estimates, do you give up in despair? Do you let the delays happen, then whine about how it’s not your fault? Naaaah.

First, set realistic expectations for yourself. Recognize that your estimates won’t be perfect. Some tasks will go past their estimated durations.

Set realistic expectations for whoever will see the estimates. Talk about probabilities with them, even if you don’t have enough to data to gauge those probabilities precisely.

Importantly, update the expectations. “See it, say it” as they say in the DC subway system. If you see things heading toward delay, you’re generally better off by being up-front about it with whoever needs to know instead of waiting until the deadline to announce you’re going to be late.

Remember those eight tasks from our hypothetical example? At first, you had to get your 90% on-time rate eight times in a row. Finish one off, and now you need to hit 90% only seven times in a row. Then six, then five, and so on. Your accuracy in the overall project deadline should go up as you finish off the tasks that make it up.

Happy estimating!

Jim

The Improvement Paradox

One of the “tells” I look for among colleagues in the workplace is a person’s eagerness to improve: improving their skills and knowledge, improving the products and services they deliver, and improving the processes they follow. The paradox is that the best workers have the most focus on improvement, but the ones who need it the most usually have the least interest in improving.

Why would the ones who need it most want it least, and the ones who need it least want it most? In both cases, it’s how they got that way. The best became the best by working at it, by frequently looking for ways to improve. The worst got complacent, vane, or passive at some point, so they’ve seen no need to work on improving.

The Time Element

The time element makes the difference. The improvers have an eye on the future. They know that no matter what they’re offering now, it’ll become stale over time. Subject matter expertise is great, but only if you keep up with the subject matter. Product expertise is a great thing, but someday that product will become irrelevant. Your current services and processes might be great, but they won’t stay great unless they keep up with evolving circumstances.

The non-improvers don’t have an eye on the future. They have an eye on some moment in the past. Some years ago, they learned one use case or procedure, or one programming language, operating system, or application, and got stuck. Call them “point-in-time Luddites” – fixated on some past state, and unwilling to go beyond it.

Lifers

This improvement tell shows up when you look at lifers – people who’ve been with the company for many years, and who want to stay for many more. The difference between the best lifers and the worst is their focus on improvement, or lack thereof. The best lifers keep updating their skills, adapting their practices, and staying in touch with what’s going on in the organization and their field. They’re always on the lookout for a better way. Their understanding of the organization is wide and deep, and therefore highly valuable. The worst lifers latched onto a technology or practice years ago, and they’ve resisted doing anything different ever since. Their understanding of the organization and their field becomes increasingly narrow and shallow as they resist anything that smacks of change.

Promotions

The improvement tell shows up when opportunities for promotions arise. I’ve always viewed promotions more as rewards than incentives – rewards for those who’ve extended themselves (improvers), not incentives for those who haven’t extended themselves (non-improvers). I’ve known employees who refused to work toward any sort of improvement unless they got a promotion first. They’re the non-improvers. I suppose you could say they’ve done me a small favor: when someone refuses to go above and beyond without a promotion, my list of candidates for promotion has gotten shorter by one.

Lessons Learned

The improvement tell shows up in lessons-learned exercises. My standard lessons-learned agenda goes like this:

  1. Review of the facts: Let’s make sure everyone understands the facts of the situation.
  2. What went well: What should we do the same way the next time? What worked? What’s most important to keep just the way it is? Paradoxically perhaps, the improvers are best at noting what doesn’t need improving. Effective improvement means being selective, recognizing where it’s not needed. Otherwise, a focus on improvement becomes pathological when you think everything needs fixing equally.Non-improvers tend to start with an assumption that whatever they were doing, that’s what went well. They’re less open to the idea that they’d need to change anything, so their assessments of what’s truly worth preserving will be distorted.

    By the way, in my lessons-learned exercises, I make a point of putting “What went well” before “What needs improvement” because it’s all too easy for a lessons-learned exercise to devolve into griping and finger-pointing. Starting with the positives generally puts people in a less complaining mood, and it softens the blow if there are some difficult improvements to discuss.

  3. What needs improvement: What should we do differently next time? What didn’t work as well as it should have? What’s the biggest improvement we can make for next time? Improvers are the better contributors here too, for the obvious reason that they care about making improvements, but also because they’re more collaborative about change. If the non-improvers see any room for improvement, it’ll be in someone else’s work. That changes the dynamic from “what can we do” to “I’m fine, but let’s talk about what’s wrong with you” – blame games instead of constructive improvement.

Improvers at Any Level

The improvement tell shows up at all skill levels and all staff levels. The seasoned pro who got to a certain level and then stopped has turned into a non-improver. The junior staff member who has little in the way of skills and knowledge now, but who brings a fresh perspective and who wants to develop skills and knowledge, is a valuable improver. Maybe today, the seasoned pro still fills an important role, and the junior staff person isn’t a major contributor. Over time, however, the junior improver will become more valuable while the senior non-improver will become less valuable.

Certainly, it can go the other way too – the seasoned pro who never goes stale, or the junior staff person who rarely makes an effort to do better. That’s the point: Any skill level can be improvement-focused or not.

I Like the Improvers

For these reasons and others, the colleagues I value most are the ones who have a healthy focus on making improvements.

Is This Project Worth Doing? The Fit Matrix

I have a tool I use as a quick first check on any effort that someone is proposing. I call it the Fit Matrix.

Someone wants to see a new service, or a new piece of software. Maybe someone wants to bring in a vendor for an overview of a product line. Or the question might be whether we should keep doing something we’re already doing.

I’m writing this from an IT management perspective, but the tool is easily generalized for other areas.

The Fit Matrix is a two-by-two matrix:

Business Fit
Low High
IT Fit High Solution Looking for a Problem Do It!
Low Don’t Do It! Problem Looking for a Solution

It’s intended for a quick assessment. It’s not a substitute for a feasibility study, a cost analysis, a risk analysis, or strategic planning. You’d use this as a quick look before you get to those more involved assessments.

Business Fit: Does it serve the organization’s directions?

Business Fit looks outside the IT group to match up the work with what the organization as a whole is up to. You’ll rate Business Fit as High or Low. Neither assessment guarantees that the work will or won’t be pursued. You’re just stacking it up against the organization’s known directions.

The proposed work has High Business Fit if:

  • It directly supports a documented organizational goal.
  • It directly supports an important business process within the organization.
  • There’s a target audience that is likely to want something along these lines.

Otherwise, the proposed work has Low Business Fit. It doesn’t support documented organizational goals or important processes, or we don’t really have an audience for it.

What about a gray middle? Maybe there’s a request people often make, but nothing in the organization’s plans call for it. I’d usually call that a Low Business Fit. For Business Fit, we’re not (yet) asking whether the idea would be popular. We’re asking whether we can match it up with known organizational directions.

IT Fit: Does it fit with IT’s directions?

This is the internal view – how well the work fits in with the current or planned IT environment. Just like Business Fit, IT Fit will be judged as High or Low, and neither assessment guarantees what we’ll do with it.

The proposed work has High IT Fit if:

  • We have the technology to do it, or we’re already planning to acquire the technology.
  • We have the know-how, or we’re planning to get it. For this purpose, I’m not making a distinction between insourced work, outsourced work, or other approaches. The question is whether we do or don’t have a reasonable expectation that the necessary expertise will be in place, wherever and however we might come by it.
  • We expect it will integrate well with the existing or planned IT environment.

Otherwise, the work has Low IT Fit. We don’t have the technology, we can’t get the expertise, or we don’t see how it’ll work well with our intended environment. We’re not ready, not willing, or not able to do it.

The Four Outcomes, and What to Do With Them

Do It! (High Business Fit, High IT Fit): This is the ideal outcome. The organization needs it, and you can deliver it. You’ve got a case for taking a closer look. You’d still need to do appropriate levels of investigation, planning, and prioritization, but for now, you can say it’s a good candidate for such consideration.

Problem Looking for a Solution (High Business Fit, Low IT Fit): This is important, because the organization has a need, but somehow the idea in question is not a good fit with what you’re already doing. Either you need a better idea that will fit your IT directions, or you need to modify your IT directions to accommodate the solution. Or you don’t need a new solution because the current solutions are fine (for now).

Solution Looking for a Problem (Low Business Fit, High IT Fit): The idea seems cool, from a technology perspective, but you can’t tie it to organizational priorities. A Solution Looking for a Problem is less important than a Problem Looking for a Solution. Either you reject the idea altogether, or you allow it as an experimental side project with the hope that it may yet prove useful, at least as a learning exercise. But you don’t throw lots of resources at it.

Don’t Do It! (Low Business Fit, Low IT Fit): This idea isn’t a good fit for the organization or for IT. It’s not a good candidate for further pursuit, even as an experimental project. This situation will come up, and it’s not a personal failure for the person who suggested it. A good-faith suggestion that turns out not to be a good fit is still a Good Thing. You now have the opportunity to increase awareness of what is or isn’t a good fit, and you’ve handled the suggestion before much time was spent on it. If you want to send the message that you welcome suggestions, treat the person well even though the idea was rejected this time. Next time, they’ll be a little smarter about what’s going to fly.

What if you don’t have documented directions for the organization, or for IT?

You might find yourself in an organization that has no clear, documented directions. You don’t know what to match against. Or you might have inherited an IT group that hasn’t spelled out its own directions, or the directions have little to do with what the organization as a whole wants.

Make your best guesses, and use the Fit Matrix anyway. Give people something to react to. Maybe you’ll shake something out of the tree if you describe why you think something is or isn’t a good fit. You have a chance to learn something, and get greater clarity about the needed directions.

Foster Awareness

The Fit Matrix gives you your elevator pitch for any work you’ve assessed – why you are, or aren’t, pursuing a particular area of effort. The quick assessment of the Fit Matrix gives you a quick way to foster awareness of your activities and directions. Give it a try!