Estimating software project deadlines

Target dateLet me tell you. It’s hard.

There are managers who never learn why software project deadlines sometimes can’t be estimated with accuracy. There are programmers who never learn how to estimate deadlines with any kind of accuracy. How do we deal with this?

Having an accurate project deadline means schedules can be planned. It means the future can be glimpsed and foretold, albeit somewhat uncertainly. It means we are in control.

Real life doesn’t happen this way. Real world applications are infused with innovation, or they should be for any kind of improvement. This means new concepts, new ways of interacting, new ways of thinking about data, new ways of improving existing processes. New, new, new.

I’ve worked on both long-term huge projects and small-term mini projects. I’ve learned that there are many variables affecting project deadline estimation and the actual project completion date. Some of them are

  • skill of individual programmer
  • understanding of the project/business logic
  • affinity with other programmers on the team
  • difficulty of the task itself

I’m continually given small projects and asked to give an estimate on how long I’d take to complete them. My experience is that the smaller the task, the easier it is to estimate. The more standard the task, the easier it is to estimate.

This doesn’t mean you can’t innovate. On the contrary, it means you must innovate even more. It means breaking down any project into bite-sized portions that you can estimate with certainty. It means you must understand the project on many levels, from how the project fits in with existing projects to how a single line of code fits in.

And it’s hard. Understanding on so many levels require a lot of thinking. It takes some serious brain mojo to figure out that on the macro scale, data is entered and stored in a certain way, and that on the micro scale, a particular if condition has to be coded, even though the condition is never mentioned, implicitly or explicitly.

It involves asking a lot of what if’s. What if the user keyed in this value? What if the user keyed in this combination of values?

The naive manager never understands how a simple project with simple interfaces require so much time, because hidden within the project are assumptions that manager thinks are obvious. The unskilled programmer never learned to gauge his work, because there are assumptions hidden within the project and need to be explicitly coded.

When I’m asked for an estimate, what they’re really looking for is, “How many man days are required?” So far, my answers range anywhere from 5 to 35 man days. In case you’re wondering, 5 man days means 5 work days. It can be 5 people cooperating to finish the task in 1 day, or 1 person doing the task in 5 days. This method gives a projection of the deadline and the cost involved (in paying the programmers).

So how can you improve the accuracy of your estimates? Two factors, skill and standards. Your thinking and programming skills make it easy to translate business logic into code. Which gives you greater confidence in estimating. The more standard a task is visualised, the more previous knowledge can be leveraged. Which reduces new concepts and code, and give you greater confidence in estimating.

Ultimately, it is simply a matter of reducing errors and uncertainty in project estimation. I wonder if L-1 linear regression or least square approximation can be used for projecting future deadlines? I’d have to go through all my documentation to find out what my estimate was and the actual time taken was for any given project… It will require enough data points to sufficiently draw a line.

On that note, FogBugz uses Evidence-Based Scheduling, involving up to 200 data points. Instead of giving you when a project can complete, it gives you a probability of likelihood that the project can complete on a certain date. For example, there’s a 50% likelihood that your project can complete by next week. But if you can extend it another week, there’s a 80% likelihood that the project can complete.

I neither have the time and effort to do least square approximations, nor the software to do historic statistical analysis of previous estimations for future estimations. I do have my experience and programming skill, so basically I trust my gut.

So, how do you estimate your software project deadlines?

  1. Will Dwinnell

    Another possibility is fuzzy logic. Earl Cox describes a fuzzy solution to a similar problem, project risk assessment, in his book, “The Fuzzy Systems Handbook” (ISBN 0-12-194270-8). The solution he describes involves a collection of rules about project risk, such as:

    “if project.funding is Low then risk is increased”

    “if project.duration is Long then risk is increased”

    Terms used in these rules are given mathematical definitions and their consequents are combined using fuzzy logic to yield a final risk assessment.

  2. Vincent Tan

    Oh yeah, I’ve been through long duration projects… looked like there’s no end in sight. Very demoralising when it feels like nothing is accomplished. Risk is definitely increased.

    I learnt about fuzzy logic while doing research on game programming. The Earl Cox book is interesting. Thanks for the tip!

Comments are closed.