Software engineers are notoriously good at underestimating how long a task will take. I’ve noticed in our own team, this topic has been brought up too many times recently. In this post, I’m going to dive into why estimates are useful, why software engineers may have this tendency to underestimate as well as a way to improve them.
Why do we need estimates?
- Planning the team’s roadmap – Knowing how long we’re going to take will give us a rough idea of when we’ll be free to work on the next big thing. It also helps plan, roughly, what we could deliver for our customers in the next FY.
- Coordinate with other parties in the organisation – If you’re working on a project that involves other departments that may not be as agile as the engineering and product teams, estimating work can help align when they can help out on your project. For e.g Deciding which sprint the can the copywriters and UX designers can start the work for your project. It also works on a larger scale when the marketing team can run its new campaign for your product. I’ve worked in both of these scenarios.
If estimates are so important to our industry and we’re quite experienced at making them, why do we get them so wrong?
So, why are the estimates always off?
Here are a couple of hunches
- Engineers are an optimistic bunch – There’s nothing wrong in being optimistic. At times it’s not always engineers thinking we’re super fast at getting things done, it could just be our ignorance to things not going to plan e.g. build servers are down, there’s an outage in the application platform or even a couple of people in the team could be sick during the week. Depending on the organisation, at times we don’t factor in our impromptu meetings and actually deploying our changes to production. Maybe we just don’t factor in the time we spend in extracurricular work activities such as hiring.
- We don’t break tasks down enough – Having super large tasks can make it difficult to track progress and guess how long it will take. “What about just making subtasks?”. Subtasks don’t actually solve the problem, in fact, it could just add a layer of unnecessary abstraction (or overhead) between the larger project and the actual work. We need to break work done into pieces can be accomplished in a day or less.
So these ideas might just be a start, now what method can we use to actually implement this?
First Principles
The idea of first principles is to break something down into the smallest fundamental truths as you can and reason up from there.
Here’s Elon Musk doing a better job at explaining it than me.
We can apply a similar bottom-up approach in making an estimate.
Start by taking a look at a piece of work that needs to be done and start breaking it down into pieces that can be developed, tested and even deployed individually. Once you’re at the smallest possible piece, then try to add an estimate. If you feel that your degree of confidence is still too low, break it down further deployment time (which you can derive from your CICD systems), testing time and development time.
To make this process easier, try and whiteboard it with your team so it’s much easier for everyone to visualise all the components involved.
This approach does have its limitations, it’s not easy to plan a large project this way. It isn’t meant to be easy either, but taking the time to make your estimate more accurate could pay dividends.
Why not have a go at this exercise in your next sprint planning?