Estimating is Liberating

Right now after having just read the title you’re probably thinking I’ve gone mad. I mean, why would a developer actively promote the use of estimation? Surely I should be joining the ranks of the #NoEstimates crowd and advocate the abolishment of estimating as a technique?

Well, yes, and no. As plenty of other much wiser people than me have already pointed out, the #NoEstimates movement, like the #NoSql one before it, is not the black-and-white issue it first appears. However this blog post isn’t really about whether or not I think you should create estimates per-se but about some of the effects I’ve observed from teams performing it. Whether the means justifies the end (even in the short term) is for you to decide.

Establishing Trust

The first time I realised there was more to estimating than simply coming up with a figure that describes how long some piece of work will likely take was almost a decade ago. I was working as “just another developer” on a greenfield project that was using something approaching DSDM as a project management style.

We had started off really well delivering a walking skeleton and fleshing it out bit-by-bit based on what the project manager and team thought was a suitable order. There were many technical risks to overcome, not least due to the cancellation of some services we were dependent on.

After what seemed like a flying start things slowly went downhill. 6 months in the project manager (PM) and I had a quiet word [1] as they were concerned things seemed to be taking so much longer than earlier in the project. I suggested that we consider going back to estimating our work. What I had noticed is that the problems we were encountering were really just delays caused by not really thinking through the problems. Hence every day the work would be finished “tomorrow” which naturally caused the project manager to start losing faith in his team.

By forcing ourselves to have to come up with an idea of how long we would need to work on a feature we started breaking it down into much smaller chunks. Not only did this mean that we thought through more clearly what issues we might need to tackle but it also allowed us to trim any obvious fat and work in parallel where possible.

The result was an increase in trust again in the team by the project manager, but also by extension the customer and PM therefore had less “awkward” conversations too [2].

Knowledge Sharing

The next moment where I began to see the positive effects of estimation was when joining a team that adopted Planning Poker as a way of estimating the work for a sprint.

In the (not so) good old days, work was assigned to individuals and they were responsible for estimating and performing that work largely in isolation. Of course many of us would prefer to seek advice from others, but you were still essentially seen as responsible for it. As a corollary to that the number of people in the team who knew what was being worked on was therefore small. Even if you did have a whiff of what was happening you probably knew very little about the details unless you stuck your nose in [3].

This team worked in a similar fashion, but by opening up the planning session to the whole team everyone now had a better idea of what was going on. So, even if they weren’t actively going to be working on a feature they still had some input into it.

What the planning poker session did was bring everyone together so that they all felt included and therefore informed. Additionally by actively canvasing their opinion for an estimate on each and every feature their view was being taken into consideration. By providing an overly small or large estimate they would have a sure-fire means of having their opinion heard because the conversation tends to focus on understanding the outliers rather than the general consensus. It also allowed members of the team to proactively request involvement in something rather than finding out later it has already been given to someone else.

I think the team started to behave more like a collective and less like a bunch of individuals after adopting this practice.

Transparency

More recently I was involved in some consultancy at a company where I got to be a pure observer in someone else’s agile transformation. The units of work they were scheduling tended to be measured in weeks-to-months rather than hours-to-days.

I observed one planning meeting where someone had a task that was estimated at 4 weeks. I didn’t really know anything about their particular system but the task sounded pretty familiar and I was surprised that it was anything more than a week, especially as the developer was fairly experienced and it sounded like it was similar to work they had done before.

I later asked them to explain how they came up with their estimate and it transpired that buried inside were huge contingencies. In fact a key part of the task involved understanding how an API that no one had used before worked. In reality there was a known part for which a reasonably sound estimate could be given, but also a large part of it was unknown. Like many organisations it never acknowledged that aspects of software development are often unknown and, when faced with something we’ve never done before, we are still expected to be able to say how long it will take.

Consequently I got them to break the task down into smaller parts and present those estimates instead. Most notable was an upfront piece around understanding the 3rd party API – a technical spike. This very consciously did not have an estimate attached and it allowed us to explain to the stakeholders what spikes are and how & when to use them to explore the unknown.

This openness with the business made both them and the delivery team more comfortable. The business were now more in the loop about the bigger risks and could also see how they were being handled. Consequently they also now had the ability to learn cheaply (fail faster) by keeping the unknown work more tightly under control and avoid unexpected spiralling costs or delays.

The benefit for the delivery team was the recognition from the business there is stuff we just don’t know how to do. For us this is hugely liberating because we can now lay our cards firmly on the table instead of hiding behind them. Instead of worrying about how much to pad our estimate to ensure we have enough contingency to cover all the stuff we definitely don’t know about, we can instead split off that work and play it out up front as a time-boxed exercise [4].Instead of being sorry that we are going to be late, again, we have the opportunity to be praised for saving the business money.

Training Wheels

What all of these tales have in common is that the end product – the actual estimate is of little importance to the team. The whole #NoEstimates movement has plenty to say on whether they are useful or not in the end, but the by-product of the process of estimating certainly has some use as a teaching aid.

A mature (agile) team will already be able to break work down into smaller chunks, analyse the risks and prioritise it so that the most valuable is done first (or risks reduced). But an inexperienced team that has had little direct contact with its stakeholders may choose to go through this process as a way of gaining trust with the business.

In the beginning both sides may be at odds, both believing that the other side doesn’t really care about what is important to them. Estimation could be used as a technique that allows the technical side to “show it’s workings” to the other side, just as an exam student proves to the examiner that they didn’t just stumble upon the answer through luck.

As trust eventually grows and the joint understandings of “value” take shape, along with a display of continuous delivery of the business’s (ever changing) preferred features, the task of estimation falls away to leave those useful practices which always underpinned it. At this point the training wheels have come off and the team feels liberated from the tyranny of arbitrary deadlines.

 

[1] This is how process improvement used to take place (if at all) before retrospectives came along.

[2] Without direct involvement from the customer all communication was channelled through the project manager. Whilst well-meaning (to protect the team) this created more problems than it solved.

[3] I guess I’m a “busy-body” because I have always enjoyed sticking my nose in and finding out what others are up to, mostly to see if they’d like my help.

[4] The common alternative to missing your deadline and being held to it is to work longer hours and consequently lose morale. Either way the business eventually loses, not that they will always realise that.