Developers Can Be Their Own Worst Enemy

Generally speaking I reckon we programmers are an optimistic bunch. By-and-large we also want to do a good job and take great delight in seeing our work unleashed on our intended audience. I also believe we do our best to understand the constraints of the business and work within them when delivering our solution. Like any normal person we also want to be acknowledged and respected. But we also like to tinker.

A New Age of Transparency

In past times, when the mentality was more Waterfall-esque, it was much harder to get late ideas in, and so the consequence was that developers might shoehorn one feature in on the back of another. This saved on paperwork and also gave us a warm fuzzy feeling that we were in control of the product’s destiny.

But it’s no longer like that, at least it’s been a long time since I’ve worked in anywhere near such a rigid framework. The modern development cycle means that the next sprint planning meeting is only a week or two away. The product owner, who should now feel that they have a good idea what is going on day-to-day, is far more amenable to localised plan adjustments [1] so the notion of widening the scope slightly or even squeezing another story in earlier is only a conversation away. Even if it doesn’t get the go-ahead immediately there’s a backlog that anyone can add features to and all the stakeholders (which includes the development team) can provide input to determine the ordering.

Maybe though that’s just not enough for some people. There are developers that still feel the need to act on highly speculative requirements. They are so very sure that we are going to need to do “X” that they will start coding the feature up straight away. After all, they have the engine open on the workbench so it’s got to be more efficient this way, right?

At Number 5 in his “11 Agile Myths and 2 Truths” Allan Kelly has:

Developers get to do what they like

The truth hurts, and much as I wish it wasn’t this way, Allan Kelly is right, we cannot always be trusted. The years have taken their toll and until we can show that we are able to be grown-ups we must accept that we still need to hold somebody’s hand all the way down to the sweet shop.

Continuous Learning

After 20 years writing software professionally you would probably have hoped that I would know what I’m doing. Perhaps if I took a different path than the Journeyman route I could easily delude myself that I now know the one true way and therefore I will know how every solution is going to end up. As such, even though I might not know the exact problem domain surely I must know all the other stuff like architecture, design, testing, debugging, logging, monitoring, documenting, etc.?

This line of reasoning is probably what gets us into hot water. In the back of our mind we believe that because we’ve needed X before that we should have X again. Not only that, but we should also do it the same way we did X last time. And why shouldn’t we, after all isn’t that what they’re paying us for – our experience?

While this might be true, they are also probably paying for us to not reinvent the wheel and to make efficiency gains by adopting modern tooling and practices. Our experience of knowing how to solve problems and where to look for help is probably more valuable than the specific solution knowledge we have. For example the very brief time I spent writing Eiffel 20 years ago has never been of direct use, but the knowledge of Design by Contract has been generally useful. Less than a decade ago I was still writing manual “for” loops and now its hardly ever. If such a basic programming construct can change why wouldn’t I assume everything else will too, eventually?

Do or Do Not

To be clear this is not about being given the solution on a plate and simply following it, definitely not. As technologists the solution domain is our space to shape and mould as we see fit. We are masters of that domain, but everything we do in it must ultimately be to serve the greater good of the business. We have plenty of room to make our mark without trying to play the hero or getting “one up” on those “technically illiterate business people”.

It’s also not about stifling innovation either. Good ideas need to come from all areas of the organisation, not just the business side. In fact having an appreciation of the technical constraints probably means we’ll filter out some of the more challenging ideas before presenting them on the grounds that we suspect they’ll never fly. This is equally undesirable because we don’t want to miss opportunities either.

No, what I’m talking about here is the half-baked features that you comes across when trying to implement another change. Sometimes it’s just a stub or bit of logic left as a placeholder for an as yet unimplemented feature. Or maybe it’s some behaviour that the author thought might be useful but didn’t rip out again. And guess what, there are no tests and the story it’s checked in against is for something else. Now you’re into software archaeology mode trying to work out how this relates to what you thought the code should be doing. There aren’t many developers I know who have the confidence to just rip stuff like this out, so it often lives for far longer than you’d hope.

The reason is that all these little bits of speculative features are just waste. For a start while you had gone off-piste the feature that should be being delivered is now delayed which means it’s costing money (i.e. it’s inventory) not generating income. When it interferes with the delivery of any other feature by becoming a distraction it once again causes delay. If someone keeps having to ask or work out why something lying around exists the cost will slowly mount up.

Have Faith in the Process

You’re probably thinking that I’m being somewhat overly melodramatic and in a way I am. However “death by a thousand cuts” starts with only one cut and continues growing. By not keeping it under control (or better yet removing it) the problem remains unchecked. When you’ve worked on a clean codebase where you don’t keep having to second guess what the code says and does you can get into a rhythm that allows new changes to be delivered really quickly. XP talks about having courage and a codebase with no superfluous crap goes towards giving you that courage.

Our best work happens when we get to practice software engineering. Whilst it’s our job to trade-off time and space so that we provide the best solution we can whilst keeping costs down, we should do so with the combined efforts of those we work with, both on the technical and business sides. It’s often hard to associate a monetary value with what we do, but doing that really helps you start thinking about your everyday actions in terms of the “cost of delay”.

So, the next time you’re tempted to just squeeze a little something else in, ask yourself whether the team would be better served by first finishing what’s already on your plate. If you really think it can’t wait then quickly canvas the opinion of a few stakeholders and/or the product owner to see whether it fits in with the current plan.

Better yet start pair programming. One of the best ways I’ve found to stay focused on the task at hand is by having someone else working with you. Whilst it’s entirely possible that the pair could wander off on a tangent I’ve found it extremely rare in practice. What normally happens is that one side begins to feel uncomfortable about the changing direction and a rapid exchange occurs to decide if this is really where they both want to go or whether it should wait for another day. It’s much easier to remain honest (if that’s what you need) when you have somebody else sat right next to you whose time you also have to account for.

 

[1] This is what your daily stand-up is for – reprioritisation.

Simple Language, Simple Process

There was a change made to the Scrum Guide a few years ago around the language used to describe what the “most important” thing is. Previously it was okay to talk about “prioritising the backlog” or “what order we should tackle the work” as these terms were synonymous.

But not any longer it seems. Now the terms “priority” and “order” mean different things, i.e. the priority of the product backlog does not necessarily define what order we tackle the work. If this sounds odd to you then you are not alone. While in theory the distinction sounds useful, in practice I’ve only found it to create more friction.

Pointless Pedantry?

The reason for introducing the subtle distinction was outlined in “Ordered Not Prioritized” by James Coplien. Anyone who has worked in a team where there has been tension over how to order the backlog so that work is done efficiently, which might not be the same order that the business would like, will know where Jim is coming from.

Whilst there is nothing intrinsically hard to change about software, as it is after all essentially non-physical, there are still constraints, such as complexity, which dictate how easy it is to achieve. The non-technical folk also may not understand that there are still limitations in the tools and materials that we work with. For example one tool may make it easier to get something simple functionally complete quicker, but its inability to scale adequately may make it the wrong choice for anything other than a prototype.

Balancing the business and technical priorities is tricky. They want features quicker, but we want to keep the quality bar high to ensure the product is able to support continued change. This is not an all-or-nothing proposition. Aside from the initial stages of development were we are mostly erecting the scaffolding that will support continuous delivery, the bulk of the work can be a mixture of the two.

I’ve been in the situation before a few times where there have been attempts to try and resolve the business and technical priorities by splitting up the backlog. This just doesn’t work, no matter how you slice-and-dice it there can only be one top priority. Making the team decide whether they should pick something off the business or technical backlog is not honouring the “negotiation” aspect that the agile manifesto promotes.

Hence, there almost certainly should be one, and only one backlog to ensure that everyone on both sides of the fence (business and technical) understand what we’re building in the shorter term and what’s further down the line (subject to renegotiation at any point). Having a single stream of work makes it really easy to see at a glance what the current plan is.

It’s All Just Work

Whilst Scrum might be straining the language in terms of priority at least it keeps the units of work simple – items. Over the years there has been a proliferation of ways to describe ever larger chunks of stuff to do from tasks, to stories, to epics and eventually to themes.

I’ve overheard conversations where people get hung up on the relative sizes of each of these bundles in the hope that they will be able to estimate the entire project delivery by just summing up the various factors. This naturally leads to a proliferation of charts showing the burning up (or down) of each of these units to help maintain the illusion that we are in control of the project and it’s progress and that we’ll meet whatever arbitrary deadline has been decided.

Breaking work down into small, manageable chunks is what matters. Each chunk of work should deliver some value, and by keeping them small we give ourselves room to manoeuvre and change direction (within limits) to meet the ever changing world around us. Just as when talking about the software architecture we tend to talk at different levels of abstraction, and so we can do the same with our features. This allows us to remain focused on, say, the bigger picture and not get unnecessarily bogged down in details which are superfluous at that level of granularity.

Many of the techniques around planning and estimation are largely exercises in getting you to break the problem down and identify risks to delivery. As long as the team is able to talk about the work and know at what “level” the current discussion is taking place, they should avoid getting distracted by arguments about whether any given feature is a task, story, epic, etc.

Back to Basics

When we cut away all the new terminology that has arisen over the last couple of decades we get back to a very simple development process. In essence we have a list of things that the business would like us to do, along with some other stuff that the implementing team also need to do, in order to ensure all this work happens efficiently.

That “to-do” list will contain work of varying size and complexity and both the business and technical sides will endeavour to break it down into small enough chunks that they can interleave them to ensure their respective needs are continually being met. The team will also periodically reflect on their working practice to help them discover even better ways of working.

And that’s pretty much it. Ultimately what we’re trying to achieve is very simple and yet the language often gets in the way and causes us to worry more about the process we’re following instead of what actually matters – creating working software.

Alternate Way to Mock the Current Time in C#

My previous post “Mocking the Current Time in C#” raised a few objections due to it’s use of a Static Façade that mirrors the underlying DateTime’s interface. Despite also disliking Singleton-like behaviour in general, I very occasionally find it acceptable [1]. In this instance reading the clock has no side-effects and the previous mocking technique is delightfully simple, but it’s not the only way. I have also worked on teams which prefer not to hide any dependencies behind static interfaces and in that situation I’ll adopt a more traditional mocking approach.

The Clock Object

The first thing to define is an interface for our Clock object:

public interface IClock
{
  DateTime Now { get; }
}

Now we can define an implementation for the real clock that we’ll use in production:

public class RealTimeClock : IClock
{
  public DateTime Now
  {
    get { return DateTime.Now; }
  }
}

So now, instead of hitting the hidden Clock façade, we’ll make our dependency more explicit by passing it via the constructor [2]:

public class DateCalculator
{
  public DateCalculator(IClock clock)
  {
    _clock = clock;
  }

  public int AgeToday(DateTime birthday)
  {
    . . .
  }

  private readonly IClock _clock;
}

Now we can make use of our clock in the real-world like this:

var clock = new RealTimeClock;
var calculator = new DateCalculator(clock);
. . .
var age = calculator.AgeToday(my.Birthday);

We can either hold onto the clock and calculator objects and reuse them (as they’re stateless) or recreate them each time. It’s likely that we won’t create them locally but will pass them down from somewhere above (as per “The PFA Papers”).

The Mock Object

Mocking the clock to test our date calculator is now as simple as writing any other test. In this example I’m defining a manual mock, but you may prefer to use a mocking framework instead [3].

public class FixedTimeClock : IClock
{
  public FixedTimeClock(DateTime time)
  {
    _time = time;
  }

  public DateTime Now
  {
    get { return _time; }
  }

  private readonly DateTime _time;
}

[Test]
public age_is_number_of_years_since_birthday()
{
  var today = new DateTime(2016, 6, 1);
  var clock = new FixedTimeClock(today);
  var calculator = new DateCalculator(clock);
  var birthday = new DateTime(2000, 1, 1);

  var age = calculator.AgeToday(birthday);

  Assert.That(age, Is.EqualTo(16));
}

Defaulting the Clock

The purpose of doing this was to make the clock an externally controllable dependency, which is why we made it an explicit argument to the DateCalculator constructor. However, despite the protestations about not using a static façade, I’ve then seen developers be happy with overloading the constructor so that you can avoid passing the real clock around everywhere in production code:

public class DateCalculator
{
  // Only used in production
  public DateCalculator()
    : this(new RealTimeClock())
  { }

  // Only used by unit tests
  internal DateCalculator(IClock clock)
  { . . . }

  public int AgeToday(DateTime birthday)
  { . . . }

  private readonly IClock _clock;
}

Another variation on this is to make the constructor that takes an explicit clock “internal” to show that it’s essentially just used for testing. At this point I have to ask again what the original objection to the Static Façade approach was, because we’ve just hidden the dependency again – it’s now only visible in unit tests – which is effectively no different to the previous approach. If you prefer to be explicit then it should be to make the production code, not test code, more transparent.

 

[1] So maybe I was off-by-one in “The Application-Wide Diagnostic Log - the One Acceptable Global Object” and there are really two acceptable global objects :o).

[2] Sorry but I can’t bring myself to use that sensationalist term “DI”, see “Terminology Overdose”.

[3] Just be aware of how they behave when you don’t configure them properly, see “Mocking Framework Expectations”.

Mocking the Current Time in C#

In my recent post “Testing 101 - A + B = C” I mentioned about mocking out calls to do with the current date & time to bring some determinism to those sorts of tests. I find this problem comes up any time you want to write tests around a thing that expires, or where you might be doing date arithmetic such as calculating someone’s current age.

I touched on my preferred solution somewhat tangentially back in 2011 when talking about a similar unit testing problem to do with the file-system (see “Unit Testing File-System Dependent Code”). However I decided it was worth writing it up explicitly as I’ve had a few conversations about it in the past and never been able to quickly point to something concrete.

A Level of Indirection

They say every problem in computer science can be solved with an extra level of indirection and that’s exactly what we need to do here. So, instead of using the DateTime.Now property everywhere:

var now = DateTime.Now;

…we instead use our own version:

var now = Clock.Now;

This might seem quite invasive, and in a sense it is, but historically to mock static methods involved specialist tools. If you just accept that you’re going to need to do this (and I’m sure you will if dates feature anywhere) you just do it from the beginning in a new project and it costs very little. Even retrofitting it later is easy as you can do it piecemeal if needs be, at least, to code that you own that is.

So, the default implementation can just chain onto the original property:

public static class Clock
{
  public static DateTime Now
  {
    get { return DateTime.Now; }
  }
}

With the abstraction in place we are now free to re-implement it so that we can replace the underlying behaviour to do whatever we want, such as return a fixed value in a test.

First we make the underlying function configurable by introducing another property to hold a delegate that we’ll invoke when Now is called [1]:

public static DateTime Now
{
  get { return NowImpl(); }
}

internal static Func<DateTime> NowImpl = () =>
{
  return DateTime.Now;
}

Naturally the default implementation does the right thing so that the production code will work out-of-the-box. However in a test we can now override it:

[Test]
public void the_current_time_is_always_fixed()
{
  var millenium = new DateTime(2000, 1, 1);

  Clock.NowImpl = () =>
  {
    return millenium;
  }

  Assert.That(Clock.Now, Is.EqualTo(millenium));
}

And that’s pretty much it. Normally I’ll add a Reset() method that can be called in the [TearDown] handler to put back the real clock just to keep things tidy and avoid a fixed date leaking out into another test.

Another Level of Indirection

You probably noticed that I marked the property that injects the mock function “internal” so that it has limited scope. I normally use the [InternalsVisibleTo] attribute to grant access to the test assembly. In the scenario where I can’t do that, such as when the Clock class resides in a shared assembly that is non-project specific (i.e. I can’t use [InternalsVisibleTo] to name an unknown assembly) I resort to a different trick.

Whilst I could give the NowImpl property a really log-winded name to discourage a developer from using it by accident, I prefer to keep the implementation private and then add a separate public Test API class to grant the necessary access. In C# a nested class has access to it’s parent’s internals and so I can do this instead:

private static Func<DateTime> NowImpl = () =>
{
  return DateTime.Now;
}

public static class TestApi
{
  public static Func<DateTime> Now
  {
    set { NowImpl = value; }
  }

  public void Reset()
  {
    NowImpl = () { return DateTime.Now; }
  }
}

This doesn’t really grant any more protection to the Clock internals but it does mean that IntelliSense won’t show up the other test methods unless you navigate into the TestApi class which feels a little cleaner.

Consequently the test would now look like this:

[TearDown]
public void ResetClock()
{
  Clock.TestApi.Reset();
}

[Test]
public void the_current_time_is_always_fixed()
{
  var millenium = new DateTime(2000, 1, 1);

  Clock.TestApi.Now = () =>
  {
    return millenium;
  }

  Assert.That(Clock.Now, Is.EqualTo(millenium));
}

 

[1] I could have collapsed the lambda down to a simple method group, but I prefer to make lambdas look like actual method bodies when explaining things. ReSharper will happily provide the relevant hint when you do it in real life.

Don’t Use Side-Effects to Verify Behaviour

I’m sure no one goes out of their way to do it, but it’s not uncommon to find tests that verify the behaviour by relying on a side-effect of the operation rather then sensing the actual operation itself. It’s easy to see how the rationale probably goes:

• I need to test that an operation has happened.
• The code under test logs when the operation occurs.
• So, I can use the presence of the log message as proof.

Here is the kind of test that results from applying that logic:

[Test]
public void manually_refreshing_requests_an_update()
{
  var cache = new Cache();
  Assume.That(Log.Messages.Count, Is.EqualTo(0));

  cache.Refresh();

  Assert.That(Log.Messages.Count, Is.EqualTo(1));
  Assert.That(Log.Messages[0].Text,
              Contains(“cache updated”));
}

This example has come up numerous times in real life and is a prime example of the confusion between correlation and causation. In essence the test is saying that the http message was sent due to the log message being written, which is clearly nonsense.

The log message or event is probably highly coupled to the action that is under test and therefore is highly correlated with it, but that does not make it any more right to use it to sense behaviour. When developers complain about brittle tests this is exactly the kind of thing they dislike because a change in non-functional behaviour can cause a test for functional behaviour to break.

The test should should make it abundantly clear what interaction is expected to be taking place. If the side-effect is behaviour that needs verifying too, then it should be done separately. This is another problem I often see with tests – one test verifying two independent behaviours. I wouldn’t want to suggest it’s down to laziness per-se but “killing two birds with one stone” might seem an easier path to take in the short term.

Consequently I’d end up with two different tests – one to sense the interaction with the HTTP client and another to describe the side-effect.

[Test]
public void manually_refreshing_requests_an_update()
{
  var httpClient = new MockHttpClient();
  var cache = new Cache(httpClient);

  cache.Refresh();

  Assert.That(httpClient.Requests.Count,
              Is.EqualTo(1));
}

[Test]
public void an_event_is_logged_when_the_cache_updates()
{
  var cache = new Cache();
  Assume.That(Log.Messages.Count, Is.EqualTo(0));

  cache.Refresh();

  Assert.That(Log.Messages.Count, Is.EqualTo(1));
  Assert.That(Log.Messages[0].Text,
              Contains(“cache updated”));
}

You may notice that the original test made no reference to the HTTP Client, which may be because it’s a singleton mocked out elsewhere. As we didn’t interact with it directly in the test it wasn’t mentioned, but in my rewritten test I’ve brought the dependency front-and-centre to make it obvious what the interaction is with.

The second one is just the original test but with a better name to more actually describe the behaviour we are interested in. Now we can change the behaviours a little more easily. Yes, the second is still coupled to the behaviour in the first, but the first is no longer tied to the behaviour in the second.

Testing 101 - A + B = C

When writing production code the functions with the lowest coupling / easiest to reason about are (generally speaking) pure functions. They only depend on their inputs and there are no side effects either which means that given the same inputs you should always get the same output. But this doesn’t just have to apply to production code, in fact it needs to apply to test code and the environment too.

I’m sure a proper mathematician will snarl at my simple analogy but the equation “A + B = C” is how I expect tests to behave. That is to say: given the same inputs and behaviours I expect to get the same output. Always. In essence I expect the tests to be as dependable as simple arithmetic.

This simple premise forms the bedrock on which all tests sit because you cannot (easily) make progress on your software project if there is a lot of noise going on around you. Every time a test fails for seemingly innocuous reasons you will be distracted from your goal. Even if you choose to completely ignore the failing tests your build pipeline will stall every time it happens and so you will still be distracted even just to keep re-starting the build! In my experience it pays to cure the disease rather than just keep treating the symptoms.

Removing sources of non-determinism in tests is definitely one problem I find myself tackling fairly regularly. Trying to ensure that A and B are the only inputs into the test, along with ensuring that A and B are always the same, is a common source of noise. In particular code and tests where date/times or concurrency is involved is likely to suffer from random distracting failures.

The other aspect of testing that this little mathematical equation is intended to convey is simplicity. The number of variables is consciously small and there is only a single operator which provides the point of focus. While you might think that this implies a discussion about low-level unit tests, it can equally apply to other forms of automated tests too. Where A and B might be simple values in a unit test they might be a single, larger compound value in an acceptance test. Similarly the operation may be just a single method in the former case and a RESTful API call in the latter. The increase in size and complexity can be offset (to a certain degree) by application of the Separation of Concerns for the scenario.

Environment

One of the first invariants to consider is the environment. It helps if you can keep it as stable as possible across test runs – if you keep switching the version of the toolchain and dependencies on each test run how will you know whether the failure is down to your change or someone in the background?

Obviously it’s not normally that blatant but there are common differences that I see developers keep getting tripped up on. One is the build server using a fresh checkout versus the developer’s workstation which often has piles of detritus building up because they don’t realise what their IDE leaves behind (see “Cleaning the Workspace”). Another is using a different, “compatible” test runner from within the IDE whilst the build script uses the real packaged command line tool. If you don’t run the same build script before publishing your changes then you’re faced with all the differences between the build server and workstation to consider when it breaks.

Many development practices have changed over time and the notion of a stable working environment is one that is now being heavily challenged. The use of cloud based build services like Travis and AppVeyor means you may have much less control over all the specific details of the build environment. Also the pace at which 3rd party packages change, such as those from NuGet or NPM, along with the number we now depend on, means that any two builds may have used different dependencies. Sitting on the bleeding edge comes at a cost so be sure you know what you’re getting yourself into.

The analogy I favour most when it comes to changing any invariants such as the build environment is to follow the advice of the rock climber – only move one thing at a time. So I’m told, a rock climber always keeps at least 3 limbs attached to the rock face and therefore only ever moves one at a time. When the moved limb is secure, only then do they move another one. I’m sure that experienced rock climbers do break the rules, sometimes, but they do it knowing what the full consequences are.

Avoiding Non-Determinism In Tests

With our environment behaving in a predictable manner we have reduced the potential for noise so that the first port of call for our test failures is the production code or test itself. Unless you are writing something inherently non-deterministic, like a true random number generator, then it should be possible to control the production code in a way that allows you to test it in a well-defined way.

This isn’t always easy, but it starts by not trying to test behaviours which are inherently unreliable, such as waiting for a fixed amount of time to pass, or an operation to run on a background thread. Whilst these operations may complete fairly quickly on your super-fast developer workstation, your build server will likely be a VM sharing a hugely overloaded box and so CPU cycles are somewhat scarcer.

My preferred method of dealing with such behaviours is either to mock system calls, such as getting the current time, or to use synchronisation objects like manual events, semaphores and countdown events to sense the transitions through mocks. If I’m scheduling work onto other threads I might mock the scheduler (or dispatch function) with one that just runs the task directly on the same thread so that I know for sure it will complete before the method returns.

The mistake I think many developers make is that they believe it’s possible to test many concurrent behaviours reliably. Whilst you might be able to prove the presence of a deadlock by testing it continuously for a period of time, you cannot prove the absence of it that way (Dijkstra taught us that) [1]. The cost is longer running tests and therefore the chances that they will be run less often. Consequently I prefer to mostly write tests that characterise the concurrent behaviour, rather than attempt to prove the concurrency aspect is correct. That I leave for testing in an environment which is much better suited to the task.

Just recently I came across a test that failed one out of every 10 attempts, but when I added another test the failure rate went right up. The reason the original test failed at all was down to the spinning up of a background thread to do a cache refresh which occasionally took longer than what the test was prepared to wait. Waiting an arbitrary amount of time is generally a big no-no, you should try to synchronise with the thread somehow. In this instance I could do that through the outbound API mock which the operation was invoking.

The reason the failure rate went up was due to the way that NUnit runs tests alphabetically. My new test came “after” the previous one, which never cleaned up the background thread, and so it could fire a refresh again when my test was running. This caused multiple refreshes to be registered instead of zero. The answer was to retain ownership of the background thread and ensure it terminated by the time the test completed.

Does this add complexity to the production code? Yes. But, as is often the case when writing tests properly, it uncovers a number of questions about the expected behaviour that might not be obvious from quickly knocking something up.

Property Based Testing

One of the problems with example based testing is that, by definition, you only run through the scenarios laid out with specific examples. If you miss one, e.g. testing for a leap year, you won’t find the bug. Property based testing takes a slightly different approach that attempts to run through a variety of scenarios made up on-the-fly by a generator.

This might seem to go against the earlier advice, but it’s not because as long as you know the seed for the test generator you can reproduce the scenario exactly. If you can explore the entire problem space on each test run do, but unless it’s a trivial feature that’s unlikely. Hence you use a generator to create inputs that explore a (different) part of it each time. Consequently you still might not unearth that leap year bug in time, but you stand a better chance of finding it other unanticipated problems too.

Wrap-Up

Not every test we write will be as simple and predictable as my little maths equation suggests, but that doesn’t mean we shouldn’t try to get as close to it as possible. Some of the unit tests I once wrote for a message bus connection pool definitely weren’t easy to understand, despite my best efforts to keep them clear, but hopefully they are very much in the minority.

We all make mistakes too, so we might not get it right first time. The unit tests I wrote for logging garbage collections ran fine for over a year before I discovered one of them was leaking slightly and causing another (very intermittent) random failure. But I managed to fix it because I believe a zero tolerance approach to test failures pays dividends in the long run.

Good tests are hard to write, but they should be treated as first class citizens, just like your production code. If they aren’t as simple as “A + B = C” then consider it a test smell and see if there is some unnecessary complexity that can be factored out.

[1] It might be possible to verify it’s behaviour through other means, such as induction, but that’s something I personally know far too little about to use effectively.

Turning Technical Tasks Into User Stories

When working on a mature system it’s all to easy to fall into describing the work as a bunch of technical tasks. Those in the trenches that have been working on the system for years will no doubt know how to implement many of the features that come up and so what happens is that they get put on the backlog as technical stories.

Whilst in the end everything will no doubt end up as a technical task to be done by a member of the team anyway, you don’t want to get to that point too quickly. The problem with putting technical stories straight on the backlog is that it often locks the product owner out of the development process due to it’s solution oriented nature.

A common malaise in the software development industry is the weak excuse of “we’ve always done it this way”. Breaking features down into technical stories plays right into this way of thinking because there is a focus on trying to leverage the skills you already have without looking at the bigger picture first. Instead of just launching straight in you need to take a step back and survey the landscape to see if it’s changed since the last time you had a look around.

Prioritisation

One of the things that can go out the window when you frame a story around the implementation is the ability of the product owner to prioritise. The developers, who know the ramifications of it not being done, can end up flannelling the product owner as they mistake the PO’s intent of discovering “when”, not “if” it will be done.

I remember a story at a company which basically described configuring a bunch of stuff. This was not an unusual task, the developers knew what needed doing, but the highly technical nature of the story description sat uneasily with me. So, along with the team, I tried to see if we could elevate it to a user focused story to try and eek out the essence of it.

I started by asking what would happen if this task wasn’t done. We had already identified plenty of waste elsewhere and I wanted to make sure this wasn’t also just waste. The conversation focused on the effects of the primary user (an advisor) but I eventually managed to get them to explain what it meant for the actual end user (i.e. the customer).

At this point the truth started to unfold. Firstly it became apparent that the behaviour enabled by this work might be invoked the day the feature when live, but that was highly unlikely because of the product (think warranty). Also, if it wasn’t done at go-live time the end user would still get a service, but you could consider it as a degraded level of service. There was probably even a manual way of mitigating the risk should the task not get completed before the feature went live and the unlikely scenario materialise.

At this point the light-bulb went on and the developers could now see that this work, whilst probably required in the longer term, was not essential now. What we had done by turning the technical task into a user story was unlock the product owner so that they could now evaluate the work in the context of the overall business plan and prioritise it appropriately, along with understanding the risk of later delivery.

This is a common problem when teams adopt a more agile approach. They assume that the same amount of work needs to be done, which may well be true, but at the very least the order in which it needs to be done could be radically different. Traditionally you get the product when it’s ready – all the value comes at the end – but an agile approach looks to unlock value sooner. To achieve that you need to elevate your thinking and look at the problem from the customer’s point-of-view. Essentially you need to be able to help the product owner weigh up the alternatives (“I Know the Cost but Not the Value”).

Problem Solving

The second issue I’ve noticed around the “we’ve always done it this way” mentality is that you end up with tasks that assume there is only one solution [1]. Once again it may turn out to be the eventual solution, but you owe it to yourselves to at least consider whether this time things might be different. Perhaps there are other constraints or changes that mean it’s not applicable this time around; just because we know one way of solving the problem does not imply it’s the best, or most cost-effective.

One example I had of this was a user story I saw which involved coding some help text into a system. Once again I tried to get the development team to view user stories not as an absolute specification of what to build, but more generally as a problem that needs to be solved. In particular I wanted them to start thinking about whether there were any low-tech or even non-IT related options.

During the discussion it became apparent that the primary user (once again an advisor) only needed the text whilst they were learning about the new feature, once they knew it the text would be redundant. My first suggestion was to just print the text out on some paper and leave it on their desks or stick it on the side of the monitor. When they were confident they knew it they could bin it or put it in their drawer for safe keeping.

Sadly there were some bizarre rules in place about not being allowed stuff on their desk. So instead I suggested pasting the text into an HTML page that was accessed from their desktop when they needed it. This was discarded for other reasons, but what it did do was cause someone in the team itself to mention another help system that the users accessed which was easier to update. The existence of this other, newer system, along with a whole raft of other nuggets of information about the users and the environment they worked in were unearthed through that one conversation. Consequently the team began to understand that their job was not to simply “write code” but was in fact to solve people’s problems through whichever means best satisfied the constraints.

I don’t know which solution was eventually chosen, but once again the wider conversation allowed the product owner to make a better judgement call about if, and when, it needed doing.

Training Wheels

Like so many aspects of adopting an agile development process this is just a technique, not a hard-and-fast rule. What I detected was a “smell” in each of these stories and so I made the team work hard to get to the crux of what really needing delivering. Like the classic story format “as a user, I want…” it’s all about trying to discover who wants what and why. Once you start thinking that way more naturally you can ditch the training wheels and use whatever techniques work best for the team. If you feel confident cutting-to-the-chase sometimes then do it.

[1] “There's nothing more dangerous than an idea if its the only one you have.” -- Émile Chartier