What’s the Check-In Frequency, Kenneth?

Last week saw two unrelated events come together to provide the basis for this post. The first was the demise of R.E.M., a rock band that I would never describe myself as being a massive fan of, but via osmosis[*] aware enough for them to provide a particularly bad pun for this post’s title. The second, and far more relevant event was the discussion on site about whether it’s possible to check-in too frequently or not.

Let’s be clear from the outset that we’re talking specifically here about integration branches. Private branches are, by definition, personal and so the frequency with which a developer chooses to commit is entirely their own choice (feature branches are usually synonymous with private branches). However an integration branch serves different purposes depending on the point in time that you wish to mine it.

Feature Development

As a developer I want to integrate my changes as fast as possible to ensure that my codebase always remains fresh. The longer I leave uncommitted changes outstanding, the more I open myself up to the possibility of tricky merges and integration problems. That’s the theory.

In practice one hopes that the entire codebase is not so volatile that this actually happens. If it does then I’d be worried that there is so much refactoring going on that no one is actually getting any real work done! Of course the team size will play a large part in this, but even then you just can’t expect developers to make sensible progress if they keep tripping over each others heels - you can’t build a house on quicksand.

For new features, where there is the greater chance of a high check-in rate as the developer goes through various twists and turns, the check-ins are mostly of interest to them and anyone reviewing. Once a feature reaches v1.0 and enters maintenance mode then every check-in becomes of interest to a larger audience because you now have something to break.

Integration/System Testing

When feature is complete the code will likely be in a state where all the unit tests pass and hopefully some amount of integration and system testing has been done too (and acceptance testing, if you’re lucky enough to have them). Depending on the testability of the system though it may only be complete to unit and rudimentary integration test level and so still have to go through some form of system testing where you get to throw higher volumes of data at it.

At the point of deployment[+] into a system test environment I find that the integration branch history takes on a different perspective. Now I’m interested in what changes are going to be deployed so that I have an idea of where trouble it likely to show up, and when it does I want to be able to drill into the changes to triage the problem quickly. If you’re working on a batch processing system, nothing kills a day more than finding that a bug has caused the system to keel over at the first hurdle as you’ve effectively lost an entire test run.

Software Archaeology

Once a feature is mature it hopefully just sits in the background doing its thing. Until it breaks, or meets a condition it never expected. If it’s a regression, or you don’t understand why the problem has only just surfaced, then you may need to go back in time and trawl the VCS logs piecing together any related changes to build a picture of what’s happened. Once again this is where noise from frequent check-ins makes the job much harder and you’ll almost certainly only be interested in changes since the feature went live. Any experimentation at this point is just noise, you’re only interested in version N and N-1 and the net effects of those changes.

If you’re new to a team then you wont have any history at all to draw on and so the VCS logs may be your only way of finding out about how the system has evolved. I’ve seen plenty of strange code and it has only been through the power of the VCS time machine that I’ve managed to work out why it’s in that state and, more importantly, how I can move it forward in the intended direction[#]. However this is also one scenario where going back over the changes in a private branch can actually be useful.

Personally I believe that after the text editor, the next most important tool you really need to grok is the version control system. This is especially true when working on legacy systems as you probably have none of the original team around.

It’s Just a Tooling Problem

My case for a little restraint may well appear weak with the counter-argument being that frequent check-ins are A Good Thing and what I describe is just a problem with the VCS tool (or my inability to drive it sufficiently well). And you’d have a point. But the VCS tools I’ve uses to date are all focused around managing single (hopefully atomic) change-sets; they do not provide an easy way for you to view the multiple change-sets that constitute “a feature” as a single set. To me “multiple change-sets” sounds awfully like what you’d use a feature/private branch for… The question then becomes “how many check-ins before I should have created a separate branch”?

The Extreme Case

In case I still haven’t made a compelling argument then perhaps the following extreme case will help you see where I’m coming from, and remember that I’m talking about doing this in an integration branch:-

Every check-in should be atomic and what is checked in shouldn’t break the build or destabilise the codebase if at all possible. That means that when writing a new feature, using a process like TDD, I could check in my changes every time I write a passing test.

I would contend that doing this adds a significant amount of noise to an integration branch, and that if you want to keep every little step you make then you should create your own branch and commit to that instead. Then integrate your changes on a less frequent basis, but still frequent enough that you avoid integration issues.

DVCS to the Rescue?

One technique I’ve yet to explore is to layer a DVCS like Git or Mercurial on top of Subversion to use as a private branch repository. This sounds attractive from the noise perspective, but I wouldn’t want to give up the check-in history of the small steps entirely in case it’s useful some time in the future. I know there are a few ACCU members who are well versed in this area and so I shall have to see whether there is more to this way of working than meets the eye.

Show Me the Money

So, putting my Tom Gilb hat on I feel compelled to somehow quantify what I feel is too often and what is too little. When you’re in maintenance mode a lot of features may literally only take minutes or hours to write and so there are natural points of integration already built-in. The basis for this post though was about significant new features because there are far less checks-and-balances (e.g. regression testing) in play to stem the natural flow of commits because you having nothing to break. In this case I would look to break the task down to see if there are any natural points where I can deliver sizeable chunks (1/2 day - 1 day) of complete functionality so that it can be pushed into system test sooner and I can then reap the system testing benefits in the background. If there is absolutely no way I can deliver something concrete without destabilising the system, or if I know I could be experimenting, only then will I resort to a private branch and only for the destabilising change. Once I can get back to delivering large chunks directly into the integration branch I will.

In essence what seems to have driven the upper limit of the commit frequency for me in recent years is the rate at which system testing occurs. Even in this day and age of unit testing and continuous integration “getting something in the build” is still something to aim for when a build, deployment and end-to-end test takes hours.

 

[*] I used to share a house with a number of people who listened to R.E.M. a fair bit. Personally I was happy enough to leave it at Eponymous and the 12” of The One I Love; for the wife of course...

[+] This is as often as possible. In theory, on the sorts of systems I work on, it would be great for it to happen automatically after each end-to-end run so that we have a continuous cycle of build, deploy, do an end-to-end run, rinse & repeat. In practice it ends up being largely automated, with the manual element being the decision to run the cycle to ensure the highest chance of getting a successful end-to-end run.

[#] I remember one very curious piece of code that changed rapidly and had lots of simple check-ins that seemed to be trying to workaround some kind of memory management problem. I came across it whilst trying to fix a very large memory leak and from the VCS log I could deduce that the programmer was trying to emulate a weak pointer with a non-weak one, hence the eventual leak.