Wednesday, 19 November 2014

TFS Commit Triggering Wrong Gated Build

This post is very much of the “weird stuff I’ve run across” variety, so don’t expect any in depth analysis and solution. I’m just documenting it purely in case someone else runs into a similar problem and then they won’t feel like they’re going bonkers either...

The other evening I wasted the best part of a couple of hours trying to track down why my TFS gated build was failing when pushing even the simplest change. I had started with a much bigger commit but in the process of thinking I was going mad I shelved my changes and tried pushing a “whitespace only” change that I knew couldn’t possibly be the cause of the unit test failures the gated build was experiencing.

What I eventually noticed was that the codebase had some commits from someone else just prior to me but I couldn’t see their gated builds in the TFS online builds page for the project. This suggested that either they were pushing changes and bypassing the gated build, or something funky was going on.

I finally worked out what was causing the build to fail (it was in their commit) and so I fixed that first. Then in the morning I started the investigation as to why my colleague’s commits did not appear to be going through the gated build. He was quite sure that he was being forced to go through the gated build process and even showed me “his builds” in the Visual Studio TFS builds page to prove it. But as we looked a little more closely together I noticed that the builds listed were for an entirely different TFS project!

Our TFS repo has a number of different projects, each for a different service, that have their own Main branch, Visual Studio .sln and gated build process [1]. Essentially he was committing changes in one part of the source tree (i.e. one project) but TFS was then kicking off a gated build for a solution in a different part of the source tree (i.e. an entirely different project). And because the project that was being built was stable, every build was just rebuilding the same code again and again and therefore succeeding. TFS was then perfectly happy to merge whatever shelveset was attached to the gated build because it knew the build had succeeded, despite the fact that the build and merge operated on different TFS projects!

My gut reaction was that it was probably something to do with workspace mappings. He had a single workspace mapping right at the TFS root, whereas I was using git-tfs and another colleague had mapped their workspace at the project level. He removed the files in the workspace (but did not delete the mapping itself) [2], then fetched the latest code again and all was good. Hence it appears that something was cached locally somewhere in the workspace that was causing this to happen.

As I was writing this up and reflecting on the history of this project I realised that it was born from a copy-and-paste of an existing project - the very project that was being built by accident. Basically the service was split into two, and that operation was done by the person whose commits were causing the problem, which all makes sense now.

What I find slightly worrying about this whole issue is that essentially you can circumvent the gated build process by doing something client-side. Whilst I hope that no one in my team would ever consider resorting to such Machiavellian tactics just to get their code integrated it does raise some interesting questions about the architecture of TFS and/or the way we’ve got it configured.


[1] That’s not wholly true, in case it matters. Each separately deployable component has its own TFS project and gated build, but there are also projects at the same level in the tree that do not have a “Main” branch or a gated build at all. I think most projects also share the same XAML build definition too, with only the path to the source code differing.

[2] To quote Ripley from Aliens: “I say we take off and nuke the entire site from orbit. It's the only way to be sure.”

No comments:

Post a Comment