One would hope in this day-and-age that when hiring a candidate for a role as a developer the interview process would include some element of actually doing what it is we do - programming. This could be as simple as submitting an offline coding test, but better yet a face-to-face session where you pair with one of the people you’re likely to work with. Either way, just as an artist would expect to show their portfolio, we should expect to do something similar – not just write a CV and then talk about it.
If that’s the case for hiring a single developer, why shouldn’t choosing a software supplier follow a similar process, after all there is probably far more money at stake. And that’s really what the whole interview process is all about – it’s a form of risk management – you’re trying to find a supplier that you’re confident is going to deliver what you really need in a timely manner and for an acceptable cost.
Show Your Workings
The Hackathon, which probably sounds a bit too hipster for some people’s taste, is the embodiment of this idea – get the potential suppliers to show how they actually produce software, albeit on a very small scale. My last two days have just been spent on one of these and it was great fun, if a little nerve-wracking.
The consultancy firm I’m currently working through (Equal Experts) has been involved in one of these before, quite recently, and they relish the chance to do this kind of selection process more as it plays to their strengths – delivering working software incrementally using modern development techniques [1].
By all accounts this particular one was slightly different to the previous one, although it still followed a similar outline. There are a number of very small teams (3) representing the different suppliers. Each team has a small number of people (~4) that covers whatever skills they think they’ll need based on the brief. In our case we had 3 devs and a UX person, but obviously we could also act in a lesser capacity as a BA, QA, Ops, etc. too to cover all the bases.
The overall structure was that we would spend a short period learning a bit about the company we were pitching to and about the problem they wanted us to tackle. Each team was then given a separate “war” room in which they could work for roughly two days before presenting back at the end.
Where in the previous Hackathon they got more freedom about what they wanted to tackle (i.e. a higher-level problem statement) this one had a more specific problem to be solved. The problem was also directly related to the actual problem they chosen supplier would eventually be asked to work on, which makes sense.
During the two days various people involved in the selection process would come round and visit us to see what we’d been up to and that would also give us an opportunity to ask any questions we had about the problem. If we really needed to we could have called upon The Business at any time to come and help us but their visits were frequent enough that it meant we never needed to go that far.
Our Approach
Naturally I can’t say anything about the problem itself, but suffice to say that it was far bigger than anything we could expect to build in just two days. However that didn’t stop us tackling it in the same way we would a real project. We did some up-front analysis to explore the problem domain a fair bit, make some key architectural decisions to decide what we were going to build, create a backlog with some stories on it and then start coding. We also littered the walls with sheets of magic whiteboard [2], post-it notes and index cards that covered our questions, answers, architecture, backlog, etc.
We were able to to do a little bit of work beforehand, such as creating a GitHub repo that we could all check we had access to, along with a dummy build in AppVeyor to cover the CI part. The guest Wi-Fi wasn’t brilliant [3] which meant pushing and pulling to/from GitHub was a bit laggy, but it was usable and the cloud based CI monitor was stable enough.
Despite it being unofficially described as a “Hackathon” we decided we would do the best we could to show how we worked in practice, rather than try and push out as much code as possible. Whilst we no doubt could have got more code working if we had cut corners on some of the other parts of process (e.g. analysis or testing) we would not have given a fair representation of what we do (IMHO). I’ve touched on this before in “Keeping the Faith Under Pressure” and I’m pleased with what we produced. I’m perfectly happy to call the code I wrote over the two days “production code”.
Day One
After doing our early analysis we started putting together the walking skeleton which was a text-book example of Conway’s Law as we initially kept out of each other’s way whilst we got the boilerplate stuff together. Of course this separation didn’t last long as by the end of the day the temporary assemblies and folders had all gone in a simple refactoring exercise so that we had a coherent codebase to build from.
We finished the day with our technology stack up and running (browser-based client + REST API), watched over by AppVeyor, and a very thin slice of functionality in play which we demoed to a couple of the stakeholders.
Day Two
The following day was essentially a few hours shorter to allow time for each team to present to the panel, that also meant we’d need to factor in some time to put the presentation together. In essence we only had ~5 hours to implement the next set of features and so got one of the stakeholders in first-thing to make a priority call so we knew where to focus our efforts during the morning.
During the time we three devs were building a working product our UX expert was building a clickable prototype to help explore the problem further ahead. This had the benefit of us being able to see the context in which the data was used and therefore we could better understand the bigger picture. In such a short timeframe it was perhaps difficult to see what beneficial effect it had on our design and thinking but what it did show clearly was how we work and how important we believe UX to be to the development process in general.
We stopped coding an hour before our presentation slot to give ourselves plenty of time to think about what we needed to say and show. Our UX expert had surreptitiously taken plenty of photos of us engaging with the stakeholders along with the evolving boards and backlog so that we could put together a compelling story about our approach.
We talked for 30 minutes about the process and architecture, and gave a demo of what we’d built to date. We then had about 15 minutes to answer questions from the panel, both about our approach and how we might tackle some of the questions we hadn’t yet answered in the code.
Each team got to present to the panel in isolation, but we all hung around until the end at which point we each did a very brief version of the presentation to the other teams. This was really interesting as up to that point we had no idea what the others were up to. For example we chose a thin-client whereas the other two chose thick-clients. We used post-its and a makeshift whiteboard for our notes and product backlog whilst another used an online tool and the third didn’t mention their approach.
Wrapping Up
Did the exercise achieve what it set out to do? As I’m not the client I have no idea what their final decision is or whether the eventual product has met their needs, because clearly it hasn’t been built yet. But I believe their behaviour suggested that they were pretty pleased with what they had seen from everyone. I think they seemed surprised that the three teams had behaved quite differently and so they probably got a lot more out of the exercise than anticipated as each team would probably have asked different questions and explored different avenues. Given that this was a real problem we were working on I’m sure there is a lot of value in that alone.
Personally I went into the process somewhat nervous. My current role is a real departure for me - it’s not a hands-on development role. As such I thought I might not be “match fit” even with 20 years of coding experience behind me. What I forgot though was that the point of the process was to be ourselves and write code as I would for real and that just came back naturally. Hopefully I added an equal amount of value to the team and gave as good an account of myself as the others appeared to.
So, is this is the way I think tenders should be done? Yes, but I’m just a developer :o). I did ask someone at Equal Experts about how it compared cost-wise for them given that the RFP (Request for Proposal) process can also be pretty time consuming too and he suggested it wasn’t that far off. Not having done any others I can’t say whether the client had a disproportionate number of people involved at their end but given the potential sums at stake I’m sure they saw it as a good investment. I certainly hope more companies do it in the future.
[1] Apologies if that sounded like a thinly-veiled sales pitch, it wasn’t mean to be. I was just trying to describe how they see themselves; I’m an associate, not an employee.
[2] Somewhat unfortunately the room we where in had frosted glass and the sheets of magic whiteboard didn’t really stick to it. Without any blu-tack or sellotape we had to use even more post-it notes to hold up the whiteboards!
[3] The Wi-Fi dropped out every now and then and had limited bandwidth which meant that any image-heavy web pages could take a while to load.
