Lose the Source Luke?

We were writing a new service to distribute financial pricing data around the trading floor as a companion to our new desktop pricing tool. The plugin architecture allowed us to write modular components that could tap into the event streams for various reasons, e.g. provide gateways to 3rd party data streams.

Linking New to Old

One of the first plugins we wrote allowed us to publish pricing data to a much older in-house data service which had been sat running in the server room for some years as part of the contributions system. This meant we could eventually phase that out and switch over to the new platform once we had parity with it.

The plugin was a doddle to write and we quickly had pricing data flowing from the new service out to a test instance of the old service which we intended to leave running in the background for soak testing. As it was an in-house tool there was no installer and my colleague had a copy of the binaries lying around on his machine [1]. Also he was one of the original developers so knew exactly what he was doing to set it up.

A Curious Error Message

Everything seemed to be working fine at first but as the data volumes grew we suddenly noticed that the data feed would eventually hang after a few days. In the beginning we were developing the core of the new service so quickly it was constantly being upgraded but now the pace was slowing down the new service was alive for much longer. Given how mature the old service was we assumed the issue was with the new one. Also there was a curious message in the log for the old service about “an invalid transaction ID” before the feed stopped.

While debugging the new plugin code my colleague remembered that the Transaction ID meant the message sequence number that goes in every message to allow for ordering and re-transmission when running over UDP. The data type for that was a 16-bit unsigned integer so it dawned on us that we had probably messed up handling the wrapping of the Transaction ID.

Use the Source Luke

Given how long ago he last worked on the old service he couldn’t quite remember what the protocol was for resetting the Transaction ID so we decided to go and look at the old service source code to see how it handled it. Despite being at the company for a few years myself this all pre-dated me so I left my colleague to do the rummaging.

Not long after my colleague came back over to my desk and asked if I might know where the source code was. Like so many programmers in a small company I was a part-time sysadmin and generally looked after some of servers we used for development duties, such as the one where our Visual SourceSafe repository lived that contained all the projects we’d ever worked on since I joined.

The VCS Upgrade

When I first started at the company there were only a couple of programmers not working on the mainframe and they wrote their own version control system. It was very Heath Robinson and used exclusive file locks to side-step the problem of concurrent changes. Having been used to a few VCS tools by then such as PVCS, Star Versions, and Visual SourceSafe I suggested that we move to a 3rd party VCS product as we needed more optimistic concurrency controls as more people were going to join the team. Given the MSDN licenses we already had along with my own experience Visual SourceSafe (VSS) seemed like a natural choice back then [2].

Around the same time the existing development server was getting a bit long in the tooth so the company forked out for a brand new server and so I set-up the new VSS repository on that and all my code went in there along with all the subsequent projects we started. None of the people that joined after me ever touched any of the old codebase or VCS as it was so mature it hadn’t needed changing in some time and anyway the two original devs where still there to look after it.

The Office Move

A couple of years after I joined, the owners of the lovely building the company had been renting for the last few decades decided they wanted to gut and renovate it as the area in London where we were based was getting a big makeover. Hence we were forced to move to new premises about half a mile away. The new premises were nice and modern and I no longer had the vent from the portable air-conditioning machine from one of the small server rooms pumping out hot air right behind my desk [3].

When moving day came I made sure the new server with all our stuff on got safely transported to the new office’s server room so that we ready to go again on Monday morning. As we stood staring around the empty office floor my colleague pointed to the old development server which had lay dormant in the corner and asked me (rhetorically) whether we should even bother taking it with us. As far as I was concerned everything I’d ever needed had always been on the new server and so I didn’t know what was left that we’d still need.

My colleague agreed and so we left the server to be chucked in the skip when the bulldozers came.

Dormant, But Not Redundant

It turned out their original home-grown version control system had a few projects in it, including the old data service. Luckily one of the original developers who worked on the contributions side still had an up-to-date copy of that and my colleague found a local copy of the code for one of the other services but had no idea how up-to-date it was. Sadly nobody had even a partial copy of the source to the data service we were interested in but we were going to replace that anyway so in the end the loss was far less significant than we originally feared.

In retrospect I can’t believe we didn’t even just take the hard disk with us. The server was a classic tower so took up a far bit of room which was still somewhat at a premium in the new office whereas the disk could probably have sit in a desk drawer or even been fitted as an extra drive in the new midi sized development server.

 

[1] +1 for xcopy deployment which made setting up development and test instances a piece of cake.

[2] There are a lot of stories of file corruption issues with VSS but in the 7 years I’d used it with small teams, even over a VPN, we only had one file corruption issue that we quickly restored from a backup.

[3] We were on the opposite side from the windows too so didn’t even get a cool breeze from those either.

 

Pair Programming Interviews

Let’s be honest, hiring people is hard and there are no perfect approaches. However it feels somewhat logical that if you’re hiring someone who will spend a significant amount of their time solving problems by writing software, then you should probably at least try and validate that they are up to the task. That doesn’t mean you don’t also look for ways to asses their suitability for the other aspects of software development that don’t involve programming, only that being able to solve a problem with code will encompass a fair part of what they’ll be doing on a day-to-day basis [1].

Early Computer Based Tests

The first time I was ever asked to write code on a computer as part of an interview was way back in the late ‘90s. Back then pair programming wasn’t much of a thing in the Enterprise circles I moved in and so the exercise was very hands-off. They left me in the boardroom with a computer (but no internet access) and gave me a choice of exercises. Someone popped in half way through to make sure I was alright but other than that I had no contact with anyone. At the end I chatted briefly with the interviewer about the task but it felt more like a box ticking affair than any real attempt to gain much of an insight into how I actually behaved as a programmer. (An exercise in separating “the wheat from the chaff”.)

I got the job and then watched from the other side of the table as other people went through the same process. In retrospect being asked to write code on an actual computer was still quite novel back then and therefore we probably didn’t explore it as much as we should have.

It was almost 15 years before I was asked to write code on a computer again as part of an interview. In between I had gone through the traditional pencil & paper exercises which I was struggling with more and more [2] as I adopted TDD and refactoring as my “stepwise refinement” process of choice.

My First Pair Programming Interview

Around 2013 an old friend in the ACCU, Ed Sykes, told me about a consultancy firm called Equal Experts who were looking to hire experienced freelance software developers. Part of their interview process was a simple kata done in a pair programming style. While I had done no formal pair programming up to that time [3] it was a core technique within the firm and so any candidates were expected to be comfortable adopting this practice where preferable.

I was interviewed by Ed Sykes, who played a kind of Product Owner role, and Adam Straughan, who was more hands-on in the experience. They gave me the Roman Numerals kata (decimal to roman conversion), which I hadn’t done before, and an hour to solve it. I took a pretty conventional approach but didn’t quite solve the whole thing in the allotted time as I didn’t quite manage to get the special cases to fall out more naturally. Still, the interviewers must have got what they were after as once again I got the job. Naturally I got involved in the hiring process at Equal Experts too because I really liked the process I had gone through and I wanted to see what it was like on the other side of the keyboard. It seemed so natural that I wondered why more companies didn’t adopt something similar, irrespective of whether or not any pair programming was involved in the role.

Whenever I got involved in hiring for the end client I also used the same technique although I tended to be a lone “technical” interviewer rather than having the luxury of the PO + Dev approach that I was first exposed to but it was still my preferred approach by a wide margin.

Pairing – Interactive Interviewing

On reflection what I liked most about this approach as a candidate, compared to the traditional one, is that it felt less like an exam, which I generally suck at, and more like what you’d really do on the job. Putting aside the current climate of living in a pandemic where many people are working at home by themselves, what I liked most was that I had access to other people and was encouraged to ask questions rather than solve the problem entirely by myself. To wit, it felt like I was interviewing to be part of a team of people, not stuck in a booth and expected to working autonomously [4]. Instead of just leaving you to flounder, the interviewers would actively nudge you to help unblock the situation, just like they (hopefully) would do in the real world. Not everyone notices the same things and as long as they aren’t holding the candidate’s hand the whole time that little nudge should be seen as a positive sign about taking on-board feedback rather than failing to solve the problem. It’s another small, but I feel hugely important, part of making the candidate feel comfortable.

The Pit of Success

We’ve all heard about those interviews where it’s less about the candidate and more about the interviewer trying to show how clever they are. It almost feels like the interviewer is going out of their way to make the interview as far removed from normal operating conditions as possible, as if the pressure of an interview is somehow akin to a production outage. If your goal is to get the best from the candidate, and it should be if you want the best chance of evaluating them fairly, then you need to make them feel as comfortable as possible. You only have a short period of time with them so getting them into the right frame of mind should be utmost in your mind.

One of the problems I faced in that early programming test was an unfamiliar computer. You have a choice of whether to try and adapt to the keyboard shortcuts you’re given or reconfigure the IDE to make it more natural. You might wonder if that’s part of the test which wastes yet more time and adds to the artificial nature of the setting. What about the toolset – can you use your preferred unit testing framework or shell? Even in the classic homogenous environment that is The Windows Enterprise there is often still room for personal preference, despite what some organisations might have you believe [5].

Asking the candidate to bring their own laptop overcomes all of these hurdles and gives them the opportunity to use their own choice of tools thereby allowing them to focus more on the problem and interaction with you and less on yak shaving. They should also have access to the Internet so they can google whatever they need to. It’s important to make this perfectly clear so they won’t feel penalised for “looking up the answer” to even simple things because we all do that for real, let alone under the pressure of an interview. Letting them get flustered because they can’t remember something seemingly trivial and then also worrying about how it’ll look if they google it won’t work in your favour. (Twitter is awash with people asking senior developers to point out that even they google the simple things sometimes and that you’re not expected to remember everything all the time.)

Unfortunately, simply because there are people out there that insist on interviewing in a way designed to trip up the candidate, I find I have to go overboard when discussing the setup to reassure them that there really are no tricks – that the whole point of the exercise is to get an insight into how they work in practice. Similarly reassuring the candidate that the problem is open-ended and that solving it in the allotted is not expected also helps to relax them so they can concentrate more on enjoying the process and feel comfortable with you stopping to discuss, say, their design choices instead of feeling the need to get to the end of yet another artificial deadline instead.

The Exercise

I guess it’s to be expected that if you set a programming exercise that you’d want the candidate to complete it; but for me the exercise is a means to a different end. I’m not interested in the problem itself, it’s the conversation we have that provides me with the confidence I need to decide if the candidate has potential. This implies that the problem cannot be overly cerebral as the intention is to code and chat at the same time.

While there are a number of popular katas out there, like the Roman Numerals conversion, I never really liked any of them. Consequently I came up with my own little problem based around command line parsing. For starters I felt this was a problem domain that was likely to be familiar to almost any candidate even if they’re more GUI oriented in practice. It’s also a problem that can be solved in a procedural, functional, or object-oriented way and may even, as the design evolves, be refactored from one style to the other, or even encompass aspects of multiple paradigms. (Many of the classic katas are very functional in nature.) There is also the potential to touch on I/O with the program usage and this allows the thorny subject of mocking and testability to be broached which I’ve found to be a rich seam of discussion with plenty of opinions.

(Even though the first iteration of the problem only requires supporting “-v” to print a version string I’ve had candidates create complex class hierarchies based around the Command design pattern despite making it clear that we’ll introduce new features in subsequent iterations.)

Mechanics

Aside from how a candidate solves a problem from a design standpoint I’m also interested in the actual mechanics of how they program. I don’t mean whether they can touch type or not – I personally can’t so that would be a poor indicator :o) – no, I mean how they use the tools. For example I find it interesting what they use the keyboard or mouse for, what keyboard shortcuts they use, how they select and move text, whether they use snippets or prefer the editor not to interfere. While I don’t think any of the candidate’s choices says anything significant about their ability to solve the problem, it does provide an interesting avenue for conversation.

It’s probably a very weak indicator but programmers are often an opinionated bunch and one area they can be highly opiniated about is the tools they use. Some people love to talk about what things they find useful, in essence what they feel improves or hinders their productivity. This in turn begs the question about what they believe “productivity” is in a software development context.

Reflection

What much of this observation and conversation boils down to is not about whether they do things the same way I do – on the contrary I really hope they don’t as diversity is important – it’s about the “reflective” nature of the person. How much of what they do is through conscious choice and how much is simply the result of doing things by rote.

In my experience the better programmers I have worked with tend to more aware of how they work. While many actions may fall into the realm of unconscious competence when “in the zone” they can likely explain their rationale because they’re are still (subconsciously) evaluating it in the background in case a better approach is suitable.

(Naturally this implies the people I tend to interview are, or purport to be, experienced programmers where that level of experience is assumed to be over 10 years. I’m not sure what you can expect to take away from this post when hiring those just starting out on their journey.)

An Imperfect Process

Right back at the start I said that interviewing is an imperfect process and while I think pairing with someone is an excellent way to get a window into their character and abilities, so much still comes down to a gut feeling and therefore a subjective assessment.

I once paired with someone in an interview and while I felt they were probably technically competent I felt just a tinge of uneasiness about them personally. Ultimately the final question was “would I be happy to work with this person?” and so I said “yes” because I felt I would be nit-picking to say “no”. As it happens I did end up working with this person and a couple of months into the contract I had to have an awkward conversation with my other two colleagues to see if they felt the same way I did about this team mate. They did and the team mate was “swapped out” after a long conversation with the account manager.

What caused us to find working with this person unpleasant wasn’t something we felt could easily and quickly be rectified. They had a general air of negativity about them and had a habit of making disparaging, sweeping remarks which showed they looked down on database administrators and other non-programming roles. They also lacked an attention to detail causing the rest of us to dot their I’s and cross their T’s. Even after bringing this up directly it didn’t get any better; they really just wanted to get on and write new code and leave the other tasks like reviewing, documenting, deploying, etc. to other people.

I doubt there is anything you can do in an hour of pairing to unearth these kind of undesirable traits [6] to a level that you can adequately assess, which is why the gut still has a role to play. (I suspect it was my many years of experience in the industry working with different people that originally set my spider senses tingling.)

Epilogue

The hiring question I may find myself putting to the client is whether they would prefer to accidentally let a good candidate slip away because the interview let them (the candidate) down or accidentally hire a less suitable candidate that appeared to “walk-the-walk” as well as “talk-the-talk” and potentially become a liability. Since doing pairing interviews this question has come up very rarely with a candidate as it’s been much clearer from the pairing experience what their abilities and attitude are.

 

[1] This doesn’t just apply to hiring individuals but can also work for whole teams, see “Choosing a Supplier: The Hackathon”.

[2] See “Afterwood – The Interview” for more on how much I dislike the pen & paper approach to coding interviews.

[3] My first experience was in a Cyber Dojo evening back in September 2010 that Jon Jagger ran at Skills Matter in London. I wrote it up for the ACCU: “Jon Jagger’s Coding Dojo”.

[4] Being a long-time freelancer this mode of operation is not unexpected as you are often hired into an organisation specifically for your expertise; your contributions outside of “coding” are far less clear. Some like the feedback on how the delivery process is working while others do not and just want you to write code.

[5] My In The Toolbox article “Getting Personal” takes a look at the boundary between team conventions and personal freedom for choices in tooling and approach.

[6] I’m not saying this person could not have improved if given the right guidance, they probably could have and I hope they actually have by now; they just weren’t right for this particular environment which needed a little more sensitivity and rigour.

When Mocks Became Production Services

We were a brand new team of 5 (PM + devs) tasked with building a calculation engine. The team was just one part of a larger programme that encompassed over a dozen projects in total. The intention was for those other teams to build some of the services that ours would depend on.

Our development process was somewhat DSM-like in nature, i.e. iterative. We built a skeleton based around a command-line calculator and fleshed it out from there [1]. This skeleton naturally included vague interfaces for some of the services that we knew we’d need and that we believed would be fulfilled by some of the other teams.

Fleshing Out the Skeleton

Time marched on. Our calculator was now being parallelised and we were trying to build out the distributed nature of the system. Ideally we would like to have been integrating with the other teams long ago but the programme RAG status wasn’t good. Every other team apart from us was at “red” and therefore well behind schedule.

To compensate for the lack of collaboration and integration with the other services we needed we resorted to building our own naïve mocks. We found other sources of the same data and built some noddy services that used the file-system in a dumb way to store and serve it up. We also added some simple steps to the overnight batch process to create a snapshot of the day’s data using these sources.

Programme Cuts

In the meantime we discovered that one of the services we were to depend on had now been cancelled and some initial testing with another gave serious doubts about its ability to deliver what we needed. Of course time was marching on and our release date was approaching fast. It was fast dawning on us that these simple test mocks we’d built may well have to become our production services.

One blessing that came out of building the simple mocks so early on was that we now had quite a bit of experience on how they would behave in production. Hence we managed to shore things up a bit by adding some simple caches and removing some unnecessary memory copying and serialization. The one service left we still needed to invoke had found a more performant way for us to at least bulk extract a copy of the day’s data and so we retrofitted that into our batch preparation phase. (Ideally they’d serve it on demand but it just wasn’t there for the queries we needed.)

Release Day

The delivery date arrived. We were originally due to go live a week earlier but got pushed back by a week because an important data migration got bumped and so we were bumped too. Hence we would have delivered on time and, somewhat unusually, we were well under budget our PM said [2]. 

So the mocks we had initially built just to keep the project moving along were now part of the production codebase. The naïve underlying persistence mechanism was now a production data store that needed high-availability and backing up.

The Price

Whilst the benefits of what we did (not that there was any other real choice in the end) were great, because we delivered a working system on time, there were a few problems due to the simplicity of the design.

The first one was down to the fact that we stored each data object in its own file on the file-system and each day added over a hundred-thousand new files. Although we had partitioned the data to avoid the obvious 400K files-per-folder limit in NTFS we didn’t anticipate running out of inodes on the volume when it quickly migrated from a simple Windows server file share to a Unix style DFS. The calculation engine was also using the same share to persist checkpoint data and that added to the mess of small files. We limped along for some time through monitoring and zipping up old data [3].

The other problem we hit was that using the file-system directly meant that the implementation details became exposed. Naturally we had carefully set ACLs on the folders to ensure that only the environment had write access and our special support group had read access. However one day I noticed by accident that someone had granted read access to another group and it then transpired that they were building something on top of our naïve store.

Clearly we never intended this to happen and I’ve said more about this incident previously in “The File-System Is An Implementation Detail”. Suffice to say that an arms race then developed as we fought to remove access to everyone outside our team whilst others got wind of it [4]. I can’t remember whether it happened in the end or not but I had put a scheduled task together than would use CALCS to list the permissions and fail if there were any we didn’t expect.

I guess we were a victim of our success. If you were happy with data from the previous COB, which many of the batch systems were, you could easily get it from us because the layout was obvious.

Epilogue

I have no idea whether the original versions of these services are still running to this day but I wouldn’t be surprised if they are. There was a spike around looking into a NoSQL database to alleviate the inode problem, but I suspect the ease with which the data store could be directly queried and manipulated would have created too much inertia.

Am I glad we put what were essentially our mock services into production? Definitely. Given the choice between not delivering, delivering much later, and delivering on time with a less than perfect system that does what’s important – I’ll take the last one every time. In retrospect I wish we had delivered sooner and not waited for a load of other stuff we built as the MVP was probably far smaller.

The main thing I learned out of the experience was a reminder not to be afraid of doing the simplest thing that could work. If you get the architecture right each of the pieces can evolve to meet the ever changing requirements and data volumes [5].

What we did here fell under the traditional banner of Technical Debt – making a conscious decision to deliver a sub-optimal solution now so it can start delivering value sooner. It was the right call.

 

[1] Nowadays you’d probably look to include a slice through the build pipeline and deployment process up front too but we didn’t get any hardware until a couple of months in.

[2] We didn’t build half of what we set out to, e.g. the “dashboard” was a PowerShell generated HTML page and the work queue involved doing non-blocking polling on a database table.

[3] For regulatory reasons we needed to keep the exact inputs we had used and couldn’t guarantee on being able to retrieve them later from the various upstream sources.

[4] Why was permission granted without questioning anyone in the team that owned and supported it? I never did find out, but apparently it wasn’t the first time it had happened.

[5] Within reason of course. This system was unlikely to grow by more than an order of magnitude in the next few years.

Stand-Up and Deliver

At the ACCU Conference last year I gave a five minute lightning talk titled “The Daily Stand-Up”, which was a short comedy routine of one-liners around programming/IT/science related topics. This year I was back again, but this time I had more material so could take the opportunity to do more than one lightning talk (they have three slots – one per evening).

In the the subsequent year I’ve used my Agile on the Beach and Equal Experts Christmas party “performances” as an excuse to trawl all 14 thousand of my tweets looking for material. After separating the wheat from the chaff (or rather the needles from the haystack) I unearthed plenty enough for two sets.

It also felt as though the name “The Daily Stand-Up” had been done now and that I should try and organise the material into separate themes which I ended up doing on the way down to this year’s Pipeline conference. As a result I came up with two new titles and grouped the material appropriately, along with creating a holding slide with a suitably pithy picture.

Here then are my two sets (61 one-liners in total) for the ACCU 2016 Conference lightning talks – “Continuous Delivery” and “Becoming a Bitter Programmer” [1].

Continuous Delivery

“I visited the opticians the other day after I started seeing printers, keyboards and mice out the corner of my eye. She said it’s okay, it’s just peripheral vision.”

“We recently needed to break into my bosses account so I thought I’d try a dictionary attack – I just kept hitting the system administrator over the head with a large copy of the OED until he let me in.”

“My wife and I have been together for twenty-five years so I thought I ought to get her a token ring. Turns out you can only get 100BASE-TX these days.”

“If you think keeping up with the Kardashians is hard, you should try JavaScript frameworks!”

“I once listened to an audio book about structs that only contained values of primitive types. It was a POD cast.”

“When Sherlock Holmes talks about a ‘three pipe problem’, does he mean one that requires grep, awk, sed and sort?”

“Is the removal of a dependency injection framework from a Java codebase known as spring cleaning?”

“Was the Tower of Pisa built using lean manufacturing?”

“I know it’s all the rage these days but I reckon the writing’s on the wall for Kanban.”

“The last time I put my phone into airplane mode it promptly assumed the crash position.”

“When I turned thirty-five I blew a whole load of cash on a monster gaming rig. I think I was suffering from a half-life crisis.”

“When creating a diagram of a micro-services architecture I never know where to draw the line.”

“If we want to adopt an agile release train, does that mean we need to start using Ruby on Rails?”

“The problem with the technical debt metaphor is that in the end everyone begins to lose interest.”

“If you send data using the Kermit protocol is the transmission speed measured in ribbits per second?”

“How do you change the CMOS battery in a virtual machine?”

“For Christmas my wife bought me some jigsaws of famous British computer scientists. When she asked how I was getting on, I replied that it was Turing complete.”

“When working from home I like to get my kids involved in my coding. I call it Au Pair Programming.”

“My son was being hassled by his school friends to share his music collection via BitTorrent. I said he shouldn’t give in to peer-to-peer pressure.”

“Our local pub has started selling a new draught beer called Git. The only way to get served is to make a pull request.”

“When they collected their Turing Award I recognised Diffie on the left and Hellman on the right, but who was the man in the middle?”

“I decided it was about time I upgraded to fibre, so I’ve started eating All Bran for breakfast.”

“If you’re looking to purchase some recreational drugs do you first consult Trip Advisor?”

“Should a microphone type be mutable?”

“When software developers work at New Scotland Yard do they have to use a special branch?”

“I recently visited the dentist and he told me I had a scaling problem. I said that’s a problem as I don’t have room for any more teeth.”

“We recently had a poll for the best escape character. I picked Steve McQueen’s Hicks.”

“I’m never quite sure, is it Heisenbergs uncertainty principle?”

“I was utterly convinced that I had found the answer to my SSL problem. In fact I thought it was a dead cert.”

“When writing software for estate agents, should you use property based testing?”

“The other day I struggled to upload a picture of Marcel Marceau. Every time I tried the server returned the error: 415 unsupported mime type.”

“Probably the hardest problem in computer science is dealing with nans. They always seem to want you to come round and fix their computer.“

Becoming a Bitter Programmer

“Is a cross functional team just a bunch of grumpy Lisp programmers?”

“I’ve given up playing poker with functional programmers, they just spend all their time folding.”

“My company recently adopted mob programming. They hired a bunch of beefy guys to stand around with baseball bats to make sure you did an 80 hour week.”

“Some people are like the char type in C++, they seem to get promoted for no apparent reason.”

“Is it any wonder modern programmers are obese when they depend so heavily on syntactic sugar?”

“The product owner asked me why all our Cucumber tests only covered the happy paths. I told him they’re rose tinted specs.”

“Every time I try and generate a digital signature I just make a hash of it.”

“C++ comes with complexity guarantees – if you use C++ it’s guaranteed to be complex.”

“Some people say C# & Java programmers overuse reflection, but given the quality of their code I’d say they’re not reflecting enough.”

“Talking of C# & Java, is it just me or are they harder to distinguish these days than The Munsters and The Addams Family?”

“Our team isn’t very good at this agile stuff – our successes are just stories and our failures are all epics.”

“Anyone who says you shouldn’t use synchronisation objects is just being racist.”

“Whenever I find a static in multi-threaded code it makes my hair stand on end.”

“If you don’t keep your promises, how will you know what the future holds?”

“Our zero tolerance approach to floating-point comparison errors didn’t work out quite as we’d hoped.”

“If technical debt is the result of poor quality code, does that make bad programmers loan sharks?”

“Don’t bother upgrading your database, the SQL is never as good as the original.”

“I blame Facebook for the declining quality of SQL code, young programmers are obsessed with likes.”

“My current system has five-nines reliability - it usually works about 45% of the time.”

“Our DR approach is less active/passive and more passive/aggressive. When anything fails we just sit around loudly tutting until someone goes and fixes it.”

“The company said it was moving all our hardware to EC2. It turned out they’d bought a dingy office in London near the Bank of England.”

“Some people just don’t know when to stop bashing Windows.”

“When I found out my new job was effectively working with legacy code I started spitting Feathers!”

“I reckon my team has taken a hypocritic oath – the comments say one thing but the code does something completely different.”

“You can always tell when you’ve got a Christmas or birthday present from an enterprise software developer – there is an excessive amount of wrapping.”

“C# supports impure methods and mutable types, but doesn’t support tail recursion or currying. Does that make it a dysfunctional programming language?”

“Finding the square root of a negative number is just not natural.”

“My team is using homeopathic unit testing. They write one test and dilute it with a thousand lines of production code.”

“The Wildlife Trust has declared our codebase a conservation area on account of the number of bugs.”

[1] The latter title was inspired by the recent book from the compere of the ACCU Conference lightning talks – Pete Goodliffe – which is called Becoming a Better Programmer.

Choosing a Supplier: The Hackathon

One would hope in this day-and-age that when hiring a candidate for a role as a developer the interview process would include some element of actually doing what it is we do - programming. This could be as simple as submitting an offline coding test, but better yet a face-to-face session where you pair with one of the people you’re likely to work with. Either way, just as an artist would expect to show their portfolio, we should expect to do something similar – not just write a CV and then talk about it.

If that’s the case for hiring a single developer, why shouldn’t choosing a software supplier follow a similar process, after all there is probably far more money at stake. And that’s really what the whole interview process is all about – it’s a form of risk management – you’re trying to find a supplier that you’re confident is going to deliver what you really need in a timely manner and for an acceptable cost.

Show Your Workings

The Hackathon, which probably sounds a bit too hipster for some people’s taste, is the embodiment of this idea – get the potential suppliers to show how they actually produce software, albeit on  a very small scale. My last two days have just been spent on one of these and it was great fun, if a little nerve-wracking.

The consultancy firm I’m currently working through (Equal Experts) has been involved in one of these before, quite recently, and they relish the chance to do this kind of selection process more as it plays to their strengths – delivering working software incrementally using modern development techniques [1].

By all accounts this particular one was slightly different to the previous one, although it still followed a similar outline. There are a number of very small teams (3) representing the different suppliers. Each team has a small number of people (~4) that covers whatever skills they think they’ll need based on the brief. In our case we had 3 devs and a UX person, but obviously we could also act in a lesser capacity as a BA, QA, Ops, etc. too to cover all the bases.

The overall structure was that we would spend a short period learning a bit about the company we were pitching to and about the problem they wanted us to tackle. Each team was then given a separate “war” room in which they could work for roughly two days before presenting back at the end.

Where in the previous Hackathon they got more freedom about what they wanted to tackle (i.e. a higher-level problem statement) this one had a more specific problem to be solved. The problem was also directly related to the actual problem they chosen supplier would eventually be asked to work on, which makes sense.

During the two days various people involved in the selection process would come round and visit us to see what we’d been up to and that would also give us an opportunity to ask any questions we had about the problem. If we really needed to we could have called upon The Business at any time to come and help us but their visits were frequent enough that it meant we never needed to go that far.

Our Approach

Naturally I can’t say anything about the problem itself, but suffice to say that it was far bigger than anything we could expect to build in just two days. However that didn’t stop us tackling it in the same way we would a real project. We did some up-front analysis to explore the problem domain a fair bit, make some key architectural decisions to decide what we were going to build, create a backlog with some stories on it and then start coding. We also littered the walls with sheets of magic whiteboard [2], post-it notes and index cards that covered our questions, answers, architecture, backlog, etc.

We were able to to do a little bit of work beforehand, such as creating a GitHub repo that we could all check we had access to, along with a dummy build in AppVeyor to cover the CI part. The guest Wi-Fi wasn’t brilliant [3] which meant pushing and pulling to/from GitHub was a bit laggy, but it was usable and the cloud based CI monitor was stable enough.

Despite it being unofficially described as a “Hackathon” we decided we would do the best we could to show how we worked in practice, rather than try and push out as much code as possible. Whilst we no doubt could have got more code working if we had cut corners on some of the other parts of process (e.g. analysis or testing) we would not have given a fair representation of what we do (IMHO). I’ve touched on this before in “Keeping the Faith Under Pressure” and I’m pleased with what we produced. I’m perfectly happy to call the code I wrote over the two days “production code”.

Day One

After doing our early analysis we started putting together the walking skeleton which was a text-book example of Conway’s Law as we initially kept out of each other’s way whilst we got the boilerplate stuff together. Of course this separation didn’t last long as by the end of the day the temporary assemblies and folders had all gone in a simple refactoring exercise so that we had a coherent codebase to build from.

We finished the day with our technology stack up and running (browser-based client + REST API), watched over by AppVeyor, and a very thin slice of functionality in play which we demoed to a couple of the stakeholders.

Day Two

The following day was essentially a few hours shorter to allow time for each team to present to the panel, that also meant we’d need to factor in some time to put the presentation together. In essence we only had ~5 hours to implement the next set of features and so got one of the stakeholders in first-thing to make a priority call so we knew where to focus our efforts during the morning.

During the time we three devs were building a working product our UX expert was building a clickable prototype to help explore the problem further ahead. This had the benefit of us being able to see the context in which the data was used and therefore we could better understand the bigger picture. In such a short timeframe it was perhaps difficult to see what beneficial effect it had on our design and thinking but what it did show clearly was how we work and how important we believe UX to be to the development process in general.

We stopped coding an hour before our presentation slot to give ourselves plenty of time to think about what we needed to say and show. Our UX expert had surreptitiously taken plenty of photos of us engaging with the stakeholders along with the evolving boards and backlog so that we could put together a compelling story about our approach.

We talked for 30 minutes about the process and architecture, and gave a demo of what we’d built to date. We then had about 15 minutes to answer questions from the panel, both about our approach and how we might tackle some of the questions we hadn’t yet answered in the code.

Each team got to present to the panel in isolation, but we all hung around until the end at which point we each did a very brief version of the presentation to the other teams. This was really interesting as up to that point we had no idea what the others were up to. For example we chose a thin-client whereas the other two chose thick-clients. We used post-its and a makeshift whiteboard for our notes and product backlog whilst another used an online tool and the third didn’t mention their approach.

Wrapping Up

Did the exercise achieve what it set out to do? As I’m not the client I have no idea what their final decision is or whether the eventual product has met their needs, because clearly it hasn’t been built yet. But I believe their behaviour suggested that they were pretty pleased with what they had seen from everyone. I think they seemed surprised that the three teams had behaved quite differently and so they probably got a lot more out of the exercise than anticipated as each team would probably have asked different questions and explored different avenues. Given that this was a real problem we were working on I’m sure there is a lot of value in that alone.

Personally I went into the process somewhat nervous. My current role is a real departure for me - it’s not a hands-on development role. As such I thought I might not be “match fit” even with 20 years of coding experience behind me. What I forgot though was that the point of the process was to be ourselves and write code as I would for real and that just came back naturally. Hopefully I added an equal amount of value to the team and gave as good an account of myself as the others appeared to.

So, is this is the way I think tenders should be done? Yes, but I’m just a developer :o). I did ask someone at Equal Experts about how it compared cost-wise for them given that the RFP (Request for Proposal) process can also be pretty time consuming too and he suggested it wasn’t that far off. Not having done any others I can’t say whether the client had a disproportionate number of people involved at their end but given the potential sums at stake I’m sure they saw it as a good investment. I certainly hope more companies do it in the future.

[1] Apologies if that sounded like a thinly-veiled sales pitch, it wasn’t mean to be. I was just trying to describe how they see themselves; I’m an associate, not an employee.

[2] Somewhat unfortunately the room we where in had frosted glass and the sheets of magic whiteboard didn’t really stick to it. Without any blu-tack or sellotape we had to use even more post-it notes to hold up the whiteboards!

[3] The Wi-Fi dropped out every now and then and had limited bandwidth which meant that any image-heavy web pages could take a while to load.

Should You Mention Your Bio When Speaking?

The day before I was due to speak at the Agile on the Beach conference a re-tweet entered my timeline suggesting that mentioning your biography during your talk was pointless. The premise seemed to be that people came to hear your content, not your life story. And anyway they can see your bio in the programme so just “get on with it already”.

I’m not sure exactly what had happened to cause that tweet but I found myself reflecting on part of my own talk just hours before I was due to give it, which was mightily uncomfortable. I reasoned that perhaps what they had seen was someone dedicating a significant amount of time, say 10 minutes, to reeling off their CV. If that was the case then I wholeheartedly agree with the sentiment. However, in case what they did was only a minute or so of back-story, then I’m going to dispense my $0.02 on the topic, for what it’s worth...

Context is everything, especially when you’re trying to understand what the presenter’s trying to say and how they may have come to their conclusions. I feel my Test-Driven SQL talk requires a bit more background that the other talks I have given in the last because I am not normally a part of the community for which it is intended, or from the community which you might normally find someone to present it - I’m a C++/C# application programmer by trade, not a SQL developer or DBA. The entire basis of the talk is about how I came to apply the same set of principles from my normal programming endeavours to the world of the relational database. Although what I say makes sense to me and my colleagues (who also apply it), I know it may seem unnatural to a native from that side of the software development fence.

In a world where one size never fits all it is the constraints (or lack of them) on the speaker’s subject matter that allows us to put the content they are presenting into focus. And as such we can either identify ourselves with those constraints and therefore become more attentive, or realise it comes from a different world (or a least one we are unlikely to face in the shorter term) and just enjoy the talk as another bit of background knowledge to bank for when the relevant time comes.

For example, a common disparity exists between the start-up and enterprise cultures. Anyone from a garage start-up reading my recent “Developer Freedom” article will probably wonder what all the brouhaha is about. Similarly they may also ignore any notion of formalising a SQL-based public interface to their database because time-to-market matters, and anyway they only have a single product so there aren’t any “data leeches” to worry about.

Like all headline grabbing tweets the truth is no doubt much murkier than it appears on the surface. Personally I welcome some relevant knowledge about a speaker’s background in the talk as I’m generally too lazy to read all the abstracts and bio’s before picking a talk to listen to, at least at a conference. However, I feel that the information must provide some useful context for the presentation they are about to undertake to be worthwhile.

Top 10 Posts So Far

With my blog approaching its 4th anniversary I feel it’s reached the critical mass of posts and time needed before I can look at what the most popular posts have been to date. I know there is a side-bar on the right at the moment displaying this, but I thought it would be useful to capture it so that I can look back in another 4 years and see whether I managed to add anything new or whether it just went downhill from then on.

One site I worked at introduced a somewhat draconian web content filtering policy. One particular category of web sites they blocked were “blogs and personal pages”. I pointed out to the security team that blogs are the modern day Knowledge Base; they are relied on heavily by IT to get their work done. Yes, you have your MSDNs and StackOverflows, but they just aren’t enough. And anyway some people only post answers that contain a link to a blog post they found that answers the original query.

So, looking at the list of most popular posts of mine to date, it’s clearly not the diatribes or random musings that people come across, but those that solve people’s everyday problems. Well, all except for number 10 that is. That anomaly has almost certainly come about as a consequence of generous publicity via Twitter by some of the ACCU’s more followed members[*].

1. PowerShell, Throwing Exceptions & Exit Codes
2. Cleaning up svn:mergeinfo Droppings
3. Simple Database Build & Deployment With SQLCMD
4. Integration Testing with NUnit
5. WCF Weirdy – NetTcpBinding, EndpointIdentity and Programmatic Configuration
6. WCF Service Refusing Connections
7. SQL Server Cursors - Avoiding the Duplicate FETCH NEXT
8. Using Visual C++ Together With GCC - Some Alternatives
9. C#/SQL Integration Testing With NUnit
10. TODO or TODO Not - There Is No Later

 

[*] Pleasing though it was to get the re-tweets of my original posting announcement, I suspect the 2 additional plugs from @KevlinHenney didn’t do any harm either :-).

OK & Apply - A Failed UI Experiment?

As you may have guessed from my previous two posts, I have been paying far more attention than usual to how other IT professionals[*] work. The most recent one that caught my eye and has no doubt been done to death in UI circles is the OK and Apply buttons on dialogs. It’s funny to watch people first push the Apply button and then the OK one. The question is whether they don’t realise that pressing OK has addition affect of applying the changes, or that they don’t trust that just pressing OK is enough? I’m sure it’s the latter - a belt and braces approach.

I know how they are supposed to operate, but I suspect that’s only because I was doing a lot of UI work when Windows 95[#] appeared and so I was paid to know all the UX guidelines. Perhaps it’s time to take a trip down memory lane and visit the “Interface Hall of Shame” along with a quick Google search to find out if/when this UI faux pas was officially deprecated...

 

[*] I’m always looking to learn new tricks and techniques to be more effective and other IT professionals is obviously as good a place as any to look as this is (supposedly) their bread-and-butter.

[#] and Windows NT 3.5 with the shell upgraded to 4.0.