Showcasing APIs – The Demo Site

Developers that work on front-ends have it easy [1]. When it comes to doing a sprint demo or showcasing your work you’ve got something glitzy to talk about. In contrast writing a service API is remarkably dull by comparison.

At the bottom end of the spectrum you have got good old reliable CURL, if you want a command-line feel to your demo. Just above this, but also still fairly dry by comparison with a proper UI, is using a GUI based HTTP tool such as the Postman or Advanced REST Client extensions for Chrome. Either way it’s still pretty raw. Even using tools like Swagger still put you in the mind-set of the API developer rather than the consumer.

If you’re writing a RESTful API then it’s not much effort to write a simple front-end that can invoke your API. But there’s more to it than just demoing which “feature” of an API you delivered, it has other value too.

Client Perspective

One of the hardest things I’ve certainly found working on service APIs is how you elevate yourself from being the implementer to being the consumer. Often the service I’m working on is one piece of a bigger puzzle and the front-end either doesn’t exist (yet) or is being developed by another team elsewhere so collaboration is sparse.

When you begin creating an API you often start by fleshing out a few basic resource types and then begin working out how the various “methods” work together. To do this you need to start thinking about how the consumer of the API is likely to interact with it. And to do that you need to know what that higher level use cases are (i.e. the bigger picture).

This process also leads directly into the prioritisation of what order the API should be built. For instance it’s often assumed the Minimum Viable Product (MVP) is “the entire API” and therefore order doesn’t matter, but it matters a lot as you’ll want to build stuff your consumers can plug-in as soon as it’s ready (even just for development purposes). Other details may not be fleshed out and so you probably want to avoid doing any part of it which is “woolly”, unless it’s been prioritised specifically to help drive out the real requirements or reduce technical risk.

A simple UI helps to elevate the discussion so that you don’t get bogged down in the protocol but start thinking about what the goals are that the customer wants to achieve. Once you have enough of a journey you can start to actually play out scenarios using your rough-and-ready UI tool instead of just using a whiteboard or plugging in CURL commands at the prompt. This adds something really tangible to the conversation as you begin to empathise with the client, rather than being stuck as a developer.

This idea of consuming of your own product to help you understand the customers point of view is known as “eating your own dog food”, or more simply “dog-fooding”. This is much easier when you’re developing an actual front-end product rather than a service API, but the demo UI certainly helps you get closer to them.

Exploratory Testing

I’ll be honest, I love writing tools. If you’ve read “From Test Harness To Support Tool” you’ll know that I find writing little tools that consume the code I’ve written in ways other than what the automated tests or end user needs is quite rewarding. These usually take the form of test tools (commonly system testing tools [2]), and as that post suggests they can take on a life later as a support or diagnostic tool.

Given that the remit of the demo site is to drive the API, it can therefore also play the role of a testing tool. Once you start having to carry over data from one request to the next, e.g. security token, cookie, user ID, etc. it gets tedious copying and pasting them from one request to the next. It’s not uncommon to see wiki pages written by developers that describe how you can do this manually by copy-and-pasting various steps. But this process is naturally rather error prone and so the demo site brings a little automation to the proceedings. Consequently you are elevated from the more tedious aspects of poking the system to focus on the behaviour.

This also gels quite nicely with another more recent post “Man Cannot Live by Unit Testing Alone” as I have used this technique to find some not-insignificant gaps in the API code, such as trivial concurrency issues. Whilst they would probably have been found later when the API was driven harder, I like to do just a little bit of poking around in the background to make sure that we’ve got the major angles covered early on when the newer code is fresher in your mind.

This can mean adding things to your mocks that you might feel is somewhat excessive, such as ensuring any in-memory test data stores (i.e. dictionaries) are thread-safe [3]. But it’s a little touch that makes exploratory testing easier and more fruitful as you don’t get blasé and just ignore bugs because you believe the mocks used by the API are overly simple. If you’re going to use a bit of exploratory testing to try and sniff out any smells it helps if the place doesn’t whiff to begin with.

3rd Party Test UI

One of the problems of being just one of many teams in the corporate machine is that you have neighbours which you need to play nicely with. In the good old days everyone just waited until “the UAT phase” to discover that nothing integrated correctly. Nowadays we want to be collaborating and integrating as soon as possible.

If I’m building an API then I want to get it out there in the hands of the consumers as soon as possible so that we can bottom out any integration issues. And this isn’t just about fixing minor issues in the contract, but they should have input into what shape it should take. If the consumers can’t easily consume it, say for technical reasons, then we’ve failed. It’s too easy to get bogged down in a puritanical debate about “how RESTful” an API is instead of just iterating quickly over any problems and reaching a style that everyone can agree on.

The consumers of the service naturally need the actual API they are going to code against, but having access to a test UI can be useful for them too. For example it can give them the power to play and explore the API at a high-level as well, potentially in ways you never anticipated. If the service is backed by some kind of persistent data store the test UI can often provide a bit of extra functionality that avoids them having to get your team involved. For example if your service sends them events by a queue, mocking and canned data will only get them so far. They probably want you to start pushing real events from your side so that they can see where any impedance mismatches lie. If they can do this themselves it decouples their testing from the API developers a little more.

Service Evangelism

When you’re building a utility service at a large company, such as something around notifications, you might find that the product owner needs to go and talk to various other teams to help showcase what you’re building. Whilst they are probably already in the loop somewhere having something tangible to show makes promoting it internally much easier. Ideally they would be able to give some immediate feedback too about whether the service is heading in a direction that will be useful to them.

There is also an element of selling to the upper echelons of management and here a showcase speaks volumes just as much as it does to other teams. It might seem obvious but you’d be amazed at how many teams struggle to easily demonstrate “working software” in a business-oriented manner.

How Simple Should It Be?

If you’re going to build a demo UI for your service API you want to keep it simple. There is almost certainly less value in it than the tests so you shouldn’t spend too long making it. However once you’ve got a basic demo UI up you should find that maintaining it with each story will be minimal effort.

In more than one instance I’ve heard developers talk of writing unit tests for it, or they start bringing in loads of JavaScript packages to spice it up. Take a step back and make sure you don’t lose sight of why you built it in the first place. It needs to be functional enough to be useful without being too glitzy that it detracts from it’s real purpose, which is to explore the API behaviour. This means it will likely be quite raw, e.g. one page per API method, with form submissions to navigate you from one API method to the next to ease the simulation of user journeys. Feel free to add some shortcuts to minimise the friction (e.g. authentication tokens) but don’t make it so slick that the demo UI ends up appearing to be the product you’re delivering.

[1] Yes, I’m being ironic, not moronic.

[2] I’ve worked on quite a few decent sized monolithic systems where finding out that’s going on under the hood can be pretty difficult.

[3] Remember your locking can be as coarse as you like because performance is not an issue and the goal is to avoid wasting time investigating bugs due to poorly implemented mocks.

My Kingdom for a C# Typedef

Two of the things I’ve missed most from C++ when writing C# are const and typedef. Okay, so there are a number of other things I’ve missed too (and vice-versa) but these are the ones I miss most regularly.

Generic Types

It’s quite common to see a C# codebase littered with long type names when generics are involved, e.g.

private Dictionary<string, string> headers;

public Constructor()
{
  headers = new Dictionary<string, string>();
}

Dictionary<string, string> FilterHeaders(…)
{
  return something.Where(…).ToDictionary(…);
}

The overreliance on fundamental types in a codebase is a common smell known as Primitive Obsession. Like all code smells it’s just a hint that something may be out of line, but often it’s good enough to serve a purpose until we learn more about the problem we’re trying to solve (See “Refactoring Or Re-Factoring”).

The problem with type declarations like Dictionary<string, string> is that, as their very name suggests, they are too generic. There are many different ways that we can use a dictionary that maps one string to another and so it’s not as obvious which of the many uses applies in this particular instance.

Naturally there are other clues to guide us, such as the names of the class members or variables (e.g. headers) and the noun used in the method name (e.g. FilterHeaders), but it’s not always so clear cut.

The larger problem is usually the amount of duplication that can occur both in the production code and tests as a result of using a generic type. When the time comes to finally introduce the real abstraction you were probably thinking of, the generic type declaration is then littered all over the place. Duplication like this is becoming less of an issue these days with liberal use of var and good refactoring tools like ReSharper, but there is still an excess of visual noise in the code that distracts the eye.

Type Aliases

In C++ and Go (amongst others) you can help with this by creating an alias for the type that can then be used in it’s place:

typedef std::map<std::string, std::string> headers_t;

headers_t headers; 

There is a special form of using declaration in C# that actually comes quite close to this:

using Headers = Dictionary<string, string>;

private Headers headers;

public Constructor()
{
  headers = new Headers();
}

Headers FilterHeaders(…)
{
  return something.Where(…).ToDictionary(…);
}

However it suffers from a number of shortcomings. The first is that it’s clearly not a very well known construct – I hardly ever see C# developers using it. But that’s just an education problem and the reason I’m writing this, right?

The second is that by leveraging the “using” keyword it seems to mess with tools like ReSharper that want to help manage all your namespace imports for you. Instead of keeping all the imports at the top whilst keeping alias style using’s separately below it just lumps them all together, or sees the manual ones later and just appends new imports after them. Essentially they don’t see a difference between these two different use cases for “using”.

A third problem with this case of “using” though is that you cannot compose them. This means you can’t use one alias inside another. For example, imagine that you wanted to provide a cache of a user’s full name (first & last) against some arbitrary user ID, you wouldn’t be able to do this:

using FullName = Pair<string, string>;
using NameCache = Dictionary<int, FullName>;

It’s not unusual to want to compose dictionaries with other dictionaries or lists and that’s the point at which the complexity of the type declaration really takes off.

Finally, and sadly the most disappointing aspect, is that the scope of the alias is limited to just the source file where it’s defined. This means that you have to redefine the alias wherever it is used if you want to be truly consistent. Consequently I’ve tended to stick with only using this technique for the implementation details of a class where I know the scope is limited.

Inheritance

One alternative to the above technique that I’ve seen used is to create a new class that inherits from the generic type:

public class Headers : Dictionary<string, string>
{ }

This is usually used with collections and can be a friction-free approach in C# due to the way that the object-initialization syntax relies on the mutable Add() method:

var header = new Headers
{
  { “location”, “http://chrisoldwood.com” },
};

However if you wanted to use the traditional constructor syntax you’d have to expose all the base type constructors yourself manually on the derived type so that they can chain through to the base.

What makes me most uncomfortable about this approach though is that it uses inheritance, and that has a habit of leading some developers down the wrong path. By appearing to create a new type it’s tempting to start adding new behaviours to it. Sometimes these behaviours will require invariants which can then be violated by using the type through it’s base class (i.e. violate the Liskov Substitution Principle).

Domain Types

Earlier on I suggested that this is often just a stepping stone on the way to creating a proper domain type. The example I gave above was using a Pair to store a person’s first and last name:

using FullName = Pair<string, string>;

Whilst this might be palatable for an implementation detail it’s unlikely to be the kind of thing you want to expose in a published interface as the abstraction soon leaks:

var firstName = fullName.First;
var lastName = fullName.Second;

Personally I find it doesn’t take long for some generics like Pair and Tuple to leave a nasty taste and so I quickly move on to creating a custom type for the job (See “Primitive Domain Types - Too Much Like Hard Work?”):

public class FullName
{
  public string FirstName { get; }
  public string LastName { get; }
}

This is another one of those areas where C# is playing catch-up with the likes of F#. Creating simple immutable data types like these are the bread-and-butter of F#’s Record type; you can do it with anonymous types in C# but not easily with named types.

Strongly-Typed Aliases

One other aside before I sign off about typedefs is that in C, C++, and the “using” approach in C#, they are not strongly-typed types, they are just simple aliases. Consequently you can freely mix-and-match the alias with the underlying type:

var headers = new Headers();
Dictionary<string, string> dictionary = headers;

In C++ you can use templates to workaround this [1] but Go goes one step further and makes the aliases more strongly-typed. This means that there is no implicit conversion even when the fundamental type is the same, which makes them just a little bit more like a real type. Once again F# with its notion of units of measure also scores well on this front.

[1] The first article I remember about this was Matthew Wilson’s “True Typedefs”, but the state of the art has possibly moved on since then. The Boost template metaprogramming book also provides a more extensive discussion on the topic.

The Curse of NTLM Based HTTP Proxies

I first crossed swords with an NTLM authenticating proxy around the turn of the millennium when I was working on an internet based real-time trading system. We had chosen to use a Java applet for the client which provided the underlying smarts for the surrounding web page. Although we had a native TCP transport for the highest fidelity connection when it came to putting the public Internet and, more importantly, enterprise networks between the client and the server it all went downhill fast.

It soon became apparent that Internet access inside the large organisations we were dealing with was hugely different to that of the much smaller company I was working at. In fact it took another 6 months to get the service live due to all the workarounds we had to put in place to make the service usable by the companies it was targeted at. I ended up writing a number of different HTTP transports to try and get a working out-of-the-box configuration for our end users as trying to get support from their IT departments was even harder work. On Netscape it used something based on the standard Java URLConnection class, whilst on Internet Explorer it used Microsoft’s J/Direct Java extension to leverage the underlying WinInet API.

This last nasty hack was done specifically to cater for those organisations that put up the most barriers between its users and the Internet, which often turned out to be some kind of proxy which relied on NTLM for authentication. Trying to rig something similar up at our company to develop against wasn’t easy or cheap either. IIRC in the end I managed to get IIS 4 (with MS Proxy Server?) to act as an NTLM proxy so that we had something to work with.

Back then the NTLM protocol was a propriety Microsoft technology with all the connotations that comes with that, i.e. tooling had to be licensed. Hence you didn’t have any off-the-shelf open source offerings to work with and so you had a classic case of vendor lock-in. Essentially the only technologies that could (reliably) work with an NTLM proxy were Microsoft’s own.

In the intervening years the need for both clients (though mostly just the web browser) and servers have required more and more access to the outside world, both for the development of the software itself, and it’s ultimate presence through a move from on-premises to cloud based hosting.

Additionally the NTLM protocol was reverse engineered and tools and libraries started to appear (e.g. Cntlm) that allowed you to work (to a degree) within this constraint. However this appears to have sprung from a need in the non-Microsoft community and so support is essentially the bare minimum to get you out of the building (i.e. manually presenting a username and password).

From a team collaboration point of view tools like Google Docs and GitHub wikis have become more common as we move away from format-heavy content, and Trello for a lightweight approach to managing a backlog. Messaging in the form of Skype, Google Hangouts and Slack also play a wider role as the number of people outside the corporate network, not only due to remote working, but also to bring strategic partners closer to the work itself.

The process of developing software, even for Microsoft’s own platform, has grown to rely heavily on instant internet access in recent years with the rise of The Package Manager. You can’t develop a non-trivial C# service without access to NuGet, or a JavaScript front-end without Node.js and NPM. Even setting up and configuring your developer workstation or Windows server requires access to Chocolatey unless you prefer haemorrhaging time locating, downloading and manually installing tools.

As the need for more machines to access the Internet grows, so the amount of friction in the process also grows as you bump your head against the world of corporate IT. Due to the complexity of their networks, and the various other tight controls they have in place, you’re never quite sure which barrier you’re finding yourself up against this time.

What makes the NTLM proxy issue particularly galling is that many of the tools don’t make it obvious that this scenario is not supported. Consequently you waste significant amounts of time trying to diagnose a problem with a product that will never work anyway. If you run out of patience you may switch tact long before you discover the footnote or blog post that points out the futility of your efforts.

This was brought home once again only recently when we had developed a nice little tool in Go to help with our deployments. The artefacts were stored in an S3 bucket and there is good support for S3 via the AWS Go SDK. After building and testing the tool we then proceeded to try and work our why it didn’t work on the corporate network. Many rabbit holes were investigated, such as double and triple checking the AWS secrets were set correctly, and being read correctly, etc. before we discovered an NTLM proxy was getting in the way. Although there was a Go library that could provide NTLM support we’d have to find a way to make the S3 library use the NTLM one. Even then it turned out not to work seamlessly with whatever ambient credentials the process was running as so pretty much became a non-starter.

We then investigated other options, such as the AWS CLI tools that could we then script, perhaps with PowerShell. More time wasted before again discovering that NTLM proxies are not supported by them. Finally we resorted to using the AWS Tools for PowerShell which we hoped (by virtue of them being built using Microsoft’s own technology) would do the trick. It didn’t work out of the box, but the Set-AWSProxy cmdlet was the magic we needed and it was easy find now we knew what question to ask.

Or so we thought. Once we built and tested the PowerShell based deployment script we proceeded to invoke it via the Jenkins agent and once again it hung and eventually failed. After all that effort the “service account” under which we were trying to perform the deployment did not have rights to access the internet via (yes, you guessed it) the NTLM proxy.

This need to ensure service accounts are correctly configured even for outbound only internet access is not a new problem, I’ve faced it a few times before. And yet every time it shows up it’s never the first thing it think of. Anyone who has ever had to configure Jenkins to talk to private Git repos will know that there are many other sources of problem aside from whether or not you can even access the internet.

Using a device like an authenticating proxy has that command-and-control air about it; it ensures that the workers only access what the company wants them to. The alternate approach which is gaining traction (albeit very slowly) is the notion of Trust & Verify. Instead of assuming the worst you grant more freedom by putting monitoring in place to ensure people don’t abuse their privilege. If security is a concern, and it almost certainly is a very serious one, then you can stick a transparent proxy in between to maintain that balance between allowing people to get work done whilst also still protecting the company from the riskier attack vectors.

The role of the organisation should be to make it easy for people to fall into The Pit of Success. Developers (testers, system administrators, etc.) in particular are different because they constantly bump into technical issues that the (probably somewhat larger) non-technical workforce (that that policies are normally targeted at) do not experience on anywhere near the same scale.

This is of course ground that I’ve covered before in my C Vu article Developer Freedom. But there I lambasted the disruption caused by overly-zealous content filtering, whereas this particular problem is more of a silent killer. At least a content filter is pretty clear on the matter when it denies access – you aren’t having it, end of. In contrast the NTLM authenticating proxy first hides in the ether waiting for you to discover its mere existence and then, when you think you’ve got it sussed, you feel the sucker punch as you unearth the footnote in the product documentation that tells you that your particular network configuration is not unsupported.

In retrospect I’d say that the NTLM proxy is one of the best examples of why having someone from infrastructure in your team is essential to the successful delivery of your product.

The Tortoise and the Hare

[This was my second contribution to the 50 Shades of Scrum book project. It was also written back in late 2013.]

One of the great things about being a parent is having time to go back and read some of the old parables again; this time to my children. Recently I had the pleasure of re-visiting that old classic about a race involving a tortoise and a hare. In the original story the tortoise wins, but surely in our modern age we'd never make the same kinds of mistakes as the hare, would we?

In the all-too-familiar gold rush that springs up in an attempt to monetize any new, successful idea a fresh brand of snake oil goes on sale. The particular brand on sale now suggests that this time the tortoise doesn't have to win. Finally we have found a sure-fire way for the hare to conquer — by sprinting continuously to the finish line we will out-pace the tortoise and raise the trophy!

The problem is it's simply not possible in the physical or software development worlds to continue to sprint for long periods of time. Take Usain Bolt, the current 100m and 200m Olympic champion. He can cover the 100m distance in just 9.63 secs and the 200m in 19.32 secs. Assuming a continuous pace of 9.63 secs per 100m the marathon should have been won in ~4063 secs, or 1:07:43. But it wasn't. It was won by Stephen Kiprotich in almost double that — 2:08:01. Usain Bolt couldn't maintain his sprinting pace even if he wanted to over a short to medium distance, let alone a longer one.

The term "sprint" comes loaded with expectations, the most damaging of which is that there is a finish line at the end of the course. Rarely in software development does a project start and end in such a short period of time. The more likely scenario is that it will go on in cycles, where we build, reflect and release small chunks of functionality over a long period. This cyclic nature is more akin to running laps on a track where we pass the eventual finishing line many times (releasing) before the ultimate conclusion is reached (decommissioning).

In my youth my chosen sport was swimming; in particular the longer distances, such as 1500m. At the start of a race the thought of 16 minutes of hard swimming used to fill me with dread. So I chose to break it down into smaller chunks of 400m, which felt far less daunting. As I approached the end of each 400m leg I would pay more attention to where I was with respect to the pace I had set, but I wouldn't suddenly sprint to the end of the leg just to make up a few seconds of lost time. If I did this I’d start the next one partially exhausted which really upsets the rhythm, so instead I’d have to change my stroke or breathing to steadily claw back any time. Of course when you know the end really is in sight a genuine burst of pace is so much easier to endure.

I suspect that one of the reasons management like the term "sprint" is for motivation. In a more traditional development process you may be working on a project for many months, perhaps years before what you produce ever sees the light of day. It’s hard to remain focused with such a long stretch ahead and so breaking the delivery down also aids in keeping morale up.

That said, what does not help matters is when the workers are forced to make a choice about how to meet what is an entirely arbitrary deadline – work longer hours or skimp on quality. And a one, two or three week deadline is almost certainly exactly that — arbitrary. There are just too many daily distractions to expect progress to run smoothly, even in a short time-span. This week alone my laptop died, internet connectivity has been variable, the build machine has been playing up, a permissions change refuses to have the desired effect and a messaging API isn't doing what the documentation suggests it should. And this is on a run-of-the-mill in-house project only using supposedly mature, stable technologies!

Deadlines like the turn of the millennium are real and immovable and sometimes have to be met, but allowing a piece of work to roll over from one sprint to the next when there is no obvious impediment should be perfectly acceptable. The checks and balances should already be in place to ensure that any task is not allowed to mushroom uncontrollably, or that any one individual does not become bogged down or "go dark". The team should self-regulate to ensure just the right balance is struck between doing quality work to minimise waste whilst also ensuring the solution solves the problem without unnecessary complexity.

As for the motivational factors of "sprinting" I posit that what motivates software developers most is seeing their hard work escape the confines of the development environment and flourish out in production. Making it easy for developers to continually succeed by delivering value to users is a far better carrot-and-stick than "because it's Friday and that's when the sprint ends".

In contrast, the term "iteration" comes with far less baggage. In fact it speaks the developer's own language – iteration is one of the fundamental constructs used within virtually any program. It accentuates the cyclic nature of long term software development rather than masking it. The use of feature toggles as an enabling mechanism allows partially finished features to remain safely integrated whilst hiding them from users as the remaining wrinkles are ironed out. Even knowing that your refactorings have gone live without the main feature is often a small personal victory.

That doesn't meant the term sprint should never be used. I think it can be used when it better reflects the phase of the project, e.g. during a genuine milestone iteration that will lead to a formal rather than internal release. The change in language at this point might aid in conveying the change in mentality required to see the release out the door, if such a distinction is genuinely required. However the idea of an iteration "goal" is already in common use as an alternative approach to providing a point of focus.

If continuous delivery is to be the way forward then we should be planning for the long game and that compels us to favour a sustainable pace where localized variations in scope and priority will allow us to ride the ebbs and flows of the ever changing landscape.

Debbie Does Development

[This was my initial submission to the 50 Shades of Scrum book project, hence it’s slightly dubious sounding title. It was originally written way back in late 2013.]

Running a software development project should be simple. Everyone has a well-defined job to do: Architects architect, Analysts analyse, Developers develop, Testers test, Users use, Administrators administer, and Managers manage. If everyone just got on and did their job properly the team would behave like a well-oiled machine and the desired product would pop out at the end — on time and on budget.

What this idyllic picture describes is a simple pipeline where members of one discipline take the deliverables from their predecessor, perform their own duty, throw their contribution over the wall to the next discipline, and then get on with their next task. Sadly the result of working like this is rarely the desired outcome.

Anyone who believes they got into software development so they could hide all day in a cubicle and avoid interacting with other people is sorely mistaken. In contrast, the needs of a modern software development team demands continual interaction between its members. There is simply no escaping the day-to-day, high-bandwidth conversations required to raise doubts, pass knowledge, and reach consensus so that progress can be made efficiently and, ultimately, for value to be delivered.

Specializing in a single skill is useful for achieving the core responsibilities your role entails, but for a team to work most effectively requires members that can cross disciplines and therefore organize themselves into a unit that is able to play to its strengths and cover its weaknesses. My own personal preference is to establish a position somewhat akin to a centre half in football — I'm often happy taking a less glamorous role watching the backs of my fellow team mates whilst the "strikers" take the glory. Enabling my colleagues to establish and sustain that all important state of "flow" whilst I context-switch helps overall productivity.

To enable each team member to perform at their best they must be given the opportunity to ply their trade effectively and that requires working with the highest quality materials to start with. Rather than throwing poor-quality software over a wall to get it off their own plate, the team should take pride in ensuring they have done all that they can to pass on quality work. The net effect is that with less need to keep revisiting the past they have more time to focus on the future. This underlies the notion of "done done" — when a feature is declared complete it comes with no caveats.

The mechanics of this approach can clearly be seen with the technical practices such as test-driven development, code reviews, and continuous integration. These enable the development staff to maintain a higher degree of workmanship that reduces the traditional burden on QA staff caused by trivial bugs and flawed deployments.

Testers will themselves write code to automate certain kinds of tests as they provide a more holistic view of the system which covers different ground to the developers. In turn this grants them more time to be spent on the valuable pursuits that their specialised skills demand, like exploratory testing.

This skills overlap also shows through with architects who should contribute to the development effort too. Being able to code garners a level of trust that can often be missing between the developer and architect due to an inability to see how an architecture will be realized. This rift between the "classes" is not helped either when you hear architects suggesting that their role is what developers want to be when they "grow up".

Similar ill feelings can exist between other disciplines as a consequence of a buck-passing mentality or mistakenly perceived job envy. Despite what some programmers might believe, not every tester or system administrator aspires to be a developer one day either.

Irrespective of what their chosen technical role is, the one thing everyone needs to be able to do is communicate. One of the biggest hurdles for a software project is trying to build what the customer really wants and this requires close collaboration between them and the development team. A strict chain of command will severely reduce the bandwidth between the people who know what they want and the people who will design, build, test, deploy and support it. Not only do they need to be on the same page as the customer, but also the same page as their fellow team mates. Rather than strict lines of communication there should be ever changing clusters of conversation as those most involved with each story arrive at a shared understanding. It can be left to a daily stand-up to allow the most salient points of each story to permeate out to the rest of the team.

Pair Programming has taken off as a technique for improving the quality of output from developers, but there is no reason why pairing should be restricted to two team members of the same skill-set. A successful shared understanding comes from the diversity of perspectives that each contributor brings, and that comes about more easily when there is a fluidity to team interactions. For example, pairing a developer with a tester will help the tester improve their coding skills, whilst the developer improves their ability to test. Both will improve their knowledge of the problem. If there are business analysts within the team too this can add even further clarity as "the three amigos" cover more angles.

One only has to look to the recent emergence of the DevOps role to see how the traditional friction between the development and operations teams has started to erode. Rather than have two warring factions we've acknowledged that it’s important to incorporate the operational side of the story much earlier into the backlog to avoid those sorts of late surprises. With all bases covered from the start there is far more chance of reaching the nirvana of continuous delivery.

Debbie doesn't just do development any more than Terry just does testing or Alex only does Architecture. The old-fashioned attitude that we are expected to be autonomous to prove our worth must give way to a new ideal where collaboration is king and the output of the team as a whole takes precedence over individual contributions.

Stand-Up and Deliver

At the ACCU Conference last year I gave a five minute lightning talk titled “The Daily Stand-Up”, which was a short comedy routine of one-liners around programming/IT/science related topics. This year I was back again, but this time I had more material so could take the opportunity to do more than one lightning talk (they have three slots – one per evening).

In the the subsequent year I’ve used my Agile on the Beach and Equal Experts Christmas party “performances” as an excuse to trawl all 14 thousand of my tweets looking for material. After separating the wheat from the chaff (or rather the needles from the haystack) I unearthed plenty enough for two sets.

It also felt as though the name “The Daily Stand-Up” had been done now and that I should try and organise the material into separate themes which I ended up doing on the way down to this year’s Pipeline conference. As a result I came up with two new titles and grouped the material appropriately, along with creating a holding slide with a suitably pithy picture.

Here then are my two sets (61 one-liners in total) for the ACCU 2016 Conference lightning talks – “Continuous Delivery” and “Becoming a Bitter Programmer” [1].

Continuous Delivery

“I visited the opticians the other day after I started seeing printers, keyboards and mice out the corner of my eye. She said it’s okay, it’s just peripheral vision.”

“We recently needed to break into my bosses account so I thought I’d try a dictionary attack – I just kept hitting the system administrator over the head with a large copy of the OED until he let me in.”

“My wife and I have been together for twenty-five years so I thought I ought to get her a token ring. Turns out you can only get 100BASE-TX these days.”

“If you think keeping up with the Kardashians is hard, you should try JavaScript frameworks!”

“I once listened to an audio book about structs that only contained values of primitive types. It was a POD cast.”

“When Sherlock Holmes talks about a ‘three pipe problem’, does he mean one that requires grep, awk, sed and sort?”

“Is the removal of a dependency injection framework from a Java codebase known as spring cleaning?”

“Was the Tower of Pisa built using lean manufacturing?”

“I know it’s all the rage these days but I reckon the writing’s on the wall for Kanban.”

“The last time I put my phone into airplane mode it promptly assumed the crash position.”

“When I turned thirty-five I blew a whole load of cash on a monster gaming rig. I think I was suffering from a half-life crisis.”

“When creating a diagram of a micro-services architecture I never know where to draw the line.”

“If we want to adopt an agile release train, does that mean we need to start using Ruby on Rails?”

“The problem with the technical debt metaphor is that in the end everyone begins to lose interest.”

“If you send data using the Kermit protocol is the transmission speed measured in ribbits per second?”

“How do you change the CMOS battery in a virtual machine?”

“For Christmas my wife bought me some jigsaws of famous British computer scientists. When she asked how I was getting on, I replied that it was Turing complete.”

“When working from home I like to get my kids involved in my coding. I call it Au Pair Programming.”

“My son was being hassled by his school friends to share his music collection via BitTorrent. I said he shouldn’t give in to peer-to-peer pressure.”

“Our local pub has started selling a new draught beer called Git. The only way to get served is to make a pull request.”

“When they collected their Turing Award I recognised Diffie on the left and Hellman on the right, but who was the man in the middle?”

“I decided it was about time I upgraded to fibre, so I’ve started eating All Bran for breakfast.”

“If you’re looking to purchase some recreational drugs do you first consult Trip Advisor?”

“Should a microphone type be mutable?”

“When software developers work at New Scotland Yard do they have to use a special branch?”

“I recently visited the dentist and he told me I had a scaling problem. I said that’s a problem as I don’t have room for any more teeth.”

“We recently had a poll for the best escape character. I picked Steve McQueen’s Hicks.”

“I’m never quite sure, is it Heisenbergs uncertainty principle?”

“I was utterly convinced that I had found the answer to my SSL problem. In fact I thought it was a dead cert.”

“When writing software for estate agents, should you use property based testing?”

“The other day I struggled to upload a picture of Marcel Marceau. Every time I tried the server returned the error: 415 unsupported mime type.”

“Probably the hardest problem in computer science is dealing with nans. They always seem to want you to come round and fix their computer.“

Becoming a Bitter Programmer

“Is a cross functional team just a bunch of grumpy Lisp programmers?”

“I’ve given up playing poker with functional programmers, they just spend all their time folding.”

“My company recently adopted mob programming. They hired a bunch of beefy guys to stand around with baseball bats to make sure you did an 80 hour week.”

“Some people are like the char type in C++, they seem to get promoted for no apparent reason.”

“Is it any wonder modern programmers are obese when they depend so heavily on syntactic sugar?”

“The product owner asked me why all our Cucumber tests only covered the happy paths. I told him they’re rose tinted specs.”

“Every time I try and generate a digital signature I just make a hash of it.”

“C++ comes with complexity guarantees – if you use C++ it’s guaranteed to be complex.”

“Some people say C# & Java programmers overuse reflection, but given the quality of their code I’d say they’re not reflecting enough.”

“Talking of C# & Java, is it just me or are they harder to distinguish these days than The Munsters and The Addams Family?”

“Our team isn’t very good at this agile stuff – our successes are just stories and our failures are all epics.”

“Anyone who says you shouldn’t use synchronisation objects is just being racist.”

“Whenever I find a static in multi-threaded code it makes my hair stand on end.”

“If you don’t keep your promises, how will you know what the future holds?”

“Our zero tolerance approach to floating-point comparison errors didn’t work out quite as we’d hoped.”

“If technical debt is the result of poor quality code, does that make bad programmers loan sharks?”

“Don’t bother upgrading your database, the SQL is never as good as the original.”

“I blame Facebook for the declining quality of SQL code, young programmers are obsessed with likes.”

“My current system has five-nines reliability - it usually works about 45% of the time.”

“Our DR approach is less active/passive and more passive/aggressive. When anything fails we just sit around loudly tutting until someone goes and fixes it.”

“The company said it was moving all our hardware to EC2. It turned out they’d bought a dingy office in London near the Bank of England.”

“Some people just don’t know when to stop bashing Windows.”

“When I found out my new job was effectively working with legacy code I started spitting Feathers!”

“I reckon my team has taken a hypocritic oath – the comments say one thing but the code does something completely different.”

“You can always tell when you’ve got a Christmas or birthday present from an enterprise software developer – there is an excessive amount of wrapping.”

“C# supports impure methods and mutable types, but doesn’t support tail recursion or currying. Does that make it a dysfunctional programming language?”

“Finding the square root of a negative number is just not natural.”

“My team is using homeopathic unit testing. They write one test and dilute it with a thousand lines of production code.”

“The Wildlife Trust has declared our codebase a conservation area on account of the number of bugs.”

[1] The latter title was inspired by the recent book from the compere of the ACCU Conference lightning talks – Pete Goodliffe – which is called Becoming a Better Programmer.

Get Considered Harmful

[I’ve tried really hard to come up with a better title than this, e.g. Thesaurus Driven Development, but none of them quite conveyed my feelings. Surely we must all be entitled to use the “??? Considered Harmful” template at least once in our careers? Well, this is mine.]

They say that one of the hardest problems in computer science is naming things [1]. We struggle over what to call our functions, classes, variables, filenames, servers, etc. in the hope that they adequately convey what it is that they do or contain.

In my own programming endeavours I’ve found the use of the Thesaurus a vital ingredient in staving off the boring repetition that so often litters the names in a codebase. In fact one of the latest additions to my “In The Toolbox” column for C Vu touched on this very topic (see “Dictionary & Thesaurus”).

Bueller, Bueller, Anyone…

Whilst the travesty that is appending the word “Service” as a suffix to some noun in the hope of making it sound “behavioural” ( and therefore also suitable for repurposing as an equivalently named interface) is a common blight, but it’s not the one I’m interested in this time.

No, in this instance the word that is the target of my ire is “get”. It’s often as if no other word exists in the English language for helping to name what a function (or method) returning a value does. I sometimes wonder if the all the effort is expended by programmers trying to decide what it is that they’re trying to obtain that they have no time or energy left to describe how they will be obtaining it.

The word has become so utterly bland and devoid of any usefulness that, by definition, it could be used for absolutely anything and therefore it is a de-facto choice that could never be wrong. Of course unless a complete lack of descriptiveness is one of your criterion for a good name. Sadly it is one of mine.

A Smorgasbord of Alternatives

As I mentioned in that C Vu article, and more recently on Twitter, I can think of a whole bunch of words that could almost certainly describe more accurately what it is that “some function” might be trying to achieve. Here once again is a similar, off-the-cuff list of choices:

build, create, allocate, make, acquire, format, locate, request, fetch, retrieve, find, calculate, derive, pull, process, format, transform, generate.

Within that simple list of words there are clearly a number of categories under which we could group our choices. For example if I’ve got a factory function then I might choose: create, build, make or allocate.

If the factory is some kind of object pool I might not want to use the create/destroy or allocate/free pair as they potentially suggest too much and so I might go for the slightly looser acquire/release pair.

If the creation is for a computed sequence of items rather than a single value or entity, then generate might be more applicable.

Another group centres around the kind of calls made on an external (or remote) service, e.g. request, fetch, retrieve, locate, find, query and search. Some of these words are equally fitting for searching a local collection too and therefore the degree of transparency may have a bearing on the choice. There is an old programming adage “Don’t hide the network”, and naming is one clue we can use to do that.

The third obvious group I can make out are words for describing the more common notion of what a traditional function is: the transformation of one value into another. Typically we put one value in and get the same value out, but changed in an interesting way, e.g. a different representation. Or maybe we do get a different value out, but with the same representation. Or even a change in both, but nevertheless one that is still similar in more ways than not [2].

For this style of (pure) function I would naturally look to words like: transform, convert, format, parse, calculate, derive, etc. Words like execute or process, which might also convey the notion of some transformation come (in my mind) with thoughts of side-effects too, rather than the purity of a mathematical function.

Accessor or Factory?

If you take a very rigid stance that says every function can either return a value it already holds, or can create a new one, then every function must either be an accessor or a factory. It’s easy to see how this kind of myopia leads to a codebase littered with “factory” classes when they’re basing their naming on a simplistic view of the function’s implementation.

The flipside to being overly generic is being overly specific. When you have a concrete method it’s easy to name it after what it does. But when the method is part of an interface which can, by definition, be implemented in many different ways we run the risk of potentially sending the reader down the wrong path.

Which is worse though: forcing the reader to look inside the box because it has no meaningful label, or asking them to suspend their disbelief when the contents eventually turn out to be blander than advertised? Personally I’d rather be surprised to find a simple calculation when I wasn’t expecting it rather than a network hop to a remote machine.

Blandness Over Configuration

I do wonder if part of the problem might not lie with modern tooling. In our desire to embrace the notion of Convention over Configuration we restrict ourselves to using only those words that the tool can recognise. Those words are very likely to be the ones that have the broadest stroke exactly because we expect them to cater for a wide degree of diversity – it’s the same issue as with interfaces, but on an even wider scale.

Another source of inspiration for such vocabulary are books on Design Patterns, and that usually means the seminal one from the Gang of Four. One might argue that the raison d'être of the Design Patterns movement was to create a common vocabulary, and it was, but that doesn’t mean you have to use the pattern names and examples in your own types and functions.

Haven’t We Been Here Before?

If any of this coming from me sounds at all familiar it might be because I’ve trodden a similar path before right back in the infancy of this very blog, see “Standard Method Name Verb Semantics”. Back then I was particularly interested in trying to overcome the horribly overloaded nature of certain words, of which “get” happened to be just one.

In a sense this blog post is a step backwards. Rather than trying to overcome the subtitles of whether Get or Find is more suitable for a method that can also return the absence of a value [3], or Allocate versus Acquire for an object pool, I’m just trying to convince programmers to use almost anything [4] other than “get” once again.

[1] The other one of course is cache invalidation (and off-by-one errors).

[2] For example I once saw an F# example that “mapped” a URL into a JSON document. While a lawyer might argue that it’s a perfectly valid “function”, I’d contend that it is hiding a huge amount of complexity. This is both a blessing and a curse depending on whether or not it’s your turn with the support pager.

[3] The answer of course is probably “either”, but return an Option instead of a value / null reference; i.e. make the return type the defining characteristic.

[4] By applying Cunningham’s Law the function stands a chance of being renamed when the mistake is noticed, whereas giving it a dreary pretty much guarantees it’ll never be changed.