Friday, 7 September 2012

Assume Failure by Default

Maybe I’m just an old pessimist but when I see code like this below, it makes me think “now there’s an optimistic programmer”:-

void doSomething()
{
    bool ok = true;
    . . .
    return ok;
}

Personally I’d start with ok = false. Actually I wouldn’t even write that because I’ve always favoured early exit (in C++) and despise the heavily nested code that this style can lead to. But does it matter? Well, yes, because developers generally don’t test their error handling with anything like the effort they’ll put into making sure it works under normal conditions. And I’m not just talking unit testing here either, what about system testing where machines run out of memory and disk space, and networks disappear and reappear like the Cheshire Cat?

So, let me give a more common example which touches on one of my pet peeves - process exit codes. How many programmers check that the exit code produced by their executable is anything other than 0? Putting aside the whole “void main()” issue for now on the basis that you’ll be using a language that forces you to return an exit code in the first place, this is not IMHO the best way to begin:-

int main(int argc, char** argv)
{
    . . .
    return 0;
}

For starters let’s stop with the magic numbers (which also happen to look like FALSE for extra mind-bending confusion[+]) and make it obvious:-

int main(int argc, char** argv)
{
    . . .
    return EXIT_SUCCESS;
}

Now that obviously looks dodgy - success, always? In reality main() is just a stub to invoke factories to get the services needed by your application logic and so you might end up with something more like this instead:-

{
    int result = EXIT_FAILURE;

    try
    {
        // Create services
        . . .
        // Invoke application logic
        . . .
        result = EXIT_SUCCESS;
    }
    catch(/* something */)
    {
    }
    catch(/* something different */)
    {
    }

    return result;
}

If you prefer the early exit approach then you might elide the result variable and just perform a return EXIT_<WHATEVER> instead at the relevant spot. The key point here being that you focus on ensuring the default paths through the code should end in failure being reported unless you happen to strike gold and manage to make it through to the end without hitting any problems. By assuming failure you have to work harder to screw it up and create a false positive, and personally I prefer code that fails by accident, a false negative, immediately right where the problem is rather than later when the context has long gone.

As if to labour the point I’ve recently had to work with some components where the code is written in the “optimistic” style - essentially the methods are written this way round[#]:-

{
    bool ok = true;

    try
    {
        if (doSomething())
            if (doSomething())
                . . .
            else
                ok = false;
        else
            ok = false;
    }
    catch(/* stuff */)
    {
        ok = false;
    }

    return ok;
}

I’ve lost count of the number of times that I’ve debugged this code only to find that somewhere earlier a return code was ignored or an exception handler swallowed an error. But hey, this is nothing new, this problem has been around far longer than I’ve been programming. Exceptions are pretty much ubiquitous these days and so it does feel less common to see this kind of problem within application logic.

For me the acid test[*] is still “if this code traps an out-of-memory error what do I want it to do?”. And I’ve yet to answer that question with “just ignore it”.

 

[+] Have you ever seen C code written that does an equality string comparison like this: if (!strcmp(lhs, rhs)).

[#] It’s actually C-style procedural code encoded as C#. The author is not one of the development team and is the first to admit it’s not exactly a beacon of quality. But it’s in production and generating value every day (along with a side order of technical debt).

[*] By this I mean when someone writes “catch (Exception e)” I ask them the acid test question, because that’s exactly what they’re going to catch eventually. Presumably it’ll be forwarded until we hit a component boundary (process, COM, DLL, etc) at which point you’re going to have to transform every error into something the caller understands (ERRORLEVEL, HRESULT, etc).

Friday, 4 May 2012

Beware the Complacency Unit Testing Brings

One phrase that is always sure to raise the ire of any good honest developer when something breaks is:-

“well, it works on my machine”

This simple statement shows a complete disregard for any other sort of testing that you might need to do to ensure that your feature works correctly and is “done, done” not just almost done. But there is a new kid on the block when it comes to showing how little some people understand about software development that I’m beginning to hear with alarming regularity:-

“well, all the unit tests passed”

It seems that modern development practices have unknowingly created the Silver Bullet that Fred Brooks has always told us never existed! Apparently good unit test coverage and automated refactoring tools means that it’s highly unlikely that any bug would only show up during integration and system testing that those ideas are just old fashioned. Or, if not altogether outdated, then reduced to just a footnote in the product’s testing strategy on the basis that there is so much less value in them than unit testing.

Don’t get me wrong I can understand a genuine mistake caused by a seemingly unrelated change - accidents happen and it could be a fault of the design - but changing the configuration file for a service and then not even bothering to see if it starts up is just laziness. Yes, it does take time and effort to do more extensive testing in your sandbox but the feedback loop could still be fast and you won’t annoy your team mates when you cost them a day’s system testing because of a silly mistake.

The rule of thumb about not checking in code until the unit tests pass was designed to make you think about writing fast tests so that the barriers to testing are as low as possible, it was not expected to be used as a justification for short-circuiting the amount of testing you do.

My “Agile SQL Tweet” Slide

Before I put the PowerPoint slides for my ACCU conference talk on Database Development Using TDD up on my web site I agonised over whether to remove the 3rd slide or not. The slide in question just contains the following tweet from Allan Kelly:-

@chrisoldwood is the only person I know with a convincing Agile SQL story -- @allankelly

I felt that viewing the slide in isolation would only amplify the arrogance which could be attached to it and that wouldn’t have a positive effect. But at the same time I felt I should put up what was shown and accept that perhaps it may also raise the same questions that I had that caused me to put it in the first place.

The slide forms part of the prologue and was originally just a picture of myself to act as a backdrop whilst I spent a couple of minutes describing who I was and what I did as I’m not a database developer per-se, I’m mostly a C#/C++ service layer guy who has slipped into the SQL arena like many similar developers. As a consequence I have tried to work with the RDBMS in the same way that I would approach anything else, by using TDD and abstracting the client from the underlying implementation. So, rather than couple the client and database code tightly with client-side SQL[*] I have kept them decoupled to allow the database to develop (and therefore be refactored) independently where possible - I don’t expose public fields in my types[#] so why directly expose tables?

And then this tweet from Allan Kelly appeared. Until that moment I suspected I was the proverbial square peg trying desperately to fit into a round hole, but maybe, just maybe, I might not be barking mad after all. There are many benefits to TDD (and unit testing in particular) over and above proving the functional correctness of a piece of code, but I don’t understand why more people can’t see that.

So the slide should hopefully provoke a reaction that forces you to question what your database development tools and processes are and whether you’re getting the most out of them. If you are then great, it’s working for you, but if you have that uneasy feeling that you’re stymied at every turn because your database can’t evolve quickly enough then perhaps what you’re about to see (and hear, if you attended) will provide food-for-thought about what you could change.

Prior to this the most ‘positive’ reaction I’ve had to what I’ve been suggesting is:-

“I suppose it makes sense, I’ve just never seen it done like that before”

[*] I’m not just talking about explicit embedded SQL here where the SQL code is a string literal (or as a format string) but also where it is implicit as a by-product of a technology such as LINQ-to-SQL. One way or another the schema is likely to be too tightly coupled to the calling code and that means your database design will be harder to change as will remediating performance problems.

[#] Some would argue that a DTO (Data Transfer Object) does exactly that, but a modern RDBMS is so much more than just a data persistence mechanism. Use its power where it makes sense to.

Wednesday, 2 May 2012

My ACCU Conference Session - Database Development Using TDD

400_image

It’s been a busy few weeks as I’ve been preparing to present at the 2012 ACCU Conference in Oxford. This has become somewhat of a pilgrimage and for the second year running I’ve been accepted as one of the speakers; this time to talk about using TDD to develop databases. This was an excuse to piece together some of the ideas that I’ve been blogging about along with the experiences gained with my current client to present one way that can use tests to drive both the development and design of a database.

Session Details

The talk started by walking you through the pre-requisites needed to embrace TDD, such as your own development sandbox and looked at the primary testing mechanism[*], i.e. unit testing SQL. I also spent a fair bit of time covering the notion of a database’s public interface as I personally feel that the modern RDBMS gives you the tools to build an abstraction layer to help decouple the client from the data model to grant freedom of implementation[#].

Naturally I explained the principles of TDD and did a 20 minute live coding session to show how I would use them to develop a reporting style stored procedure. For this I used my own SS-Unit testing framework and SQL Server Express; switching rapidly between test code & production code. With the main body of the talk in the bag I turned to the opportunities that TDD (although really unit testing) opens up such as continuous integration, continuous deployment and schema refactoring.

The PowerPoint slides are available on my web site here.

Abstract

Just for the record this was the abstract that I submitted...

The modern day RDBMS is a complex product that offers so much more than just data persistence. The SQL language, with its vendor specific variants such as T-SQL, provides the ability to develop code in various forms to read, transform & write that data efficiently. This code requires constant testing right from its inception through its various incarnations until it is finally retired.

TDD is a technique that puts writing those tests at the front of the development process, whether that be because you’re writing new code or changing existing code. The knock-on effect of this approach is that your client-based perspective opens your eyes to potential variations in the implementation, and that is where the second ‘D’ in TDD turns from Development into Design. With a solid automated test suite and Continuous Integration under your belt too the world of refactoring opens itself up so that your database design can safely evolve.

This session looks at applying the same principles and disciplines used in other areas of system development to tame the ever increasing complexity that has arisen from the maturity of the RDBMS.

Photo

The above photo of yours truly comes courtesy of Mark Ridgwell. Here are the rest of his photos for Wednesday.

[*] TDD does not prescribe any particular type of testing but I believe that most people associate TDD with unit testing as it probably forms the lions share. I did point out that one of the benefits of a top-design and implementation is that you can start development with a system test and drill down from there.

[#] ORM tools like Entity Framework actually push you in the other direction which may be exactly what you want. But as I’ve said before, Enterprise culture has a habit of routing around nicely designed service layers and attempting to go straight to the heart of the data.

Wednesday, 22 February 2012

OK & Apply - A Failed UI Experiment?

As you may have guessed from my previous two posts, I have been paying far more attention than usual to how other IT professionals[*] work. The most recent one that caught my eye and has no doubt been done to death in UI circles is the OK and Apply buttons on dialogs. It’s funny to watch people first push the Apply button and then the OK one. The question is whether they don’t realise that pressing OK has addition affect of applying the changes, or that they don’t trust that just pressing OK is enough? I’m sure it’s the latter - a belt and braces approach.

I know how they are supposed to operate, but I suspect that’s only because I was doing a lot of UI work when Windows 95[#] appeared and so I was paid to know all the UX guidelines. Perhaps it’s time to take a trip down memory lane and visit the “Interface Hall of Shame” along with a quick Google search to find out if/when this UI faux pas was officially deprecated...

 

[*] I’m always looking to learn new tricks and techniques to be more effective and other IT professionals is obviously as good a place as any to look as this is (supposedly) their bread-and-butter.

[#] and Windows NT 3.5 with the shell upgraded to 4.0.

Copy-and-Paste Can Help Avoid Mistakes Too

In the software development world the term “copy-and-paste” is synonymous with bad practice. Web sites and books are filled with advice about how you should avoid duplication in code and maintain a Single Point of Truth (SPOT) or Don’t Repeat Yourself (DRY)[*]. One could almost be forgiven for ripping the X,C, & V keys out of the keyboard for fear of succumbing to such a sinful practice. And yet I do it all the time, way more than most of my colleagues it seems. Of course I’m not talking about duplicating code though, that would clearly be wrong...

Some months back we had a bug which caused quite a bit of head scratching as it was a subtle spelling mistake in a .cmd script (%valueData% instead of %valueDate%). Watching the developer at work intrigued me because where I would naturally have copy-and-pasted the variable name from a few lines below, he typed it out by hand. Just yesterday a colleague was surprised when they couldn’t see the files they were expecting in a folder and it turned out they had mistyped the folder path. What I found amusing was that the folder name was readily available in a text editor on the other screen (which itself was a copy of the wiki page I use for the same task) and so a quick copy-and-paste would have avoided the error.

The way I write wiki pages and release plans is very much with the copy-and-paste approach in mind. The textual description is like a comment in the code, there to advise the newbie and provide links to the supporting rationale and concepts. But what the experienced[#] developer wants is just a checklist of items, or perhaps a template that they can copy-and-paste onto the command line, into their text editor, run with their SQL tool, etc. Perhaps it’s just me, but once I step outside the woolly realm of writing English prose and into the precise world of programming I immediately look for that safety net that will help me avoid losing time due to “simple” mistakes, no matter how small. In some cases it may only save a minute or two, such as learning on the compiler, but in others it could be hours if the outcome is to bork system testing which then has a knock-on effect onto my team-mates.

Of course my way is far from perfect too and it is fraught with a different set of problems instead. Have you ever copy-and-pasted a command line from a Word document and found it doesn’t work? Did you then waste time, grumble constantly and gnash your teeth only to discover that Word had “intelligently” replaced a pair of en-dashes (--) with a single em-dash (—), or worse converted them from their ASCII to Unicode counterparts? They call it “smart formatting” but in this instance it’s anything but smart! Maybe that’s the reason why Microsoft eschews the “-“ command line switch convention in favour of “/”?

The second common mistake I run into is with copy-and-pasting from the very wiki pages that are designed to save people from making mistakes. Web browsers[+] like to be intelligent when you copy a block of text (such as an example command line) so that if you select an entire line they will “helpfully” terminate it with a newline. This has the wonderful effect of executing the command right after pasting it into a console window! No matter how slowly or carefully you select the text, the moment you reach that last character the selection box automagically expands to include the nauseating newline.

In the end mistakes are always going to happen, so choose your poison...

 

[*] Matthew Wilson even likes to combine the two and call such artefacts a “DRY SPOT” which creates a wonderful metaphor.

[#] “Newbie” and “experienced” in this context relate to their knowledge of the system, not how long they’ve been in the profession.

[+] No doubt this will incite the usual tirade of “why are you using a stupid browser like IE, get a ‘real’ browser!” comments. I’ve answered that before.

Tuesday, 21 February 2012

Don’t Let Your Tools Pwn You

Modern development tools such as IDEs and their associated multitude of plug-ins, such as ReSharper, can definitely improve productivity. But they can also reduce it too if welded without due care. In some cases it may not be yourself that suffers the loss in productivity but your team-mates because you were unaware of what that tool actually did.

The original impetus for this post was actually watching someone dig themselves into a right mess with ReSharper. I suspect they had got so used to accepting every pop-up that ReSharper threw at them that they didn’t notice when ReSharper had suggested the wrong change. The code didn’t compile now but it wasn’t obvious why as the wrong turn was some minutes ago and they were focused on the code they were writing now. After all, the previous code did compile so why could the mistake be back there?

Being “old school” I used the compiler output to point the finger rather than ReSharper’s advice and traced it back to the source. The earlier mistake was to mistype the inheritance of a class and then let ReSharper implement it automatically. This mistake initially swapped compile time errors for runtime errors (throw NotImplementedException is the default implementation) and unless you use a test-first approach the feedback loop between making and discovering the mistake grows rapidly.

The more recent impetus is another tale, once again involving ReSharper, that caused another person pain - me. As part of a large refactoring exercise one of my team mates had to split some core classes off into a common assembly for use by the new client application. They chose to use ReSharper for the job which makes perfect sense as refactoring is one of its main features. Sadly they didn’t know what ReSharper was also going to do at the same time...

These changes all happened in the background on the trunk and it wasn’t until I came to merge some last minute changes from our release branch that things started to get dicey. I was getting a lot of merge conflicts on the changes even though I knew that no one would have touched the code[*]. Well, I knew that the namespace at the top of the file would have changed as a consequence of the other person’s changes, but definitely no more. After looking through the Subversion log I could see that some whitespace has also changed and my gut reaction was that someone had also “fixed the formatting” at the same time as making another change. I didn’t spot this myself at first because the diff tool I originally used had “ignore whitespace changes” still set to ON after an earlier unrelated comparison! Unfortunately the piecemeal way the refactoring had been done also meant there was more noise in the SVN log[+] than a single commit.

Ultimately it took me an hour and half to convince myself that the merge was sound when it should have been trivial. I didn’t notice at the time but the unexpected whitespace changes had also destroyed the careful formatting of a table that needed to be kept in sync with a similar block in some SQL code[#]. This cost me another 30 mins or so to sort out. In retrospect I would have been better off reverting all his check-ins first, fixing the build and then merging mine in after. But that’s part of the problem with finding unexpected changes - you then go looking for the proverbial needle in the haystack that you’re convinced must exist.

My natural reaction to this second issue is that the original refactoring changes should have been diff’ed before check-in (as any change should) and then the problem would have been spotted and corrected at its source. The argument against doing that would no doubt be that they don’t have the time to diff 100 or so files and anyway the change was “automatic”. The question now is whether it would have taken that person more than 2 hours (the disruption caused to me, let alone others) to diff those files and spot that the tool was doing something unexpected? I sincerely doubt it, even doing it manually. And you could probably automate it to produce a unified diff that would clearly have shown up lots of unintended changes.

OK, so that’s only two issues, hardly enough to suggest we give up our modern tools and return to the stone age. And I’m not suggesting that. What I am suggesting is that you’re careful not to get drawn into relying on the tools so much that you cannot remember how do things “long hand”. Although I mainly use Visual Studio with ReSharper for day-to-day C# development I still like to use tools even as primitive as Notepad because they are fast, will spot issues other tools hide[$] and most importantly are even available on production hardware.

 

[*] It was the kind of legacy code you really don’t want to touch unless you absolutely have to. And if you do you want tests. But you can’t.

[+] Given the limitations of Subversion is was probably the right way to have done it as its Copy+Delete approach to moving and renaming files can get you into hot water, especially when mixed with branching.

[#] Yes, in an ideal world these two bodies of code would be generated from a single source, but life is never that easy in reality.

[$] The best way to spot a file that has a mixture of tabs and spaces is to open it in Notepad. Has anyone ever chosen 8 spaces-per-tab out of choice?