The OldWood Thing

Diagnostic & Support User Interfaces

2011-12-20T00:58:00.001Z

When people talk about user interfaces they are invariably talking about the face of a product or system that the end-users see and/or interact with. But within a complex system there are other user interfaces too that are not seen or interacted with by end-users at all; instead it is fellow developers and, more importantly, support staff that get to see this less salubrious side.

In a small well-established team of developers there will naturally be a lot of cross fertilisation meaning that they will be well versed in significant portions of the system. In contrast, a larger team will have skills spread across different disciplines and so from a support perspective it starts to approach the knowledge levels of a support team - far less. In my experience support teams do not get the luxury of devoting themselves to learning one system instead out; instead they invariably have to support a whole bunch of in-house systems written by different teams using different technologies and having different ideas about what is required to make a product or system “supportable”.

So what kinds of interfaces am I referring too? Well, the most traditional are status monitoring pages and custom admin tools; the kind of utilities built specifically for the purpose of administering and supporting the system. Due to the time constraints imposed and the lack of direct business value they are usually not given any real TLC but are just thrown together, presumably under the misguided assumption that it will never really be used except by them or their close colleagues. But these scripts and command line tools are not the only faces of the system you’ll see when doing support duties; before that you’ll probably be faced with hunting through the custom format log files, which may or may not be easily consumable. Stretching the analogy somewhat further you could encompass configuration files and the database schema as interfaces of sorts. They’re not graphical in themselves, but careful though about their structure can ensure that the off-the-shelf tools stand more chance of making interaction a more productive experience.

Anyway, here is my list of things to think about. I’ve not done any “production” UI programming for a few years and it’s been even longer since I read About Face (the 1st edition no less) so I’ll no doubt contradict some modern wisdom along the way. Your target audience may not be novice computer users, but they do have the clock against them and they won’t thank you for making an already difficult situation any harder.

Command-Line Tools

Server-side developers aren’t renowned for their user interface programming. After all, that’s why you have UX designers and developers whose major skills lie in the area of UI programming. But that doesn’t mean we have be to sloppy about the interfaces for the scripts and tools we do create. I’m not suggesting for a moment that we spend hours gold-plating our utilities, but I am proposing that you adhere to some of the common conventions that have been established over the years:-

Provide a help switch (-?, -h, --help) so that the user can see what the options are. Documentation, if you can even find it for a utility, can easily get out of step with the tool usage and so you need to be able to see which switch it is that you’re apparently missing, or that is no longer relevant for the task at hand. Even if you’re au fait with the tool you might just want to double-check its usage before going ahead and wiping out last years data.

Provide a version switch (-v, --version) so the user can check its compatibility, or if it’s become out of date. Many tools are installed on the application servers or alongside client binaries, but sometimes those tools are just “passed around” and then become available on some desktop share without you realising. You can even dump the version out on every invocation as the first line; this can be particularly useful if you’re capturing stdout when running it as a scheduled task.

Provide long switch names (--switch-name) by default and use them in documentation so it is obvious what the example command is doing. Short switch names are for advanced users and so should be considered the optional case. Positional parameters may save the user some typing but when you use concepts like integer based surrogate keys it’s not exactly obvious what an example may be suggesting:-

C:\> DC 1234 2001-02-03 1

compared to:-

C:\> CustomerAdmin delete --customer-id 1234
--date 2001-02-03 --force

Use a consistent switch format. I’ve had to use utilities where one developer has made the switch names case-sensitive and another case-insensitive. Sometimes they use ‘/’ as the switch marker and other times it’s a ‘-‘. Often the case-sensitivity issue is down to lazy programming - using == or Equals() instead of the correct method. If you’re on Windows try and support both styles as even Microsoft’s developers aren’t consistent. Better yet download a 3rd party library that lets you use either style as there are plenty to choose from[#].

Return a meaningful result code. The convention (on Windows) is that zero means success and non-zero means failure or partial success. Some tools do provide more detailed result codes, but fundamentally you want assume that anyone trying to automate your tool on Windows is going to write something along these lines as a first-order approximation:-

CustomerAdmin.exe --do-something-funky
if errorlevel 1 exit /b 1

Sadly, 0 is also the default result code that most frameworks will return for an unhandled exception which really makes the whole process rather messy and means that you need to get a Big Outer Try Block in place in main() as part of your bootstrap code.

Support a read-only/test/what-if mode. Most of my support tools have grown out of test harnesses and other development tools and so I’ve naturally added switches that allow you to run them against production in a benign mode so that you can see what would happen. PowerShell has taken this idea forward formally with its common -WhatIf switch. When you’re not 100% sure of what the expected action is or how many items it will act upon this can be a life saver.

Most of the same comments apply to scripts, but they have the added advantage that they are deployed in source code form and so you have the ability to see the logic first hand. Of course when you’re in the thick of it you need to be able to trust the tool to do what it says it will.

Logging

When faced with a production issue, log files will often become the first port of call after you’ve established which process it is that has failed. But log files[+] often serve two different masters and the latter one can get forgotten once the development work has finished. During development it is probably the developer that spends the most time pouring over the output and therefore they have a habit of dumping every last bit of detail out. After the feature is complete though the main consumer then becomes the support team. What they don’t want to see is reams of output so that the proverbial wood is lost behind the trees.

Make sure your log messages make sense. This may seem obvious but spelling mistakes, whilst cute in a time of stability, are a distraction when you’re up to your eyes in it. Log messages should form a narrative that lead you up to the moment of interest, and that narrative should be written with the reader in mind - mostly likely a support engineer - someone who doesn’t know the code intimately. By all means be succinct, but ensure you’re not being ambiguous either.

Use the console output for high-level commentary, warnings and errors only. If extra output is required it should be enabled by an extra --verbose switch (which can easily be added temporarily under automation) or by having a companion log file with more detail. I tend to favour the second approach so that the detail is always there if needed but so that it’s not in your face the majority of the time. Remember we’re talking about in-house stuff here and so you have the luxury of controlling the environment in ways a 3rd party can often only dream of.

Keep to a low number of severity levels. If you have a large number of severity levels developers will not be able to log messages consistently. I recently saw a tweet that suggested you use only the following levels - FYI, WFT & OMG. If you put yourself in the support person’s shoes that’s actually a pretty good scale. Once the issue is triaged and the development team are involved the TRACE or DEBUG level output suddenly becomes of more use, but until then keep it simple.

Write the messages as text. A simple lined-based text format may seem very old-fashioned, but there are a plethora of tools out there designed to slice-and-dice text based output any way you want, e.g. grep, awk & sed. Plus you have modern GUI based Tail programs that support highlighting to really make those important messages shine, e.g. BareTail & LogExpert. Layer on top something like LogParser which supports a SQL like syntax and support for many formats out of the box and you’re in good shape to triage quickly. Also don’t underestimate the importance of NotePad as a tool of last resort.

Make the log message format simple. If you’re writing a separate log file include at least the date & time, and preferably the process ID (PID). If the tool is multi-threaded then the thread ID (TID) will be invaluable as well, along with the highest precision[*] of time (e.g. milliseconds or better) to spot potential race conditions. I’ve found the following format to be very easy to scan by eye and parse by tool as the left-hand side is of fixed width (the PID & TID are padded appropriately):-

2001-01-01 01:02:03.456 1234 3456 INF Starting...
2001-01-01 01:02:04.123 1234 5678 ERR Stuff broke...

Enclose string values in some form of quotes or braces to make empty strings more visible. As I posted very recently, empty strings are a form of Null Object and so they have a habit of being silently propagated around before finally causing odd behaviour. Personally I like to enclose strings in single quotes because I often have to use those values in SQL queries and so then it’s already quoted for me. As an example I once spent far longer than necessary investigating an issue because the log message said this:-

Valuing trade ...

instead of:-

Valuing trade ''...

Include all relevant context, preferably automatically. Messages that just tell you something bad happened are useless without the context in which the operation is taking place. There is nearly always some higher-level context, such as a request ID or customer name that you can associate with the PID or TID to help you narrow down the data set. Debugging a complex or lengthy process just so you can find out the set of inputs to find that the data is dodgy is not a good use of time and costs the business money, especially if the relevant variables are in scope at the point of writing the error message![^]

Don’t spam the log, write a summary. If you have a process that deals with particularly dirty data, or you have a service that is temperamental try not to log every single indiscretion as a warning or error. Instead try and batch them up, perhaps by item count or time window and write a single message that gives a summary of the issue. At a high-level you’ll just be interested in seeing if the problem is getting worse and so using a tolerance is even better as once you have an idea of what “normal operation” is you can save the scary messages for when they’re really needed.

One of the reviews I do during system testing is to look at the log output. I try and put myself in the position of the support engineer and read the log messages to see if they make sense to me (I often only have a vague appreciation of the process in question). The thing I’m most looking out for is log spam which is usually solved by downgrading messages from, say, INFO to TRACE so that the main output is all about the big picture. It should contain virtually no warnings or errors to avoid support staff becoming desensitised by noise and when they are present they should hopefully be backed by more detailed diagnostics somewhere else.

[*] Don’t be fooled into thinking that because a datetime value can be formatted with a milliseconds part that it is actually that accurate. IIRC 16-bit Windows used to have a resolution of 50 ms whilst the NT lineage was better at 10 ms. If you wanted better accuracy than that you needed to resort to the Performance Counters API.

[#] My own free C++ Core library has a reasonably decent parser should you want to tap into the 3rd party option.

[+] I have mixed sentiments about the use of log files. On the one hand I feel they should largely be unnecessary because we should be able to anticipate, test for and handle most types of failure gracefully. But I’m also aware that in-house systems rarely have that level of robustness as a requirement and so we’re often forced into using it as a defensive measure against inadequate test environments and the other in-house external systems.

[^] I clearly remember spending a few hours at the weekend debugging a financial process that was failing with the message “Failed to correlate pairs”. What annoyed me most was that the developer had the two failing currency names in variables in scope but had not bothered to include them in the message! I fixed the error message first thing Monday morning...

New SQL Tools - SS-Unit, SS-Cop & sql2doxygen

2011-12-11T22:30:00.001Z

At my current client I’ve spent far more time working with SQL (and specifically SQL Server) then ever before; but that is probably quite apparent from the bias of blog posts over the last year or so. With any new venture you soon find yourself drawing on your past experiences and in this case I’ve been looking for some of the equivalent tools in the SQL world that I’d use normally use with C# & C++. But I’ve found them hard to come by. It’s possible I’m looking in the wrong places but it’s not as if there are a stream of vendor adverts either trying to distract me and that gives me a feeling it’s not worthy of a prolonged hunt.

There is another reason why I may have not searched overly hard, and that’s because I find it more fun to build something myself - it’s a great learning exercise. Mind you that’s not the right answer for my client where we have both money and far more important things to be doing than building tooling - assuming this is we can find the right alternatives[*]. What I’m releasing below are my clean-room[+] versions of some of the tools we knocked-up to get ourselves going.

Naturally as these begin to surpass what we lashed up they become 3rd party candidates themselves to take over the same duties and my client wins too because it no longer has to consider supporting the code internally. I’ve always made the source code available for my stuff so that others have a chance to tinker or fix bugs and these are no exception. Of course given that they are T-SQL based (or PowerShell in the case of sql2doxygen), it would be pretty hard for me to keep it secret anyway!

SS-Unit - SQL Server Unit

This is a pure T-SQL based unit testing framework. You write your unit tests, in T-SQL, in the familiar xUnit style and then execute them by invoking the SS-Unit test runner (a stored procedure). The framework has support for both the test and fixture level SetUp and TearDown helpers which are more prevalent in SQL tests because of the static data dependencies you often need to satisfy. It also distinguishes between being run interactively, in something like SQL Server Management Studio (SSMS), and being run in batch mode with, say, SQLCMD so that it can be integrated into your Continuous Integration system for automated SQL unit testing.

The .zip file package, which includes a trivial example database and the unit tests for the framework itself is available on my web site here on the SQL section page. There is also an online copy of the manual if you just want to have a nosy.

SS-Cop - SQL Server Schema Cop

Earlier this year I posted about “DbCop”, a StyleCop/FxCop like tool that a team-mate had put together for use on the project. That generated quite a bit of interest and a few people asked if there was any chance of it being released. As I point out in a footnote[+] that isn’t really an option so instead here is my take on the idea. Clearly FxCop does some pretty deep analysis and I’m not suggesting this is going to provide anything of that magnitude. But there are a number of checks that we can do that ensure style conventions are being adhered to and common mistakes, like forgetting a primary key, are avoided. I have a few other new ideas too, such as trying to detect when a foreign key might be missing.

The .zip package is available from the same SQL page as SS-Unit and also contains an online copy of the manual. Not unsurprisingly the unit tests for SS-Cop are written using SS-Unit and so you could consider it a more realistic example of how to use SS-Unit - SS-Cop was written using a Test-Driven Development (TDD) approach.

sql2doxygen - SQL Doxygen Input Filter

Funnily enough the chap that wrote DbCop was the same person that introduced me to Doxygen - a tool for generating API style documentation from source code. What attracted me was the fact that it could generate useful documentation even when no effort had been made by the developer! Better still was that with only a small change in your comment style Doxygen could handle so much more - no horrendous tags or mark up like you get with JavaDoc or the standard C# offering. There are still a bunch of special tags if you need them but they don’t detract from the comments which is what you need when you’re actually trying to read them as you understand the source code.

Anyway, Doxygen supports many programming languages out of the box, but sadly not SQL. However it does provide you with a mechanism for supporting any language as long as you can transform the source files into something Doxygen understands; these are called input filters and are configured with the INPUT_FILTER setting. This is the command line of a process or batch file to execute for each source file and in the case of sql2doxygen it’s a PowerShell script that turns T-SQL code into C.

The sql2doxygen .zip package contains the Doxygen filter script and a manual describing what T-SQL styles the script understands. It is of course available from the same page I’ve mentioned twice already above. Both SS-Unit and SS-Cop are documented with Doxygen using the sql2doxygen filter and so examples of the output are available online here and here.

[*] These tools are not deployed into production and so if we picked, say, an open source project that went stale and became unusable, or bought a tool for a lesser known vendor that went bust the direct impact on the business is none. There is of course an indirect impact because we use the tools to help us deliver faster and with better quality and that would be compromised.

[+] Although the stuff we knocked up at my client’s is not the kind of thing I could imagine they would ever try and turn into a product (IT is not their business for starters) the code is clearly their property. I also wouldn’t know where to start trying to get permission to release it. Then there is the whole Fred Brooks approach which is to just throw it away anyway and learn from your mistakes instead. And this is the approach I have decided to take here.

Null String Reference vs Empty String Value

2011-11-28T22:33:00.001Z

One of the first things I found myself in two minds about when making the switch from C++ to C# was what use to represent an empty string - a null reference or an empty string (i.e. String.Empty).

C++ - Value Type Heaven

In C & C++ everything is a value type by default and you tend to avoid dynamically allocated memory as a matter of course, especially for primitive types like an int. This leads to the common technique of using a special value to represent NULL instead:-

int pos = -1;

Because of the way strings in C are represented (a pointer to an array of characters) you can use a NULL pointer here instead, which chalks one up for the null reference choice:-

int pos = -1;
const char* name = NULL;

But in C++ you have a proper value type for strings - std::string - so you don’t have to delve into the murky waters of memory management[*]. Internally it still uses dynamic memory management[+] to some degree and so taking the hit twice (i.e. const std::string*) just so you can continue to use a NULL pointer seems criminal when you can just use an empty string instead:-

int pos = -1;
std::string name(“”);

I guess that evens things up and makes the use of a special value consistent across all value types once again; just so long as you don’t need to distinguish between an empty string value and “no” string value. But hey, that’s no different to the same problem with -1 being a valid integer value in the problem domain. The c++ standard uses this technique too (e.g. string::npos) so it can’t be all bad…

If you’re less concerned about performance, or special values are a problem, you can happily adopt the use of the Swiss-army knife that is shared_ptr<T>, or the slightly more discoverable boost::optional<T> to handle the null-ness consistently as an attribute of the type:-

boost::shared_ptr<int> pos;
boost::optional<std::string> name;

if (name)
{
std::string value = *name;
. . .

C# - Reference Type Heaven

At this point in my transition from C++ to C# I’m still firmly in the empty string camp. My introduction to reference-based garbage-collected languages where the String type is implemented as a reference-type, but has value-type semantics does nothing to make the question any easier to answer. Throw the idea of “boxing values” into the equation (in C# at least) and about the only thing that you can say is that performance is probably not going to matter as much as you’re used to and so you should just let go. But, you’ll want something a little less raw than doing this though for primitive types:-

object pos = null;
string name = null;

if (pos != null)
{
int value = (int)pos;
. . .

If you’re using anything other than an archaic[$] version of C# than you have the generic type Nullable<T> at your disposal. This, with its syntactic sugar provided by the C# compiler, provides a very nice way of making all value types support null:-

int? pos;
string name;

if (pos.HasValue())
{
int value = pos.Value;
. . .

So that pretty much suggests we rejoin the “null reference” brigade. Surely that’s case closed, right?

Null References Are Evil

Passing null objects around is a nasty habit. It forces you to write null-checks everywhere and in nearly all cases the de-facto position of not passing null’s can be safely observed with the runtime picking up any unexpected violations. If you’re into ASSERTs and Code Contracts you can annotate your code to fail faster so that the source of the erroneous input is highlighted at the interface boundary rather than as some ugly NullReferenceException later. In those cases where you feel forcing someone to handle a null reference is inappropriate, or even impossible, you you can always adopt the Null Object pattern to keep their code simple.

In a sense an empty string is the embodiment of the Null Object pattern. The very name of the static string method IsNullOrEmpty() suggests that no one is quite sure what they should do and so they’ll just cover all bases instead. One of the first extension methods I wrote was IsEmpty() because I wanted to show my successor that I was nailing my colours to the mast and not “just allowing nulls because I implemented using a method that happens to accept either”.

Sorry, No Answers Here… Move Along

Sadly the only consistency I’ve managed to achieve so far is in and around the Data Access Layer where it seems sensible to map the notion of NULL in the database sense to a null reference. The Nullable<T> generic fits nicely in with that model too.

But in the rest of the codebase I find myself sticking to the empty string pattern. This is probably because most of the APIs I use that return string data also tend to return empty strings rather than null references. Is this just Defensive Programming in action? Do library vendors return empty strings in preference to null references because their clients have a habit of not checking properly? If so, perhaps my indecision is just making it harder for them to come to a definitive answer…

[*] Not that you would anyway when you have the scoped/shared_ptr<T> family of templates to do the heavy lifting and RAII runs through your veins.

[+] Short string optimisations notwithstanding.

[$] I know I should know better than to say that given there are people still knee deep in Visual C++ 6 (aka VC98)!

I Know the Cost but Not the Value

2011-11-23T22:13:00.001Z

[I wrote the vast majority of this post “the morning after the night before”. I hope that 9 months later I’ve managed to finish it in a more coherent manner than I started it…]

Back in February I made a rare appearance at the eXtreme Tuesday Club and got to talk to some of the movers and shakers in the Agile world. Professionally speaking I’ve mostly been on the outside of the shop looking in through the window wondering what all the commotion is inside. By attending XTC I have been hoping to try and piece together what I’ve seen and read about the whole “movement” and how that it works in practice. As I documented in an earlier post “Refactoring – Do You Tell Your Boss?” I’ve long been struggling with the notion of Technical Debt and how I can help it get sold correctly to those who get to decide when we draw on it and when it’s paid back.

That night, after probably a few too many Staropramen, I finally got to enter into a discussion with Chris Matts on the subject and what I can’t decide is whether we were actually in agreement or, more likely, that I didn’t manage to explain correctly what I believe my relationship is with regards to identifying Cost and Value; in short, I can the former (Cost), but not the latter (Value). So maybe my understanding of what cost and value are is wrong and that’s why I didn’t quite get what he was saying (although I can’t discount the effects of the alcohol either). This post is therefore an opportunity for me to put out what I (and I’m sure some of my colleagues agree) perceive to be our place in the pecking order and the way this seems to work. I guess I’m hoping this will either provide food for thought, a safe place for other misguided individuals or a forum in which those in the know can educate the rest of us journeymen...

From the Trenches

The following example is based on a real problem, and at the time I tried to focus on what I perceived to be the value in the fixes so that the costs could be presented in a fair way and therefore an informed[*] choice would then be made.

The problem arose because the semantics of the data from an upstream system changed such that we were not processing as much data as we should. The problem was not immediately identified because the system was relatively new and so many upstream changes had been experienced that it wasn’t until the dust started to settle that the smaller issues were investigated fully.

The right thing to do would be to fix the root problem and lean on the test infrastructure to ensure no regressions occurred in the process. As always time was perceived to be a factor (along with a sprinkling of politics) and so a solution that involved virtually no existing code changes was also proposed (aka a workaround). From a functional point of view they both provide the same outcome, but the latter would clearly incur debt that would need to be repaid at some point in the future.

Cost vs Value

It’s conceivable that the latter could be pushed into production sooner because there is notionally less chance of another break occurring. In terms of development time there wasn’t much to choose between them and so the raw costs were pretty similar, but in my mind there was a significant difference in value.

Fixing the actual problem clearly has direct value to the business, but what is the difference in value between it being fixed tomorrow, whilst incurring a small amount of debt, and it being fixed in 3 days time with no additional debt? That is not something I can answer. I can explain that taking on the debt increases the risk of potential disruption to subsequent deliveries but only they are in a position to quantify what a slippage in schedule would cost to them.

Perhaps I’m expected to be an expert in both the technology and the business? Good luck with that. It feels to me that I provide more value to my customer by trying to excel at being a developer so that I can provide more accurate estimates, which are very tangible, than trying to understand the business too in the hope of aiding them to quantify the more woolly notion of value. But that’s just the age old argument of Generalists vs Specialists isn’t it? Or maybe that’s the point I’m missing - that the act of trying to quantify the value has value in itself? If so, am I still the right person to be involved in doing that?

I’m clearly just starting out on the agile journey and have so much more to read, mull over and discuss. This week saw XP Day which would have been the perfect opportunity to further my understanding but I guess I’ll have to settle for smaller bites at the eXtreme Tuesday Club instead - if I could only just remember to go!

Epilogue

The workaround cited in the example above is finally going to be removed some 10 months later because it is hopelessly incompatible with another change going in. Although I can’t think of a single production issue caused by the mental disconnect this workaround created, I do know of a few important test runs the business explicitly requested that were spoilt because the workaround was never correctly invoked and so the test results were useless. Each one of these runs took a day to set up and execute and so the costs to the development team has definitely been higher. I wonder how the business costs have balanced out?

[*] Just writing the word “informed” makes me smile. In the world of security there is the Dancing Pigs problem which highlights human nature and our desire for “shiny things”. Why should I expect my customer to ever choose “have it done well” over “have it done tomorrow” when there are Dancing Pigs constantly on offer?

Cookbook Style Programming

2011-11-22T22:03:00.001Z

Changing a tyre on a car is reasonably simple. In fact if I wasn’t sure how to do it I could probably find a video on YouTube to help me work through it. Does that now make me a mechanic though? Clearly not. If I follow the right example I might even do it safely and have many happy hours of motoring whilst I get my tyre fixed. But, if I follow an overly simplistic tutorial I might not put the spare on properly and cause an imbalance, which, at best will cause unnecessary wear on the tyre and at worst an accident. Either way the time lapse between cause and effect could be considerable.

If you’re already thinking this is a rehash of my earlier post “The Dying Art of RTFM”, it’s not intended to be. In some respects it’s the middle ground between fumbling in the dark and conscious incompetence...

The Crowbar Effect

The StackOverflow web site has provided us with a means to search for answers to many reoccurring problems, and that’s A Good Thing. But is it also fostering a culture that believes the answer to every problem can be achieved just by stitching together bunches of answers? If the answer to your question always has a green tick and lots of votes then surely it’s backed by the gurus and so is AAA grade; what can possibly be wrong with advice like that?

while (!finished)
{
var answer = Google_problem();
answer.Paste();
}

At what point can you not take an answer at face value and instead need to delve into it to understand what you’re really getting yourself into?

For example, say I need to do some database type stuff and although I’ve dabbled a bit and have a basic understanding of tables and queries I don’t have any experience with an ORM (Object Relational Mapper). From what I keep reading an ORM (alongside an IoC container) is a must-have these days for any “enterprise” sized project and so I’ll choose “Super Duper ORM” because it’s free and has some good press.

So I plumb it in and start developing features backed by lots of automated unit & integration tests. Life is good. Then we hit UAT and data volumes start to grow and things start creaking here and there. A quick bit of Googling suggests we’re probably pulling way too much data over the wire and should be using Lazy Loading instead. So we turn it on. Life is good again. But is it? Really?

Are You Qualified to Make That Decision?

Programmers are problem solvers and therefore it’s natural for them to want to see a problem through to the end, hopefully being the one to fix it on the way. But fixing the problem at hand should be done in a considered manner that weighs up any trade-offs; lest the proverbial “free-lunch” rears its ugly head. I’m not suggesting that you have to analyse every possible angle because quite often you can’t, in which case one of the trade-offs may well be the very fact that you cannot know what they are; but at least that should be a conscious decision. Where possible there should also be some record of that decision such as a comment in the code or configuration file, or a maybe a task in the bug database to revisit the decision at a later date when more facts are available. If you hit a problem and spent any time investigating it you can be pretty sure that your successor will too; but at least you’ll have given them a head start[*].

One common source of irritation is time-outs. When a piece of code fails due to a timeout the natural answer seems to be to just bump the timeout up to some ridiculous value without regard to the consequences. If the code is cut-and-pasted from an example it may well have an INFINITE timeout which could make things really interesting further down the line. For some reason configuration changes do not appear to require the same degree of consideration as a code change and yet they can cause just as much head-scratching as you try to work out why one part of the system ended up dealing with an issue that should have surfaced elsewhere.

Lifting the Bonnet

I’ll freely admit that I’m the kind of person who likes to know what’s going on behind the scenes. If I’m going to buy into some 3rd party product I want to know exactly how much control I have over it because when, not if, a problem surfaces I’m going to need to fix it or work around it. And that is unlikely to be achievable without giving up something in return, even if its psychological[+] rather than financial.

However I accept that not everyone is like that and some prefer to learn only enough to get them through the current problem and onto the next one; junior programmers and part-time developers don’t know any better so they’re [partly] excused. In some environments where time-to-market is everything that attitude is probably even highly desirable, but that’s not the environment I’ve ended up working in.

It’s Just a Lack of Testing...

It’s true that the side-effects of many of the problems caused by this kind of mentality could probably be observed during system testing - but only so long as your system tests actively invoke some of the more exceptional behaviour and/or are able to generate the kinds of excessive loads you’re hoping to cope with. If you don’t have the kind of infrastructure available in DEV and/or UAT for serious performance testing, and let’s face it most bean counters would see it as the perfect excuse to eliminate “waste”, you need to put even more thought into what you’re doing - not less.

[*] To some people this is the very definition of job security. Personally I just try hard not to end up on the The Daily WTF.

[+] No one likes working on a basket-case system where you don’t actually do any development because you spend your entire time doing support. I have a mental note of various systems my colleagues have assumed me I should avoid working on at all costs...

PowerShell & .Net - Building Systems as Toolkits

2011-11-19T00:05:00.001Z

I first came across COM back in the mid ‘90s when it was more commonly known by the moniker OLE2[*]. I was doing this in C, which along with the gazillion of interfaces you had to implement for even a simple UI control, just made the whole thing hard. I had 3 attempts at reading Kraig Brockschmidt’s mighty tome Inside OLE 2 and even then I still completely missed the underlying simplicity of the Component Object Model itself! In fact it took me the better part of 10 years before I really started to appreciate what it was all about[+]. Not unsurprisingly that mostly came about with a change in jobs where it wasn’t C++ and native code across the board.

COM & VBScript

Working in a bigger team with varied jobs and skillsets taught me to appreciate the different roles languages and technologies play. The particular system I was working on made heavy use of COM internally, purely to allow the VB based front-end and C++ based back-ends communicate. Sadly the architecture seemed upside down in many parts of the native code and so instead of writing C++ that used STL algorithms against STL containers with COM providing a layer on top for interop, you ended up seeing manual ‘for’ loops that iterated over COM collections of COM objects that just wrapped the underlying C++ type. COM had somehow managed to reach into the heart of the system.

Buried inside that architecture though was a bunch of small components just waiting to be given another life - a life where the programmer interacting with it isn’t necessarily one of the main development team but perhaps a tester, administrator or support analyst trying to diagnose a problem. You would think that automated testing would be a given in such a “component rich” environment, but unfortunately no. The apparent free-lunch that is the ability to turn COM into DCOM meant the external services were woven tightly into the client code - breaking these dependencies was going to be hard[$].

One example of where componentisation via COM would have been beneficial was when I produced a simple tool to read the compressed file of trades (it was in a custom format) and filter it based on various criteria, such as trade ID, counterparty, TOP 10, etc. The tool was written in C++, just like the underlying file API, and so the only team members that could work on it were effectively C++ developers even though the changes were nearly always about changing the filtering options; due to its role as a testing/support tool any change requests would be well down the priority list.

What we needed was a component that maintains the abstraction - a container of trades - but could be exposed, via COM, so that the consumption of the container contents could just as easily be scripted; such as with VBScript which is a more familiar tool to technical non-development staff. By providing the non-developers with building blocks we could free up the developers to concentrate on the core functionality. Sadly, the additional cost of exposing that functionality via COM purely for the purposes of non-production reasons is probably seen as too high, even if the indirect benefits may be an architecture that lends itself better to automated testing which is a far more laudable cause.

PowerShell & .Net

If you swap C#/.Net for C++/COM and PowerShell for VBScript in the example above you find a far more compelling prospect. The fact that .Net underpins PowerShell means that it has full access to every abstraction we write in C#, F#, etc. On my current project this has caused me to question the motives for even providing a C# based .exe stub because all it does is parse the command line and invoke a bootstrap method for the logging, configuration etc. All of this can, and in fact has been done in some of the tactical fixes that have deployed.

The knock-on effects of automated testing right up from the unit, through integration to system level means that you naturally write small loosely-coupled components that can be invoked by a test runner. You then compose these components into ever bigger ones, but the invoking stub, whether a test harness or production process rarely changes because all it does is invoke factories and stitch together a bunch of service objects before calling the “logical” equivalent of main.

Systems as Toolkits

Another way of building a system then is not so much as a bunch of discrete processes and services but more a toolkit from which you can stitch together the final behaviour in a more dynamic fashion. It is this new level of dynamism that excites me most because much of the work in the kind of batch processing systems I’ve worked on recently has focused on the ETL (Extract/Transform/Load) and reporting parts of the picture.

The systems have various data stores that are based around the database and file-system that need to be abstracted behind a facade to protect the internals. This is much harder in the file-system case because it’s too easy to go round the outside and manipulate files and folders directly. By making it easier to invoke the facade you remove the objections around not using it and so retain more control on the implementation. I see a common trend of moving from the raw file-system to NoSQL style stores that would always require any ad-hoc workarounds to be cleaned up. This approach provides you with a route to refactoring your way out of any technical debt because you can just replace blobs of script code with shiny new unit-tested components that are then slotted in place.

On the system I’m currently working with the majority of issues seem to have related to problems caused by upstream data. I plan to say more about this issue in a separate post, but it strikes me that the place where you need the utmost flexibility is in the validation and handling of external data. My current project manager likens this to “corrective optics” ala the Hubble Space Telescope. In an ideal world these issues would never arise or come out in testing, but the corporate world of software development is far from ideal.

Maybe I’m the one wearing rose tinted spectacles though. Visually I’m seeing a dirty great pipeline, much like a shell command line with lots of greps, sorts, seds & awks. The core developers are focused on delivering a robust toolkit whilst the ancillary developers & support staff plug them together to meet the daily needs of the business. There is clearly a fine line between the two worlds and refactoring is an absolute must if the system is to maintain firm foundations so that the temptation to continually build on the new layers of sand can be avoided. Sadly this is where politics steps in and I step out.

s/PowerShell/IronXxx/g

Although I’ve discussed using PowerShell on the scripting language side the same applies to any of the dynamic .Net based languages such as IronPython and IronRuby. As I wrote back in “Where’s the PowerShell/Python/IYFSLH*?” I have reservations about using the latter for production code because there doesn’t appear to be the long-term commitment and critical mass of developers to give you confidence that it’s a good bet for the next 10 years. Even PowerShell is a relative newcomer on the corporate stage and probably has more penetration into the sysadmin space than the development community which still makes it a tricky call.

The one I’m really keeping my eye on though is F#. It’s another language whose blog I’ve followed since its early days and even went to a BCS talk by its inventor Don Syme back in 2009. It provides some clear advantages to PowerShell such as its async behaviour and now that Microsoft has put its weight behind F# and shipped it as part of the Visual Studio suite you feel it has staying power. Sadly its functional nature may keep it out of those hands we’re most interested in freeing.

I’ve already done various bits of refactoring on my current system to make it more amenable for use within PowerShell and I intend to investigate using the language to replace some of the system-level test consoles which are nothing but glue anyway. What I suspect will be the driver for a move to a more hybrid model will be the need automate further system-level tests, particularly of the regression variety. The experience gained here can then feed back into the main system development cycle to act as living examples.

[*] In some literature it stood for something - Object Linking & Embedding, and in others it just was the letters OLE. What was the final outcome, acronym or word?

[+] Of course by then .Net had started to take over the world and [D]COM was in decline. This always happens. Just as I really start to get a grip on a technology and finally begin to understand it the rest of the world has moved on to bigger and better things…

[$] One of the final emails I wrote (although some would call it a diatribe) was how the way COM had been used within the system was the biggest barrier to moving to a modern test driven development process. Unit testing was certainly going to be impossible until the business logic was separated from the interop code. Yes, you can automate tests involving COM, but why force your developers to be problem domain experts and COM experts? Especially when the pool of talent for this technology is shrinking fast.

Merging Visual Studio Setup Projects - At My Wix End

2011-11-15T22:12:00.001Z

When the Windows Installer first appeared around a decade ago (or even longer?) there was very little tooling around. Microsoft did the usual thing and added some simple support to Visual Studio for it (the .vdproj project type); presumably keeping it simple for fear of raising the ire of another established bunch of software vendors - the InstallShield crowd. In the intervening years it appears to have gained a few extra features, but nothing outstanding. As a server-side guy who has very modest requirements I probably wouldn’t notice where the bells and whistles have been added anyway. All I know is that none of the issues I have come across since day one have ever gone away...

Detected Dependencies Only Acknowledges .DLLs

In a modern project where you have a few .exes and a whole raft of .dlls it’s useful that you only have to add the .exes and Visual Studio will find and add all those .dll dependencies for you. But who doesn’t ship the .pdb files as well*? This means half the files are added as dependencies and the other half I have to go and add manually. In fact I find it easier to just exclude all the detected dependencies and just manually add both the .dll and .pdb; at least then I can see them listed together as a pair in the UI and know I haven’t forgotten anything.

Debug & Release Builds

In the native world there has long been a tradition of having at least two build types - one with debug code in and one optimised for performance. The first is only shipped to testers or may be used by a customer to help diagnose a problem, whereas the latter is what you normally ship to customers on release. The debug build is not just the .exe, but also the entire chain of dependencies as there may be debug/release specific link-time requirements. But the Setup Project doesn’t understand this unless you make it part of your solution and tie it right into your build. Even then any third party dependencies you manually add causes much gnashing of teeth as you try and make the “exclude” flag and any subsequent detected dependencies play nicely together with the build specific settings.

Merging .vdproj Files

But, by far my biggest gripe has always been how merge-unfriendly .vdproj files are. On the face of it, being a text file, you would expect it to be easy to merge changes after branching. That would be so if Visual Studio stopped re-generating the component GUID’s for apparently unknown reasons, or kept the order of things in the file consistent. Even with “move block detection” enabled in WinMerge often all you can see is a sea of moved blocks. One common (and very understandable) mistake developers make is to open the project without the binaries built and add a new file which can cause all the detected dependencies to go AWOL. None of this seems logical and yet time and time again I find myself manually merging the changes back to trunk because the check-in history is impenetrable. Thanks goodness we put effort into our check-in comments.

WiX to the Rescue

WiX itself has been around for many years now and I’ve been keen to try it out and see if it allows me to solve the common problems listed above. Once again for expediency my current project started out with a VS Setup Project, but this time with a definite eye on trying out WiX the moment we got some breathing space or the merging just got too painful. That moment finally arrived and I’m glad we switched; I’m just gutted that I didn’t do the research earlier because it’s an absolute doddle to use! For server-side use where you’re just plonking a bunch of files into a folder and adding a few registry keys it almost couldn’t be easier:-

<?xml version="1.0"?>
<Wix xmlns="http://schemas.microsoft.com/wix/2006/wi">
<Product Id="12345678-1234-. . ."
           Name="vdproj2wix"
           Language="1033"
           Version="1.0.0"
           Manufacturer="Chris Oldwood"
           UpgradeCode="87654321-4321-. . .">

</Product>
</Wix>

The format of the .wxs file is beautifully succinct, easy to diff and merge, and yet it supports many advanced features such as #includes and variables (for injecting build numbers and controlling build types). I’ve only been using it recently and so can’t say what versions 1 & 2 were like but I can’t believe it’s changed that radically. Either way I reckon they’ve got it right now.

Turning a .wxs file into an .msi couldn’t be simpler either which also makes it trivial to integrate into your build process:-

candle vdproj2wix.wxs
if errorlevel 1 exit /b 1

light vdproj2wix.wixobj
if errorlevel 1 exit /b 1

My only gripe (and it is very minor) is with the tool naming. Yes it’s called WiX and so calling the (c)ompiler Candle and the (l)inker Light is cute the first few times but now the add-ons feel the need to carry on the joke which just makes you go WTF? instead.

So does it fix my earlier complaints? Well, so far, yes. I’m doing no more manual shenanigans than I used to and I have something that diffs & merges very nicely. It’s also trivial to inject the build number compared with the ugly VBScript hack I’ve used in the past to modify the actual .msi because the VS way only supports a 3-part version number.

Converting Existing .vdproj Files to .wxs Files

Although as I said earlier the requirements of my current project were very modest I decided to see if I could write a simple script to transform the essence (i.e. the GUID’s & file list) of our .vdproj files into a boiler-plate .wxs file. Not unsurprisingly this turned out to be pretty simple and so I thought I would put together a clean-room version on my web site for others to use in future - vdproj2wix. Naturally I learnt a lot more about the .wxs file format in the process** and it also gave me another excuse to learn more about PowerShell.

If you’re now thinking that this entire post was just an excuse for a shameless plug of a tiny inconsequential script you’d be right - sort of. I also got to vent some long standing anger too which is a bonus.

So long .vdproj file, I can’t say I’m going to miss you...

* OK, in a .Net project there is less need to ship the .pdbs but in a native application they (or some other debug equivalent such as .dbg files) are pretty essential if you ever need to deal with a production issue.

** Curiously the examples in the tutorial that is linked to from the WiX web site have <File> elements with many redundant attributes. You actually only need the Source attribute as a bare minimum.

C#/SQL Integration Testing With NUnit

2011-10-09T00:57:00.001+01:00

Just over 18 months ago I wrote a post about Integration Testing using NUnit, and at the end I stated that I would cover database integration testing in a separate post. Well here it is. Finally!

The first thing to clear up is what scope of testing we’re covering here because it seems that many teams test their database entirely through their Data Access Layer. Now, I’m not saying that’s wrong, but in “You Write Your SQL Unit Tests in SQL?” I describe the reasons why I believe that it’s not the only approach and that in fact there is great value in doing it separately. So, in this post I’m starting out with a fundamental assumption that you’ve already unit tested both your SQL and C# code and that what we’re doing here is building on that level of trust.

Testing what then?

If we’re not testing the behaviour of our SQL or C# code then you may rightly ask what exactly are we testing? Well, as the name Integration Test implies we’re focusing on the interaction between our C# and SQL code. What sits between these two worlds is the client-side database API (which itself is probably layered) and the database server - a lot of code. Not only that but these two worlds have different ideas of how numbers and strings are represented and errors handled.

There are two kinds of common errors that can only come out at this level of testing (and higher) - schema changes and data type mismatches. The former is essentially a breaking change in the database’s public interface, such as the renaming or re-ordering of parameters and result set columns; the breakage can be quite subtle sometimes[*]. A draconian change policy would try to ensure this never happens, but then that also closes the door to refactoring the database schema too. The second common error revolves around data representation, with floating-point numbers being the prime example as databases often have a far more flexible way of defining the scale and precision of non-integral type numbers. Unexpected trailing whitespace caused by the use of a char(n) instead of varchar(n) type can cause surprises too.

These effects are made all the more apparent when you have separate SQL and C# developers because their timescales may differ, along with their code commits, and that can cause feature-level integration testing to be passed up in favour of going straight to system-level testing.

Schema only, no data

It is a pre-perquisite that before you can run your C# tests against your database you must ensure that it is in a known empty state. This means that you cannot just restore a production database, unless you also empty it of data. The alternative is to build it from scratch before running the entire test suite. And that is what we do.

However, because this can take some time we optimise the process. The Continuous Integration build first builds the database and runs the set of SQL tests, then it builds the C# code and runs those tests too. Finally it re-uses the SQL unit test database to run the integration tests, and because this is one single build & testing sequence the SQL/C# source code is guaranteed to be consistent (assuming the check-ins are themselves atomic and consistent at feature-level).

For our day-to-day development we optimise the process even further by creating a one-off backup of the current production schema and then restoring that each time we want to run the integration tests (after having applied any new patches). This restore & patch approach takes just minutes compared to the length of time it takes to create the database from scratch.

The test framework

The tests themselves are written almost exactly as you would write any other in NUnit, but we have our own helpers which are exposed via a base class to work around some of the problems inherent in using a unit testing framework to do other sorts of testing. For example the biggest problem is that you have the large dependency that is The Database to somehow configure for use within the test, and for this we use an environment variable. I generally dislike any sort of configuration like this normally[+], but it’s a necessary evil, particularly when you can’t just assume that every developer (and the build machine) will be using a local copy of SQL Server Express. By default we all have a PERSONAL_DATABASE variable configured that we use day-to-day and can be discovered even if testing through Visual Studio/Resharper. If the need arises a batch file can always be used to redirect the tests to another instance with little fuss.

The skeleton of a test looks something like this:-

[TestFixture, Category(”Database”)]
public class CustomerDatabaseTests : DatabaseTestBase
{
[Test]
public void Add_ShouldInsertCustomer()
{
    using (var connection = Connection)
    {
      . . .
    }
}
}

As you can see we use a separate category for the database tests so that you only run them when you know your “personal database” is correctly built. The fixture derives from the DatabaseTestBase class as that is how all the helpers are exposed, such as the Connection property that reads the connection string stored in the PERSONAL_DATABASE variable and serves up a freshly opened connection for it.

One design constraint this implies is that we use Parameterise From Above, rather than ugly Singletons to get our connection passed to the code under test. This a good design practice anyway and we exploit that further by ensuring that we control the outer transaction that surrounds it. This is made possible because we also have our own thin wrapper around the underlying .Net database classes. Controlling the real transaction means we can undo all changes made in the test by simply rolling back the transaction no matter which way the test ends (e.g. intercepting Commit()).

Sadly the need to manually scope the connection with a using() requires a level of discipline when writing the test as otherwise the effects can leak across tests. The alternative, which has yet to be implemented by us, is to use that old stalwart Execute Around Method so that we can add as many smarts as we need in the helper method to ensure the test is as isolated as possible. This would make the test look like this instead:-

[Test]
public void Add_ShouldInsertCustomer()
{
Execute
(
    (connection) =>
    {
      . . .
    }
);
}

Arranging

The first part of any test “dance” involves the arrangement of the dependent data and objects - which is hopefully minimal. With databases though it is common practice to use referential integrity to keep the data sane and so that may mean you have add some static data to avoid falling foul of it; it’s either that or drop the constraints which are there exactly to help find bugs.

NUnit supports both fixture and test level SetUp() methods to help keep them readable by factoring out common code. As a general rule we have used the fixture level setup for static (or reference) data that is orthogonal to the test and then used the other method or the test itself for data that is directly relevant to it. You often find yourself inserting the same static data time and again and so you have a choice of whether to create a C# based helper class or create some helper stored procedures in a separate schema so they can be shared across both the SQL and C# worlds, e.g.

[TestFixtureSetUp]
public void FixtureSetUp()
{
using (var connection = Connection)
{
    connection.Execute(“exec
                      test.InsertCustomerAndDetails
                      (1, ‘Bob’, ‘London’, . . .);”);
}
}

Acting

The second part of the dance - acting - was pretty much covered in the overview of the framework as you just need to get your test controlled connection into the code under test.

Asserting

The final step is to verify the outcome by writing a bunch of asserts. If you’re just reading data then you can use the same style of setup shown above and then use the normal features of NUnit that you’d use to compare in-memory based sequences. But if you’re writing data then it’s a slightly more complicated affair and this is another area where a bunch of helpers are required. We still use the basic Assert mechanism, but invoke custom methods to query aspects of the data we expect to have written, e.g.

Assert.That(RowCount(“Customer”), Is.Equal.To(1));

Depending on the system you’re building you may be able to round-trip the data and get two tests for the price of one, as long as any caching is disabled between the write/read calls:-

{
var expected = new Customer(1, . . .);

dataMapper.InsertCustomer(expected);
var actual = dataMapper.FindCustomer(expected.Id);

Assert.That(actual.Name, Is.EqualTo(expected.Name));
}

However testing more than one thing at once is generally frowned upon, and rightly so because you don’t know whether the read or the write action failed. But you can also end up duplicating a lot of SQL in your test code if you don’t leverage your own API and that creates a test maintenance burden instead. If you’re purely writing data then you may have little choice but to write some form of query:-

{
. . .
string actual = String.Format(“SELECT count(*) FROM
Customer WHERE Id={0} AND Name=’{1}’”,
expected.Id, expected.Name);

Assert.That(RowCount(“Customer”), Is.Equal.To(1));
Assert.That(QueryResult(actual), Is.Equal.To(1));
}

The NUnit constraint model allows you to build your own more fluent style of interface so that you can avoid hand cranking SQL if you prefer:-

Assert.That(Table(“Customer”).HasRow.Where(“Id”).Equals(expected.Id).And(“Name”).Equals(expected.Name).Go());

One of the benefits of reusing the SQL unit test database is that you’ll be DBO and as such you’ll be able to exploit your god-like status to allow you to get access to the internals and write this sort of test, even if you’ve carefully constructed a tight interface to the database. It may feel painful writing out such long-winded queries but if you’re trying to ensure you maintain high-fidelity of your data through all the layers is there any alternative?

[*] The perfect example of a subtle breaking change comes with SQL Server and OLE-DB. In SQL Server any procedure parameter can have a default value, but with OLEDB defaulted parameters have[**] to be at the end of the argument list. A SQL developer could quite rightly add a new parameter anywhere in the argument list and provide a default value to ensure the interface appears to remain unchanged, but if its not at the end an OLE-DB based query would then fail.

[**] This may have been sorted now, but it was definitely still the case a few years ago.

[+] A new developer should be able to just install the version control client software, pull the source code (and preferably tools) locally and be able to build the software and run the tests out-of-the-box.

Unit Testing File-System Dependent Code

2011-10-05T21:43:00.001+01:00

Way back last year before being distracted by my impending ACCU conference talk I wrote a post about integration testing using NUnit. At the time I was still in two minds about whether or not it was worth the effort trying to mock the file-system API, especially given that you often have some extra layer of code between you and the file-system to actually read and parse the file, e.g. an XML reader. The alternatives seem to be either to focus on writing integration tests that actually do touch the file-system (which is reasonably quick and reliable as a dependencies go) or injecting more abstractions to create other seams through which to mock, thereby allowing you to write unit tests that get you close enough but not all the way down to the bottom.

Of course if you’re creating some sort of data persistence component, such as the aforementioned XML reader/writer, then you probably have a vested interest in mocking to the max as you will be directly accessing the file-system API and so there would be a good ROI in doing so. What I’m looking at here is the code that lightly touches the file-system API to provide higher-level behaviour around which files to read/write or recovers from known common error scenarios.

Impossible or hard to write test cases

The main incentive I have found for making the effort of mocking the file-system API is in writing tests for cases that are either impossible or very hard to write as automated integration/system tests. One classic example is running out of disk space - filling your disk drive in the SetUp() helper method is just not a realistic proposition. Using a very small RAM disk may be a more plausible alternative, but what you’re really likely to want to test is that you are catching an out-disk-space exception and then performing some contingent action. The same can apply to “access denied”[*] type errors and in both cases you should be able to get away with simulating the error by throwing when the code under test tries to open the file for reading/writing rather than when they actually try to pull/push bytes to the file (this assumes you’re doing simple synchronous I/O).

The reason this makes life easier is that the file Open() method can be a static method and that saves you having to mock the actual File object. It was whilst discussing this kind of mocking with my new team-mate Tim Barrass that we made some of the existing API mocks I had written much simpler. Whereas I had gone for the classic facade, interface and factory based implementation without thinking about it Tim pointed out that we could just implement the facade with a bunch of delegates that default to the real implementation[+]:-

namespace My.IO
{

public class File
{
public static bool Exists(string path)
{
    return Impl.Exists(path);
}

public static File Open(string path, . . .)
{
    return Impl.Open(path, . . .);
}

. . .

public static class Impl
{
    public Func<string, bool> Exists =
                              System.IO.File.Exists;
    public Func<string, . . ., File> Open =
                                 System.IO.File.Open;
    . . .
}
}

}

The default implementation just forwards the call to the real API, whereas a test can replace the implementation as they wish, e.g. [#]

{
File.Impl.Exists = (path) =>
{
return (path == @“C:\Temp\Test.txt”)
? true : false
}
}

{
File.Impl.Open = (path, . . .) =>
{
throw new UnauthorizedAccessException();
}
}

This is a pretty low cost solution to build and may well suffice if you only have this kind of restricted usage. You can easily add a simple File mock by using memory based streams if you just need to simulate simple text or binary files, but after that you’re getting into more specialised API territory.

Replacing the file-system with strings

So what about the case where you are using a 3rd party component to provide simple serialization duties? I’m not talking about large complex data graphs here like a word document, but the simpler text formats like .ini files, .csv files or the <appSettings> section of .config files. If you’ve decided to leverage someone else’s work instead of writing your own parser it’s better if the parser exposes its behaviour through interfaces, but not all do. This is especially true in C++ where there are no formal interfaces as such and concrete types or templates are the norm.

However many text file parsers also support the ability to parse data stored as an in-memory string. You can exploit this in your testing by introducing a static facade (like that above) that encapsulates the code used to invoke the parser so that it can be redirected to load an “in-memory” file instead. This allows you to avoid the performance and dependency costs of touching the actual file-system whilst remaining in full control of the test.

namespace My.IO
{

public class XmlDocumentLoader
{
public static XmlDocument Load(string path)
{
    return Impl.Load(path);
}

public static XmlDocument LoadFromFile(string path)
{
    // Load document via file-system.
    . . .
}

public static XmlDocument LoadFromBuffer(string
                                            document)
{
    // Load document from in-memory buffer.
    . . .
}

public static class Impl
{
    public Func<string, XmlDocument> Load =
                                       LoadFromFile;
}
}

}

... and here is an example test:-

{
XmlDocumentLoader.Impl.Load = (path) =>
{
    string testDocument = “<config>. . .”;

    return XmlDocumentLoader.LoadFromBuffer
                                     (testDocument);
}
}

Strictly speaking this fails Kevlin’s definition of a unit test because of the “boundary of trust” that we have crossed (into the parser), but we do control the test input and we should be able to rely on a parser giving consistent performance and results for a consistent small in-memory input and so we’re pretty close. Fundamentally it’s deterministic and isolated and most importantly of all it’s automatable.

With a more complex component like an XML parser it may even require a fair amount of work to mock even though you only use a tiny subset of its features; but that in itself may be a design smell.

Reinventing the wheel

The use of static facades is often frowned upon exactly because it isn’t possible to mock them with the non-industrial strength mocking frameworks. I’m a little surprised that the mocking frameworks focus all their attention on the mechanics of providing automated mocks of existing interfaces rather than providing some additional common facades that can be used to simplify mocking in those notoriously hard to reach areas, such as the file-system and process spawning. Perhaps these ideas just haven’t got any legs or I’m not looking hard enough. Or maybe we’re all just waiting for someone else to do it...

[*] Depending on the context an access denied could be an indication of a systemic failure that should cause alarm bells to go off or it could just be a transient error because you’re enumerating a file-system that is outside your control.

[+] As he was refactoring the lambda used for the initial implementation the notion of “methodgroups” suddenly came into focus for me. I’d seen the term and thought I roughly knew what it was about, but I still felt smug when I suggested it was a methodgroup a split second before Resharper suggested the same. Another +1 for Resharper, this time as a teaching aid.

[#] I doubt you’d ever really do a case-sensitive path name comparison, but hopefully you get the point.

Your Task Bar Can Hold More Than 7 Windows Open, but Can Your Brain?

2011-10-03T21:21:00.001+01:00

Here is something that never fails to amuse me when watching other users[#] - the number of windows open on their desktop. The task bar is an interesting idea that has somehow managed to stand the test of time. Personally I miss the old 16-bit Windows desktop with all those neat little apps that had animated minimised icons like Coffee Mug, Cigarette, Bit Recycler and Tiny Elvis[+]. Exactly what metaphor is the task bar aiming at? Whatever it is it isn’t working because users are still moaning about the fact that they can’t see which icon is the one they’re looking for. And grouping similar icons is just a band aid that was only needed in the first place to overcome the ridiculous one-window-per-web-page model of everyone’s most hated web browser - IE6.

Here’s the thing though. You do know that you can close those windows, don’t you? You may have heard of The Window Tax, but that was an historical event and anyway it was based on the number of windows you had, not how many times you opened them. Making the task bar bigger or stretching it across multiple desktops still doesn’t make it any easier because the fundamental limitation is in your short term memory, not the size of your desktop or how many monitors you can wire up. A long time ago (but in the same galaxy) I went to university where I was taught that the brain can only store 7 “chunks” of short term information (give or take a couple of extra slots) and so you can open as many windows as you like, but you’ll never remember why you opened them all and that’s why the task bar looks cluttered and confusing.

My wife complains that my short term memory is more limited than that of a goldfish, but that’s not true. It’s just that it gets flushed whenever it encounters shiny things, and unfortunately the world is full of shiny things. Therefore I have a policy of closing windows the moment I believe I don’t need them anymore. This also has the jolly useful side-effect of meaning I don’t fall foul to those niggling reliability problems in applications caused by long-term use and I’ve never had the pleasure of (unintentionally) exhausting the Windows desktop heap.

I know these days everyone’s big on recycling and so shouldn’t I be leaving a few spare Explorer and IE windows around rather than expending fresh carbon on firing up new instances whenever the need arises? I don’t think so because most mainstream applications start up pretty quickly these days[~] and companies like Microsoft spend more effort on improving the warm start-up rather than the cold start-up time. Also by judicious use of favourites, network places, desktop icons, etc. I can navigate to the 80% of places I use most often within a few clicks which probably balances out the time lost “tool tipping” all the task bar icons looking for an existing window that I can reuse...

...Oh, wait, so that’s what the Search window in the Start menu is for - finding things on your task bar!

[#] And by users I mostly mean other developers. And my wife. And kids.

[+] I wonder what Toggle Booleans are up to these days?

[~] The one exception I have recently come across is the “Issue Management System” that my current client uses. This literally takes minutes to start up and is a truly painful tool to use, not least because every issue I’ve tried raising is met with an error message box that fills the screen and contains more SQL than our entire database!

SQL Server Cursors - Avoiding the Duplicate FETCH NEXT

2011-09-30T22:21:00.001+01:00

The example below comes from SQL Server Books Online (or just BOL as it’s affectionately known) about how to use a cursor. I have trimmed the predicates and the body of the cursor loop to remove some unnecessary noise, but it essentially remains the same:-

DECLARE authors_cursor CURSOR FOR
SELECT au_id, au_fname, au_lname
FROM authors

OPEN authors_cursor

FETCH NEXT FROM authors_cursor
INTO @au_id, @au_fname, @au_lname

WHILE @@FETCH_STATUS = 0
BEGIN
--
-- stuff done here with inner cursor elided
--

-- Get the next author.
FETCH NEXT FROM authors_cursor
INTO @au_id, @au_fname, @au_lname
END

CLOSE authors_cursor
DEALLOCATE authors_cursor
GO

What bugs me about this code, and other examples I’ve come across[*], is that there is obvious duplication; the FETCH NEXT statement is specified twice - once before entering the loop and again at the bottom.

Now, as a software developer we are taught that cut-and-paste coding is frowned upon because you increase the cost of maintenance. Yes, it’s just a simple loop and no one is really going to forget to keep them in sync are they (especially if SQL is their bread and butter)... But, if we did want to remove the duplication, could we?

Of course we can and the revised version below is how a colleague has tackled this for many years - we just need to move the FETCH NEXT inside the loop and then change the loop condition. But this is where we swap one dubious construct (duplicate code) for another - the infinite loop:-

OPEN authors_cursor

WHILE (1 = 1)
BEGIN
-- Get the next author.
FETCH NEXT FROM authors_cursor
INTO @au_id, @au_fname, @au_lname

IF (@@FETCH_STATUS <> 0)
BREAK

--
-- stuff done here with inner cursor elided
--
END

CLOSE authors_cursor

If this code were written in a static language like C++ and you ran a Lint tool over it, it would probably wince at the (1 = 1) expression - I certainly did when I first came across it. There are some constructs that send shivers down your spine and the infinite loop is one of them because we expect the loop condition to have a natural point of exit[#]. Here is an attempt to set that record straight:-

DECLARE @status int = 0

WHILE (@status = 0)
BEGIN
-- Get the next author.
FETCH NEXT FROM authors_cursor
INTO @au_id, @au_fname, @au_lname

SET @status = @@FETCH_STATUS

IF (@status = 0)
BEGIN
    --
    -- stuff done here with inner cursor elided
    --
END
END

Single fetch statement, check. No early loop exit, check. No duplication, che... Sadly we’ve just swapped duplication of the FETCH NEXT for duplication of the loop test. Hmmm, what about swapping the BREAK for a CONTINUE and then changing the loop tests:-

DECLARE @done bit = 0

WHILE (@done = 0)
BEGIN
-- Get the next author.
FETCH NEXT FROM authors_cursor
INTO @au_id, @au_fname, @au_lname

IF (@@FETCH_STATUS <> 0)
BEGIN
SET @done = 1
CONTINUE
END

--
-- stuff done here with inner cursor elided
--
END

To be honest I think that’s cheating because we’ve still got two tests to control the loop. But the switch from a BREAK to CONTINUE somehow feels a little less dirty. Sadly my SQL skills are fairly average and so perhaps someone more experienced can come along and finish the job off properly.

[*] Hey, it’s just example code and examples are meant to be simple so that people can understand them! True, I’m just saying that I’ve not seen it done any other way when Googling for stuff that utilised cursors which is what prompted me to write it up.

[#] Even services that sit there and spin indefinitely handling requests still need to be coded to terminate gracefully at some point and so in the very least you’ll be waiting on a synchronisation object of some sort.

What’s the Check-In Frequency, Kenneth?

2011-09-29T18:37:00.001+01:00

Last week saw two unrelated events come together to provide the basis for this post. The first was the demise of R.E.M., a rock band that I would never describe myself as being a massive fan of, but via osmosis[*] aware enough for them to provide a particularly bad pun for this post’s title. The second, and far more relevant event was the discussion on site about whether it’s possible to check-in too frequently or not.

Let’s be clear from the outset that we’re talking specifically here about integration branches. Private branches are, by definition, personal and so the frequency with which a developer chooses to commit is entirely their own choice (feature branches are usually synonymous with private branches). However an integration branch serves different purposes depending on the point in time that you wish to mine it.

Feature Development

As a developer I want to integrate my changes as fast as possible to ensure that my codebase always remains fresh. The longer I leave uncommitted changes outstanding, the more I open myself up to the possibility of tricky merges and integration problems. That’s the theory.

In practice one hopes that the entire codebase is not so volatile that this actually happens. If it does then I’d be worried that there is so much refactoring going on that no one is actually getting any real work done! Of course the team size will play a large part in this, but even then you just can’t expect developers to make sensible progress if they keep tripping over each others heels - you can’t build a house on quicksand.

For new features, where there is the greater chance of a high check-in rate as the developer goes through various twists and turns, the check-ins are mostly of interest to them and anyone reviewing. Once a feature reaches v1.0 and enters maintenance mode then every check-in becomes of interest to a larger audience because you now have something to break.

Integration/System Testing

When feature is complete the code will likely be in a state where all the unit tests pass and hopefully some amount of integration and system testing has been done too (and acceptance testing, if you’re lucky enough to have them). Depending on the testability of the system though it may only be complete to unit and rudimentary integration test level and so still have to go through some form of system testing where you get to throw higher volumes of data at it.

At the point of deployment[+] into a system test environment I find that the integration branch history takes on a different perspective. Now I’m interested in what changes are going to be deployed so that I have an idea of where trouble it likely to show up, and when it does I want to be able to drill into the changes to triage the problem quickly. If you’re working on a batch processing system, nothing kills a day more than finding that a bug has caused the system to keel over at the first hurdle as you’ve effectively lost an entire test run.

Software Archaeology

Once a feature is mature it hopefully just sits in the background doing its thing. Until it breaks, or meets a condition it never expected. If it’s a regression, or you don’t understand why the problem has only just surfaced, then you may need to go back in time and trawl the VCS logs piecing together any related changes to build a picture of what’s happened. Once again this is where noise from frequent check-ins makes the job much harder and you’ll almost certainly only be interested in changes since the feature went live. Any experimentation at this point is just noise, you’re only interested in version N and N-1 and the net effects of those changes.

If you’re new to a team then you wont have any history at all to draw on and so the VCS logs may be your only way of finding out about how the system has evolved. I’ve seen plenty of strange code and it has only been through the power of the VCS time machine that I’ve managed to work out why it’s in that state and, more importantly, how I can move it forward in the intended direction[#]. However this is also one scenario where going back over the changes in a private branch can actually be useful.

Personally I believe that after the text editor, the next most important tool you really need to grok is the version control system. This is especially true when working on legacy systems as you probably have none of the original team around.

It’s Just a Tooling Problem

My case for a little restraint may well appear weak with the counter-argument being that frequent check-ins are A Good Thing and what I describe is just a problem with the VCS tool (or my inability to drive it sufficiently well). And you’d have a point. But the VCS tools I’ve uses to date are all focused around managing single (hopefully atomic) change-sets; they do not provide an easy way for you to view the multiple change-sets that constitute “a feature” as a single set. To me “multiple change-sets” sounds awfully like what you’d use a feature/private branch for… The question then becomes “how many check-ins before I should have created a separate branch”?

The Extreme Case

In case I still haven’t made a compelling argument then perhaps the following extreme case will help you see where I’m coming from, and remember that I’m talking about doing this in an integration branch:-

Every check-in should be atomic and what is checked in shouldn’t break the build or destabilise the codebase if at all possible. That means that when writing a new feature, using a process like TDD, I could check in my changes every time I write a passing test.

I would contend that doing this adds a significant amount of noise to an integration branch, and that if you want to keep every little step you make then you should create your own branch and commit to that instead. Then integrate your changes on a less frequent basis, but still frequent enough that you avoid integration issues.

DVCS to the Rescue?

One technique I’ve yet to explore is to layer a DVCS like Git or Mercurial on top of Subversion to use as a private branch repository. This sounds attractive from the noise perspective, but I wouldn’t want to give up the check-in history of the small steps entirely in case it’s useful some time in the future. I know there are a few ACCU members who are well versed in this area and so I shall have to see whether there is more to this way of working than meets the eye.

Show Me the Money

So, putting my Tom Gilb hat on I feel compelled to somehow quantify what I feel is too often and what is too little. When you’re in maintenance mode a lot of features may literally only take minutes or hours to write and so there are natural points of integration already built-in. The basis for this post though was about significant new features because there are far less checks-and-balances (e.g. regression testing) in play to stem the natural flow of commits because you having nothing to break. In this case I would look to break the task down to see if there are any natural points where I can deliver sizeable chunks (1/2 day - 1 day) of complete functionality so that it can be pushed into system test sooner and I can then reap the system testing benefits in the background. If there is absolutely no way I can deliver something concrete without destabilising the system, or if I know I could be experimenting, only then will I resort to a private branch and only for the destabilising change. Once I can get back to delivering large chunks directly into the integration branch I will.

In essence what seems to have driven the upper limit of the commit frequency for me in recent years is the rate at which system testing occurs. Even in this day and age of unit testing and continuous integration “getting something in the build” is still something to aim for when a build, deployment and end-to-end test takes hours.

[*] I used to share a house with a number of people who listened to R.E.M. a fair bit. Personally I was happy enough to leave it at Eponymous and the 12” of The One I Love; for the wife of course...

[+] This is as often as possible. In theory, on the sorts of systems I work on, it would be great for it to happen automatically after each end-to-end run so that we have a continuous cycle of build, deploy, do an end-to-end run, rinse & repeat. In practice it ends up being largely automated, with the manual element being the decision to run the cycle to ensure the highest chance of getting a successful end-to-end run.

[#] I remember one very curious piece of code that changed rapidly and had lots of simple check-ins that seemed to be trying to workaround some kind of memory management problem. I came across it whilst trying to fix a very large memory leak and from the VCS log I could deduce that the programmer was trying to emulate a weak pointer with a non-weak one, hence the eventual leak.

From C++ to C# – Two Years On

2011-09-26T22:23:00.001+01:00

Two years ago I interviewed for what was originally a C++ position, but which for internal reasons switched to being a C# role at the last minute. I was still offered the contract even though I knew virtually no C# with the expectation that I would pick it up pretty quickly as the two languages aren’t exactly worlds apart. After doing 15 years of C++ I was somewhat pensive of switching my main language, but as I wrote back then in “Why is C++ Getting Left on the Bench” the market for C++ is slowly diminishing. So of course I took it!

Comparing languages is somewhat of a dubious pursuit as the reason we have so many is exactly because they are intended to solve different types of problems. However these two cover similar ground and I’ll be using them to solve the same kinds of problems so I think it’s probably an interesting comparison on some level...

Tooling

I’ve only ever worked on Windows and Visual Studio has pretty much been the only IDE I’ve used for the past 15 years, so naturally I was instantly at home with the default toolset. The only big difference I’ve noticed is that IntelliSense actually seems to work with C#, unlike C++[#]. However our system is still quite small so it’ll be interesting to see how that scales. Build times are also pretty good, but the same caveats about project size apply.

I’m sure this is obvious to any seasoned C# programmer but the biggest win seems to come from Resharper[*], a plug-in that provides on-the-fly static code analysis to point out potential bugs, stylistic violations and all round good practices. This tool just makes the refactoring step in TDD that bit more pleasurable and easy. There is a tool in the C++ world vying for similar acclaim (Visual Assist) that I used some years ago on another contract and it does add value to Visual C++, but I suspect that C++ is just so much harder to parse than C# and so I doubt it will match something like Resharper.

Something you should bare in mind that I only write services and so have no use for all the GUI related features. After all, I still code the few html pages I do write the way I’ve always done - with a text editor.

Libraries

Resharper aside the other big win has been the bundled libraries. C++ gives you a basic set of libraries to cover simple I/O needs, string formatting etc, but avoids giving you the other things a modern developer needs - networking, xml support, threading. Ok, so this is being rectified to some degree in the new standard and Boost can fill in a huge number of gaps in the other areas - but fundamentally, out-of-the-box, you’re on your own. Being able to do so much more from the get go means the project can progress in many areas without getting bogged down in technology problems quite so soon.

Of course Windows and .Net come from the same stable and so any notion of being “cross-platform” (Mono notwithsdtanding) is a drop in the ocean compared to the vastly differing platforms that C++ aims to support. But as you should already have gleaned by now that is of little interest to me, at least professionally.

Language

At a basic language level there are so many similarities between the two, constructs such as classes and enumerations are roughly equivalent and the private/protected/public access levels are pretty close too. Yes there are various small notational differences, such as using a . instead of :: to separate namespace names, but I’ve covered this in an earlier post “C# + C++ + C++ CLI – Context Switch Hell”. I’ve also documented my comments on resource management in the past too in “IDisposable’s Should Assert In Their Finalizer”.

This means that by-and-large I’m designing and writing code in a very similar manner. But then that was to be expected as the expression of OO is through interfaces and classes and that was never going to be much different. However the inclusion of Closures (which C++ has recently gained too) has meant the Functional Programming element probably feels more natural now - Functors and Boost::Function are just very clunky mechanisms in comparison.

There are many small touches that make certain common C++ pitfalls a thing of the past, but most seasoned C++ programmers have long since learnt how to avoid making them. There are a couple of things in C# though that are still bugging me:-

Sorely Missing Const

The thing that I miss the most by far is the concept of “constness” in C++. In C# immutability is a property of a type (assuming you choose to design the class that way), not a trait of an instance of a type. One place where this really comes into play is when concurrency rears its head because immutable instances are shareable across threads. Another is when dishing data out from a cache - you want to return a const reference to ensure the instance is not accidentally modified. I have written a number of bits of code in C# already where I know that if a cache was introduced the compiler would not be able to help highlight the violations - that scares me[^].

It didn’t take me long to notice the loss of constness, but that may be largely due to me regressing to old habits such as returning a const reference to an internal container rather than exposing the functionality on the type (the proverbial Law of Demeter), or exposing iterators via begin()/end() methods (ala Extended STL). In C# IEnumerable covers similar ground and more besides, so careful thought eliminates many cases in the end.

Don’t even mention ReadOnlyCollection! That is an abomination. I’m using a statically compiled language because I want the compiler to point out my mistakes at compile time, I don’t want to rely on it happening at run time. Yes, good unit test coverage does minimise the chance of a bug slipping through the net, but I don’t understand why there isn’t an IReadOnlyList, from which IList derives?

My current system uses a complex in-house container type that can be very expensive to copy and it too provides a single interface for reading and writing. The later addition in C# of the “initialiser syntax” seems to actively promote the creation of mutable types by default[+] which is not the de facto stance to take when you want thread-safe code to be the norm and not the exception. Even if you’re not bothered about immutability, this syntax still relies on you supporting 2-phase construction which is another thing to avoid by default.

Equality Comparison is Hard

In C++ everything is essentially a value type and so comparison is simple to implement. If you try and implement it on a type and anything it aggregates doesn’t support it you’ll get notified at compile time. C# is the other way round, which is both a blessing and a curse. Manually implementing the Equals() method correctly is non-trivial and that’s why everyone uses a tool like Resharper. Steve Love and Roger Orr gave an excellent talk on the subject at this year’s ACCU conference which is what made me realise the subject is way deeper than I ever imagined.

So what’s the blessing? Well it’s caused me to try and properly understand the difference between value types and reference types. This is turn has forced me to question whether I should be providing a comparison operator on a type in the first place or am I just doing it because it used to be easy. Unit testing makes this question harder because you often see a need for comparison as your unit test asserts are usually implemented by comparing the actual and expected outcomes. Implementing Equals() once in the type seems easier, but you can always implement it as a test helper method instead and refactor later if it turns out you got it wrong.

Productivity

So do I feel more productive in C# than in C++? Probably. Slightly.

I’m one of those people that believes most time is spent in the designing phase, not in the actual coding. I spend way more time thinking about the problem and when the moment comes I probably write it in much the same way in either language. I’ve mentioned how nice it is to have a decent set of libraries on tap, and we definitely managed to leverage that in the early part of the project, but once the infrastructure was written it was pretty much business as normal.

If anything, the biggest win so far has been the tight integration between PowerShell and .Net as that has provided us with an ability to work around production issues you could only dream about in the days of C++/COM/VBScript.

[#] Throw something macro heavy like STLport into the mix and IntelliSense seems to fall apart.

[*] Nope, I have no relationship with them whatsoever, just a happy [corporate] customer.

[^] After what I said about TODOs the other day, this is one example where I have turned turned a TODO into a formal non-functional requirement. This is because the code is dangerous and building a facade over the mutable type in question is non-trivial, as is creating copies.

[+] The C# initialiser syntax relies on using public setter methods to assign values to properties. Curiously, when creating an anonymous type, you use the same syntax, but the compiler does not generate the public setters. Of course the new named argument syntax in C# 4 will hopefully push people back onto the right course once again.

TODO or TODO Not - There Is No Later

2011-09-22T00:21:00.001+01:00

The “TODO” style comment is a blight on the modern codebase. Not so much the wizard generated ones that MFC programmers never bother to remove, but ones blindly added in some vain hope that they are going to be understood and then actioned by some other programmer in their spare time. I once remember seeing a comment like this, which through the power of the version control system I discovered had been unanswered for nearly 10 years:-

// TODO: See if this is right.

One would hope that after so long it’s either right or on a dead code path that no one cares about. Hey, how about writing “a test” to answer the question instead of writing the comment? Going back to the wishful thinking style TODO I particularly liked this one I saw in a piece I code I then had to work on:-

catch(...)
{
// TODO: How to handle errors?
}

I answered the question by removing the catch-all handler and letting them propagate to the caller. Funnily enough this unearthed an occasional error that then explained some other mysterious behaviour. Performance related TODO’s always amuse me because they are so likely to be wrong when the time comes:-

// TODO: Switch to a map if this needs to be faster
std::list<. . .> something;

What are the chances that at the time performance becomes a problem that this comment will still be correct? I would never blindly make a change based on someone’s gut instinct 5 years ago about how the code is likely to evolve. If you’re worried enough that you’ve chosen the wrong container type up front to comment on it then change it. Otherwise let the profiler do the talking.

One of the first things I disable in Visual Studio is the auto-detection of TODO’s being added to the Task window. After all I don’t want to get bogged down in a sea of unscheduled work left by my predecessors. In the modern era of programming where we’re all doing TDD our entire workload is already planned out of the next few weeks in stories and any refactoring will be done along with it. You simply don’t have the time to mark a point in the code with the intention of coming back some time later to finish it off - done has to mean done.

If TODOs in the source code comments are bad enough, some teams go one step further and make use of compiler features like #pragma message in Visual C++ to make sure they also appear in the build output just in case you managed to avoid them in the code. This just makes a clean build output an impossibility as you’re constantly drawn to check out the distracting build message as it passes by each time in case it wasn’t the TODO this time but an important message…

On the last few projects I’ve worked on I’ve instigated a “No TODO can live past a release” policy. This means I grep the codebase just before a release and either promote the TODO to a fully fledged work item if it is still worthy of attention (probably with the lowest priority in the hope it’ll just go away), or more commonly, delete it. So many are just as superfluous as the code comments and will be just as apparent when the code is next visited - refactoring will happen naturally if it needs to be done.

Copy & Rename (Like Copy & Swap But For File-Systems)

2011-09-20T21:53:00.001+01:00

In C++ one of the most popular idioms for writing exception safe code is the “copy & swap” idiom. The idea is that you perform all “dangerous” work off to the side and then “commit” the changes using an exception safe mechanism[*] such as std::swap[#]. The idiom is not exactly intuitive because what you want to do most of the time is commit by assignment, but that will likely cause a copy and therefore open you up to the possibility of an exception (e.g. out of memory) which is exactly what you’re trying to avoid. In contrast the implementation of std::swap (assuming it’s been implemented efficiently for the type in question) is usually very simple and just swaps some internals members. The swapped out value is then cleaned up later.

In the world of distributed systems it’s very common to receive data in “flat files”, published by some external system that you then consume as inputs, perhaps during some batch process, before publishing your results to someone else. These files can often be quite large and take a non-trivial amount of time to write by the publishing system and so it’s important that you don’t start reading them before writing has finished[^]. If you do happen to start processing a file before it’s completely written you can end up with some real head scratching moments as you try and diagnose it later (after writing has then finished and the file is intact!).

The solution is to something akin to the copy & swap idiom, but using a file rename in place of std::swap to commit the change. The rename is just a metadata change and so torn files are avoided. It will also be safe in the face of a potential out-of-disk-space scenario as no “visible” data will be committed if the write terminates abruptly - the subscriber always gets all or nothing.

File renaming can also be used in a manner similar to the atomic CompareExchange function. When writing a service that caches data (in the file-system) and you get two simultaneous requests for the same data you have the potential for a race condition as both could try and update the cache together. What you can do is to let both continue as normal but write their data to a temporary file or folder. Then you “commit” by renaming this temporary file/folder to its final name at the end. The winner gets to be oblivious whereas the loser gets an error on renaming to signal it lost and has to cleanup before returning results; possibly out of the winners cache. If at any point an error occurs the temporary cache can be cleaned without affecting other requests.

[*] Yes it’s possible that std::swap could generate an Access Violation (or the equivalent thereof) and still cause corruption but that is no doubt considered to be covered under the realms of Undefined Behaviour.

[#] The new C++ standard introduces the concept of r-value references and std::move() which should make for a more intuitive solution in future.

[^] Assuming the file share mode allows it and you have a format that supports it you can do this but you’ll be adding a lot of complexity that you probably don’t need or can avoid through other means.

NTL/Virgin Emails Going Missing

2011-09-19T23:04:00.001+01:00

The problem with being a software developer is that sometimes you [think you] know too much and, as the old saying goes, a little knowledge can be dangerous. Such was the case with a bizarre email problem my wife seemed to be having with her new iPhone…

She told me that emails were going missing. Now we all know that end users see the world in a different light to developers and so the first thing I did was to put aside her interpretation of the problem and get some cold hard facts. She swore there were emails she’d seen on her iPhone that hadn’t come through to Outlook on the desktop machine, even though a cursory glance didn’t show up any obvious casualties. So we left it that she would keep an eye out and investigate again when the problem knowingly resurfaced.

No doubt at this point anyone who knows anything about setting up email accounts will be shouting “Leave messages on the server”. This was my gut instinct too - that the iPhone was defaulting (or had been overridden) to delete messages from the server after retrieval. Up to this point I’d had zero experience with smart phones and so it took a little longer than usual to convince me that this wasn’t happening. I naturally did a little Googling too to see if this was a known issue with iPhones but nothing showed up.

A month or so later she commented that it was still happening and so we took another look. This time she did an extensive comparison between her various email accounts (she has many because of her freelance nature) and it transpired that it was only happening with her Virgin Media account (what was originally called NTL). This new information made the problem much more specific and it triggered me remembering something about Virgin changing the email system a while back so that they were using Google under the covers, or something like that. And so once again I fired up Google, this time armed with some additional keywords.

Many of the results still seemed to be about the opposite problem caused by the “Leave messages on server” setting. But finally I found one that seemed to mirror my wife’s problem and it talked about something called “recent mode”. This didn’t look like a standard POP feature but sounded more like a workaround NTL/Virgin might have added to the email system for the problem of multiple devices trying to access the same mailbox, e.g. desktop + smart phone. At the time I couldn’t find an official word on the subject, but I’ve since revisited the problem with my new phone and found this page on the Virgin Media site[*].

On the one hand I was somewhat relieved to discover that my reasonable knowledge of POP and email wasn’t totally flawed but my diagnostic skills certainly took more of a bashing. My early assumption was that internet mail is well understood and so the problem must be device specific. Add to this my wife’s complaint about iPhones’ only supporting one Exchange server and it only strengthened my prejudice against Apples’ implementation. However once the finger was pointing to a single email provider as well things were easier. Of course now I know about “recent mode” it’s a doddle to find those articles again and the relevant page of Virgins’ web site :-)

[*] This workaround is not without its own problems. One side-effect is that you can see the same email multiple times, even after the initial fetch, depending on what email client you’re using.

Every Solution Starts With “FOR /F”

2011-07-22T00:24:00.001+01:00

Our team has recently grown and as part of that process I’ve been showing how I’ve tended to do some of the sysadmin type stuff, such as deployment, managing log files[*], checking system health etc. For a brief period it felt as if the answer to every question started with “FOR /F” - the Swiss Army Knife of Windows batch file programming. One week it also ended up being the answer to two questions via Twitter!

Given that I work on Distributed Systems is that such a shock? After all, whatever I want to do, I’m likely to want to do the same thing to every server in the farm and that implies use of some looping construct either to generate a batch file with the same command multiple times:-

@echo off
psexec \\SERVER-1 cmd.exe /c . . .
psexec \\SERVER-2 cmd.exe /c . . .
psexec \\SERVER-3 cmd.exe /c . . .
. . .

Or just one command executed multiple times, controlled by a simple list:-

for /f %h in(machines.txt) do @psexec \\%h cmd.exe /c . . .

This latter variant requires a text file (called machines.txt) with one server hostname per line like so:-

SERVER-1
SERVER-2
SERVER-3

Of course rather than crafting one on-the-fly every time I have a number of files, one called DEV-machines.txt, another called UAT-machines.txt and finally a PROD-machines.txt which are pre-configured.

Deleting Folders

Whilst PSEXEC pretty much makes an appearance in every one-liner I write, usually to invoke a command on a remote host, the simpler task of cleaning up folders in a non-sequential way doesn’t feature it for once. Sometimes you’d like a little more progress than a straight “RMDIR /S /Q” will give you (i.e. none) and so a sprinkling of FOR /D allows you to iterate a bunch of folders and output their name before recursively deleting them:-

for /d %d in (folder\2010_??_??) do @echo %d & rmdir /s /q %d

A little extra parallelism can then also easily be achieved by firing up a few more command prompts with START (which will default to the CWD of the parent) and splitting up the work:-

Prompt-1> for /d %d in (folder\2010_01_*) do @echo %d & rmdir /s /q %d

Prompt-2> for /d %d in (folder\2010_02_*) do @echo %d & rmdir /s /q %d

For those that have successfully avoided batch file programming the “&” separates commands. A “&&” ensures the second is only executed if the first succeeded, whilst a “|” does the opposite and executes the second only if the first fails.

Reading Variables

Probably one of the most unintuitive aspects of Windows batch file programming is creating variables from data in a file. For example our current Heath Robinson overnight batch scheduler uses a combination of the Windows Task Scheduler and batch files to sequence it[#]. The “batch date” is stored in a text file (BatchDate.txt) like so:-

2011-01-01

To read this into a variable so that it can then be passed onto the child scripts and processes requires the following:-

for /f %v in (\\dfs\share\BatchDate.txt) do @set BatchDate=%v

Who would of thought you need a loop to read a single value! If you want to store it as a “key=value” pair instead you need a little more magic to parse the line:-

for /f "delims== tokens=1,2" %i in (c:\temp\test.txt) do @set %i=%j

There are various options that you can pass to FOR to control the parsing, such as “eol=;” if you want to allow comments in your input file like so:-

D-SERVER-1
D-SERVER-2
; NB: Server being repaired
; D-SERVER-3

PowerShell Integration

PowerShell is clearly a better tool for doing any serious scripting under Windows but it suffers one major drawback when you’re using it as we are to sequence a set of operations - PowerShell can’t modify the variables of the calling process. If a PowerShell script forms the root process this isn’t a problem, but when old school batch files are leading the way it’s harder work. If it wasn’t for the annoying problems PowerShell has with doing simple stdout redirection[$] it would be a no brainer to switch.

Anyway, going back to our previous example of reading in variables, you can invoke a PowerShell one-liner and read the output into a variable, as I showed in this StackOverflow answer to get yesterday’s date:-

for /f "usebackq" %i in (`PowerShell ^(get-date^).adddays^(-1^).tostring^('yyyy-MM-dd'^)`) do set Yesterday=%i

However there is a fair bit of ugliness in this technique because you need to escape the parenthesis with the ^ (hat) character and to allow the date format string to be passed through correctly (it either needs to be in single or double quotes) you need to enclose the command with the ` (backtick ) character instead of the usual ‘ (apostrophe).

You Can Remember This Stuff?

Even after all this time I still can’t remember half the switches and options that you can use with “FOR” so it’s lucky that I do remember this command to bring up the manual:-

help for

Bizarrely enough it also lists the set of modifiers you can use on the loop variables to munge paths, such as getting the parent folder, file extension etc.

[*] This should of course be automated, but with limited resources it’s been well down the priority list.

[#] See [*] for the reasons why this came about :-)

[$] I already have a post queued up on this as I think it’s one of the most annoying things you discover when trying to switch from using batch files to PowerShell scripts.

Recovering From Unknown Exceptions

2011-07-07T23:23:00.001+01:00

In my last post I talked about using the Execute Around Method pattern to implement a common Big Outer Try Block to handle exceptions that propagate up as far as the service handler. But what do you do with them? Can you do anything with them? After all, if they’re unhandled you clearly never expected them so you need to recover from something you never anticipated...

Recoverability

The point of an exception handler is to allow you to recover from an exceptional condition. Now, there is much debate about what “exceptional” means but in this post I’m talking about scenarios for which no recovery has been attempted. That could be due to lack of knowledge about how to recover, or more likely, because the scenario never came out during testing.

There are really only two ways to behave after catching an unhandled exception - continue or terminate yourself. The former is often classified as the “best effort” approach, while the latter is described as “fail fast”. These are often seen as two opposing views, but in reality they are both valid approaches so long as you can decide how to loosely classify the error to ensure that the stability of the service isn’t compromised as a result of an unanticipated failure.

Systemic vs Domain Exceptions

What we are concerned with at the service handler entry point is trying to decide if the exception we caught indicates that the service has become unstable and will therefore cause problems with subsequent requests or if there will be no residual effects and we can carry on as normal (perhaps after logging a message to notify someone etc). For example a divide by zero exception is not likely to indicate that the process is stuffed whereas an access violation probably is[*].

We can split these two types of errors into two broad categories - Systemic and Domain. The former implies that there is something technically wrong with the process while the latter implies that there was something wrong with the request. We can then create two exception hierarchies rooted on them (ignoring the ultimate base such as System.Exception in .Net) - ServerException and DomainException. Then our Big Outer Try Block will look something like this:-

try
{
. . .
}
catch (ServerException e)
{
// Process borked - shutdown service
}
catch (DomainException e)
{
// Dodgy request - return error
}

3rd Party Exceptions

Of course .Net and the rest of the world won’t know about our exception hierarchy and so we’ll need to categorise any known 3rd party exceptions into the Systemic or Domain groups and then extend the list of catch handlers appropriately. Taking a leaf out of my last post you can also encapsulate the exception handlers into a separate method that will catch and translate so you reduce to amount of cutting-and-pasting:-

public void TranslateCommonErrors(Action<T> method)
{
try
{
    method();
}
catch (OutOfMemoryException e)
{
    throw new ServerException(e);
}
. . .
catch (ArgumentOutOfRangeException e)
{
    throw new DomainException(e);
}
. . .
}

We can use nested exceptions to ensure the caller has a chance to react appropriately. One of the first extension methods I (and no doubt many others) wrote was to flatten an exception hierarchy so it could be logged in all its glory or passed across the wire.

Remote Services

If we’re saying that a ServerException signifies the unstable nature of the service process then we need to translate that when it crosses a remote boundary. Due to the requirement that exception types must be serializable (at least with .Net & WCF) to cross the wire and we will be catching everything, including 3rd party exceptions which may not be serializable, we need to marshal the exception chain ourselves.

So, for remote service calls I like to have a mirror pair of exceptions - RemoteServerException and RemoteDomainException. These act as both the container for the marshalled exception chain and more importantly a signal to the client that the service is unstable. This gives the client a chance to perform recovery such as retrying the same request via a different server:-

while (!serverList.Empty())
{
try
{
    m_connection.DoTrickyStuff(parameters);
}
catch (RemoteServerException e)
{
    // Out of servers!
    if (serverList.Empty())
      throw new ServerException(“No servers left”, e);

    // Server borked - try another one
    m_connection.Close();
    m_connection.Open(serverList.NextServer());
}
catch (RemoteDomainException e)
{
    // Request stuffed - never gonna work...
    throw new DomainException(e);
}
}

Another approach could be to immediately translate the RemoteServerException into a DomainException and throw that because the stability of the remote service does not indicate any instability within the client. However, as I mentioned last time, you need to be careful here because a technical error can grow from being a local problem on one server to a bigger one affecting the entire system when all the load balancing and retry logic starts kicking in for many failed requests.

If there is one thing the recent Amazon outage shows it’s that even with lots of smart people and decent testing you still can’t predict how a complex system is going to behave under duress.

The Lazy Approach to Exception Discovery

The eagle-eyed will have noticed that I’ve managed to avoid writing one particular catch block - the ‘catch all’ handler (catch(...) in C++ and catch(System.Exception) in C#). This is the ultimate handler and it’s going to handle The Unknown Unknown, so what do you do? Experience has probably taught you what you would consider a systemic error (e.g. Out of Memory) and so you’ll likely have those bases already covered leaving you to conclude that you can apply the “best effort” rule for everything else. And that is the approach I’ve taken.

In theory (given adequate documentation[#]) for every line of code you write you should be able to determine what the set of exceptions are and come up with a plan for recovering from all of them. Or deciding not to recover. Either way you can make an informed choice about what you’re going to do. But quite frankly who actually has the time to do this (except maybe for programmers writing software where lives are at stake)? What is more likely is that you’ll know the general pitfalls of what you’re invoking, such as when dealing with file-systems, networks or databases, and you’ll explicitly test those scenarios up-front but leave out the crystal-ball gazing. I think the effort is better spent ensuring the integration test environment runs continuously and is suitably enabled for injecting plausible faults than trying to second guess what is going to go bang under any given scenario.

This lazy approach also serves a secondary purpose and that is to see if your diagnostic tooling is up to the job when the time comes to use it. Have you just logged some simple message like “it failed” or do you have a message with the entire chain of exceptions, and do they tell you enough? Did you remember to log the stack trace or create a memory dump? Did the event register on your “system console” so you get a heads-up before it goes viral?

When a new scenario does come up that you realise you can recover from more gracefully that’s the time you’ll be glad you spent the effort mocking the underlying file-system, database and networking API’s so that you can write an automated test for it.

[*] Although I’d argue that an attempt to de-reference a NULL pointer probably is recoverable because it is the most common default initialisation state. In my experience access violations involving NULL pointers have nearly always been due to some logic error and not an indication of a wayward service that has started doing random stuff that eventually ended up with a NULL pointer. On the contrary an arbitrary memory location nearly always signals something pretty bad has gone before.

[#] Exception specifications don’t appear to have alleviated that.

Execute Around Method - The Subsystem Boundary Workhorse

2011-07-04T23:34:00.001+01:00

I first came across the pattern Execute Around Method at an ACCU London talk by Paul Grenyer. The talk was about factoring out all the boilerplate code you need when dealing with databases in Java at the lower level. The concept revolves around the ubiquitous Extra Level of Indirection[*] with you passing either an object interface (Java/C++) or free/member function (C++) into another function so that the callee can perform all the boilerplate stuff on your behalf whilst your method can get on with focusing on the real value. The original article, like Paul’s talk, looks at resource management but many see it as a more general purpose pattern than that. There is also some correlation with the classic Gang of Four pattern Template Method but without the use of inheritance. In my mind they are two sides of the same coin it just depends on whether the discussion focuses on the encapsulation of the surrounding code or the inner behaviour being invoked.

Going The Extra Mile

Here’s a C++ example from the start-up code I’ve been employing for many moons[+]:-

typedef int (*EntryPoint)(int argc, const char* argv[]);

int bootstrap(EntryPoint fn, int argc, const char* argv[])
{
try
{
    return fn(argc argv);
}
catch (const std::exception& e)
{
    // Log it...
}
. . .
catch (...)
{
    // Log it...
}

return EXIT_FAILURE;
}

int main(int argc, const char* argv[])
{
return bootstrap(applicationMain, argc, argv);
}

int applicationMain(int argc, const char* argv[])
{
// Parse command line, run app etc.
. . .
return EXIT_SUCCESS;
}

The physical process entry point (main) invokes the logical process entry point (applicationMain) via the start-up code (bootstrap). This leaves applicationMain() to do application specific stuff whilst leaving the grunge work of dealing with unhandled exceptions and other common low-level initialisation to bootstrap().

Naturally the advent of Boost means that you can adapt the code above to pass member functions or free functions with the same construct (Boost::Function). You could also do it with interfaces and the Command pattern but it’s even more verbose. Either way the extra level of indirection is still in your face.

A Brave New World

Now that I’m firmly entrenched in the world of C# I have the opportunity to write exactly the same code once more. And right back at the beginning that’s exactly what I did. Of course they’re called Delegates in C# and not Function Pointers, but the extra level of indirection was still there... Until I realised that I was finally working with a language that supported Closures. So now I can eschew the extra method and do it all inline:-

public static class ConsoleApplication
{
protected int bootstrap(Func<string[], int> main,
                          string[] args)
{
    try
    {
      return main(args);
    }
    catch
    . . .
    return ExitFailure;
}
public const int ExitSuccess = 0;
public const int ExitFailure = 1;
}

public static class Program : ConsoleApplication
{
public static int Main(string[] args)
{
    return boostrap(args =>
    {
      // Parse command line, run app etc.
      . . .
      return ExitSuccess;
    });
}
}

The use of a closure with Execute Around Method has made many of those ugly parts of the system (most notably the subsystem boundaries) far more readable and maintainable. What follows are the scenarios where I find it’s been put to constant use…

SQL execution

On the face of it executing a SQL query is quite trivial, but when trying to write a robust and scalable system you soon start to deal with many thorny issues such as transient errors and performance problems. These are the behaviours I tend to wrap around SQL queries:-

Instrument the query execution time
Drop and reconnect the connection on a cluster failover
Catch and retry when a timeout (or other recoverable transient error) occurs

Dealing with transient errors is particularly tricky because you can make matters worse if you keep retrying a query when the database is already under heavy load. It’s also important to ensure you’re dealing with the error that you think you are by catching specific exception types; SQL exceptions come through as one exception type and so you also need to inspect its properties carefully.

Remote Service Stub

Working back up the call stack we end up at the server-side stub to the remote service call. The caller of that method will probably be some framework code (e.g. WCF) and so it’s our last chance saloon to handle anything. Once again duties are in the realms of performance and error recovery:-

Instrument the remote call
Use a Big Outer Try Block to catch all exceptions and handle as appropriate

As a rule of thumb you shouldn’t let exceptions propagate across a “module boundary” because you don’t know what the callee will be able to do with it. In practice this is aimed at COM components which can be written in, say C++, while the application may be written in the “exception-less” C. It also applies to services too because they have to marshal anything you pass (or throw) back to them and if they can’t do that then you’ll get a significantly less helpful error to work with.

The Big Outer Try Block in the remote stub is also the point where you get to decide what “best effort” means. It may mean anything from “catch everything and do nothing” to “catch nothing and let everything through” but I would suggest you pick some middle ground. You want to avoid something trivial like an invalid argument causing the server to fail whilst at the same time not letting a systemic issue such as an access violation cause the server to enter a toxic state. If possible I like systemic issues to cause the server to take itself down ASAP to avoid causing more harm, but that can be worse than the disease[$].

Remote Service Proxy

Stepping back once more across the wire we come to the client-side invocation of a remote request. This is really just a generalised version of the SQL execution scenario but with one extra facility available to us:-

Instrument the request invocation time
Drop and reconnect to the service on a transport failure
Catch and retry when a timeout (or other recoverable transient error) occurs

One obvious question might be why you would instrument the service request from both the client proxy and service stub? Well, the difference tells you how much overhead there is in the transport layer. When talking using a low-level transport like a point-to-point socket there’s not much code in-between yours, but once you start using non-trivial middleware like DCOM, MSMQ and 3rd party grid computing products knowing where the bottleneck is becomes much harder. However if you can measure and monitor the request and response latencies automatically you’ll be in much better shape when the inevitable bottleneck appears.

Aspect Orientated Programming?

Not that it really matters, but is Execute Around Method just an implementation vehicle for AOP? I’ve never really got my head around where AOP truly fits into the picture, it always seems like you need external tooling to make it work and I’m not a big fan of “hidden code” as it makes it harder to comprehend what’s really going on without using the debugger. The wrappers listed above tend to have all the aspects woven together rather than as separate concerns so perhaps that’s a code smell that true AOP helps highlight. I’m not going to lose sleep over it but I am aware it’s a technique I don’t understand yet.

[*] And I don’t mean Phil Nash’s blog :-)

[+] You would have thought this problem would have been solved by now, but most runtimes seem to insist on letting an unhandled exception be treated as a successful invocation by the calling application!

[$] In a grid computing system you can easily create a “black hole” that sucks up and toasts your entire workload if you’re not careful. This is created by an engine process that requests work and then fails it very quickly either because itself or one of its dependent services has failed thereby causing it to request more work much quicker than it’s siblings.

Unit Tests Obscure Dead Code

2011-06-29T22:54:00.001+01:00

I’ve been having a bit of a clean-up in our codebase as I know of various large bits of functionality that for one reason or another never made it into production and now they are just becoming a distraction. Even though we are using the thoroughly useful ReSharper, which can point out some obvious dead code, it doesn’t perform enough analysis to see past code that is only referenced by unit tests.

Oh the irony… I spend all that time banging on about writing unit tests and now I’m cursing their existence because they stop me from seeing dead code! But my team is using NUnit, not some obscure framework, and ReSharper has native support for NUnit, so it’s not out of the realms of possibility that JetBrains could “join the dots” and provide a facility to highlight dead code that only has unit tests for it. If you’re writing a library I presume you’d want this feature off by default :-) but if you’re writing an application you’d probably like it on. Of course if you have a really large codebase and you’ve split it to ensure you only build what you need then I guess it becomes less accurate and false positives are not exactly a confidence builder. I’d still prefer to have the choice of enabling a “be aggressive” flag when I’m in cleanup mode.

Clearly there are other analysis tools, like NCover, that could help determine dead code, but I’m not talking about squeezing every last line of redundant code out (which would probably involve actually running it and then deciding whether you covered enough scenarios), but just pointing out the really obvious stuff like unused classes, methods, interface implementations etc.

Anyway, until such time as JetBrains (or a suitable alternative) comes entirely to my rescue I’m using the following semi-automatic ReSharper assisted approach:-

Load the solution and remove all unit test projects from it
Use ReSharper to show up dead code
Delete dead code
Goto 2 to find what new dead code ReSharper has unearthed[+]
Reload the solution with unit test projects
Delete redundant unit tests
Fix any false positives[*]

Of course having said all that someone will go and point out that I should RTFM or show me how FxCop already does this… Please do!

[+] ReSharper uses the term “heuristically determined unused method”, and yet it still doesn’t point out a lot of dead code - you often have to go and delete the unused callers before it will mark something as dead. And these aren’t methods with deep call graphs, maybe only 1 or two deep. Perhaps it’s more pessimistic when interfaces are involved?

[*] Yes we have other dependent code (such as various PowerShell scripts) that means you can’t just carelessly swing the axe about but it’s pretty obvious what would be called externally.

User Defined Types in SQL Server

2011-06-28T22:33:00.001+01:00

In my recent post The Public Interface of a Database I talked about using User Defined Types and how I believe they help make the interface more maintainable. Now being in production and having real data stored in our tables means that changing them is going to be more difficult. The other day I needed to do just that and I found it’s not quite as simple as I had hoped…

Domain Types (Theory)

Although I’ve never studied database theory in any formal way I can see how User Defined Types are probably meant to be used - to allow you to express the commonality between data of the same nature and also to attach constraints to deal with restrictions that the underlying storage type may not possess.

For example, if you were storing a percentage (as an integer) you could use any of the integer types - tinyint, smallint, int etc, however it makes sense to use the smallest amount of storage required, i.e. tinyint (1 byte). But this type has a range of 0 - 255 which is wider than a percentage (0 - 100) so you need to bind a rule to say the value is actually limited to just (0 <= value <= 100).

Domain Types (Practice)

The problem is that the rule (at least under SQL Server) only applies to a column in a table; it is effectively a Check Constraint[+]. This means that declaring them in your stored procedure parameter list and using them within your SQL code only has a partial effect - you get the effects from abstracting the underlying storage type, but not the additional validation rules.

To make matters worse when you try to change the UDT and go searching for the ALTER TYPE statement you’ll find it doesn’t exist[*]. The only way you can change a UDT is to drop it and then create it again. But, to drop it you need to have nothing referencing it - no tables, functions or procedures.

Putting aside the difficulties in changing them for the moment let’s see what benefits they can bestow…

Type Aliases

Even though they do not enforce their constraints when as variables or arguments, they still have a marked effect on the maintainability of your code. Using the kind of example I cited in my previous post, which do you think is more expressive?

declare @name varchar(100)
vs
declare @name dbo.CustomerName_t

With good variable naming it is arguable how much the type adds in this example. If I changed the variable name to “@customerName” then they would both be just as explicit; but at least in the latter case you have the choice to let the type name enhance the readability.

There is also the temptation, especially with varchar columns, to just pick a size that is “big enough”. This means you might see varchar(100) in some places, varchar(128) in others and perhaps a very specific varchar(33) elsewhere. User defined types as simple aliases take all that guesswork away so that you don’t have to think nearly so hard. Other common sources of discrepancy are the various numbers used in financial scenarios that have a habit of being stored with different scales and precision, e.g. float(n), decimal(s, p) depending on the paranoia level of the developer.

Abstracting Identities

Entities, such as customers and products, are commonly referred to by some integer based surrogate key (perhaps generated by an IDENTITY column), but they may also be based on a natural key if, say, a short code exists. Once again, using a type alias means that you don’t have to think so hard; the example below could be based on an underlying integer or varchar type:-

declare @productId dbo.ProductId_t

This is a big win because unlike some cases where you could argue that you need to know what the underlying storage type is (perhaps you’re going to perform some manipulation of its contents) an entity ID is (in my experience) nearly always a unique & opaque value - it’s contents are unimportant. I’ve come to find this construct very unsatisfactory (in any language):-

declare @id int

The Integer type supports so many operations and allows for unnecessary promotions that it’s an accident waiting to happen[#] when used for the identity of an entity. Sadly, using UDT’s will not disable the undesirable arithmetic operations, but it should help avoid any unexpected type mismatches by virtue of not being able to get it wrong in the first place.

Mechanisms for Change

Let’s come back to that gnarly issue about how to change them later because that’s what prompted this post in the first place…

As mentioned earlier there is no magical ALTER TYPE statement and so that means you have two real choices:-

Drop every referenced object, drop the old type and then recreate everything
Introduce a new type, migrate everything across to that and then drop the old type

The second option may sound simpler, and it is, slightly; but only because you can recreate objects in an order that is less susceptible to dependency problems. But functions and procedures are the easy part anyway, what’s hard is altering a tables’ schema, especially big ones!

Ultimately your choice for updating a UDT that is used in a table depends very much on the change you need to make to the UDT and whether it requires data in the existing table to be touched. For example, if you widen a column from varchar(10) to varchar(20) SQL Server only has to change the table metadata to say the maximum length has increased because the actual length of each value does not change. Shrinking the column size or changing an integral type will likely cause data to be touched, or even just examined to ensure it still conforms. According to one of the Inside SQL Server books you can tell a priori whether data will need to be touched just by looking at the estimated execution plan.

So far my needs to change a UDT have only revolved around varchar columns and so I’ve been lucky that the table change has been the simple one. The changes to the functions and procedures were also pretty easy because we have the ability to rebuild the database from scratch (our Continuous Integration process does this each build) so I could ensure I caught and checked every place where the type was used. It’s a somewhat laborious process, but with the right development process around you it can be pretty easy.

Updating Tables

As I understand it the most efficient way to update an existing table schema (where you’re not just appending a new column) is to rename it, create the new one and then copy the data back over. When I first came to change my UDT I knew that I could either try and fix-up the existing one or create a new “version” of it. When I considered what actually needed to be done to fix the type and how big the table was I buckled and created a new type, called “MyType2_t”. This was because it felt like there were far less ordering issues and I didn’t have to generate scripts to drop things first. So, instead of this lengthy tirade:-

Drop dependent functions and procedures
Create copies of tables without the UDT to be modified
Copy data to backup tables
Recreate the UDT with the new definition
Create new tables with the new UDT
Copy the data back again
Recreate the functions and procedures

…I could just use ALTER TABLE and switch the types, taking whatever hit was necessary to adjust the data in the tables (which was none in this case):-

Create the new version of the UDT
Alter the tables to use the new version of the UDT
Recreate the functions and procedures[$]
Drop the old version of the UDT

Yes, the deployment was easier, but we’re now left with a wart - a type with a version suffix (e.g. MyType2_t). After having gone through all this it dawned on me soon after that if you could use ALTER TABLE to switch types then surely you can just do this instead:-

Drop dependent functions and procedures
Alter the table to use a non-UDT based equivalent type
Recreate the UDT with the new definition
Alter the table to use the new UDT
Recreate the functions and procedures

If SQL Server is smart enough to just apply a metadata change when growing a varchar(n) then it should be able to just replace the metadata when switching from a UDT that is effectively a varchar(n) to a non-UDT alias of a varchar(n). This would remove the dependency on the UDT without any upheaval at all. Sadly it’s an experiment I cannot run on this netbook, but I intend to.

Unnecessary Context Switching

Having worked for some time on a codebase that uses UDTs virtually everywhere, now, when I do see an explicit type I begin to question whether it’s correct or not and I find that very distracting. I begin to question whether any thought was put into it or whether the author was just being lazy. Perhaps it’s a new type and they just didn’t have time to create a UDT for it? Or maybe they did but missed fixing up this case? Any way you look at it I’m now being diverted from my real goal - but that’s just the age old argument about being consistent I suppose.

[+] According to the documentation for CREATE RULE that these are now deprecated anyway in favour of using Check Constraints. Was that because they just weren’t as useful as expected?

[*] There was a “petition” on Microsoft Connect about the lack of an ALTER TYPE statement which I discovered whilst trawling StackOverflow for alternate solutions.

[#] This is one area where C++ enumerations are a boon because they disallow the arithmetic operations and also give the value a type-safe holder. This jolly useful use of enumerations was pointed out to me by fellow ACCU member Jonathan Wakely.

[$] Each function and procedure script has the common “if (object_id(Xxx) is not null) drop Xxx” prologue.

Out vs Ref For TryXxx Style Methods

2011-05-18T18:09:00.001+01:00

None of the popular programming languages that I know of allow you to overload a method based on error semantics. A common pattern to workaround this is to provide two overloads – one named normally that throws an exception on failure and another no-throw version who’s name is prefixed with “Try” and returns a bool instead (with any additional return values handled by output parameters). A classic example is the parsing functions on the C'# DateTime type:-

DateTime Parse(string value);
bool TryParse(string value, out DateTime result);

In principle it’s easy enough to move from the exception throwing form:-

{
DateTime output = DateTime.Parse(input);
. . .
}

…to the alternate non-throwing form when you decide you need the different error handling semantics:-

{
DateTime output;

if (DateTime.TryParse(input, output))
{
. . .
}
}

But what about when you don’t care if the method succeeded or not? On a number of occasions I have used a TryXxx style method and have not cared about the boolean return code, I just want it to use my default value if it fails:-

{
DateTime output = DateTime.Now; // default

DateTime.TryParse(input, output);
. . .
}

Unfortunately this won’t have the desired effect (it actually won’t compile as is, but hold on) because your default value gets clobbered. Consider the following method on my IConfiguration interface that attempts to retrieve a configuration setting, if it exists:-

bool TryGetSetting(string key, out string value);

If I use the ‘out’ keyword as part of the interface I am forced to provide a value for all code paths. Consequently the implementation will probably look like this:-

bool TryGetSetting(string key, out string value)
{
// Attempt to retrieve the setting
if (. . .)
{
    value = . . .;
    return true;
}
else
{
    // Must initialise the output value on all paths
    value = null;
    return false;
}
}

The only general default value you can provide for a reference is null. Yes, for a string (or any other class that defines it) you could use the ‘Empty’ value, but that still clobbers any input from the caller. And so you force the caller to acknowledge the failure and write the slightly more verbose:-

{
string value;

if (!config.TryGetSetting("setting", out value))
value = "my default value";
. . .
}

The alternative is to use ‘ref’ instead, which allows the caller to provide a default value and you no longer have to clobber it in your implementation:-

bool TryGetSetting(string key, ref string value)
{
// Attempt to retrieve the value
if (. . .)
{
    value = . . .;
    return true;
}
else
{
    // Leave caller’s value untouched
    return false;
}
}

Finally, as a caller I can now just write this:-

{
string value = "my default value";

config.TryGetSetting("setting", ref value)
. . .
}

So, I wonder why the C# designers picked ‘out’ over ‘ref’ in the first place? Perhaps they felt it was safer. But is it that much safer? If you use ‘out’ and don’t check the return code you’ll probably end up either accessing a null reference or continuing with the equivalent of 0 for a value type, i.e. whatever default<type>) returns. OK, this is far superior to the C++ world where an uninitialized variable could be anything[*]. If you use ‘ref’ then you let the caller choose the uninitialised value, which, if they are following best practice will result in the same effect because they won’t be reusing existing variables for other purposes.

There is of course a semantic difference between ‘out’ and ‘ref’, but I think what I’m suggesting blurs the line between them. If you look at ‘out’ and ‘ref’ through COM’s eyes and put a network in the middle then it’s all about whether you need to marshal the value to the callee and this is not the behaviour we want. The callee doesn’t need the value and we certainly don’t want to allow it to be able to modify it, so ‘ref’ is out (if you’ll pardon the pun). What we want ‘out’ to mean in this scenario is “don’t clobber the existing variable if no output value was provided, and don’t bother marshalling the value into the callee either”.

It’s great that C# points out where you have attempted to use an uninitialised variable, but sadly I think it’s that same mechanism that also gets in the way sometimes.

[*] I once got bitten by an uninitialized ‘bool’ during my C++ days. Somewhat ironically it was exactly because we were using all the debug settings during development and they very cleverly initialise stack variables and heap memory to a non-zero value that it went unnoticed. This is because the “uninitialized” value was always reinterpreted for a ‘bool’ as ‘true’. There is a reason why you always write a failing unit test first…

Testing Drives the Need for Flexible Configuration

2011-05-16T18:21:00.001+01:00

If you look at our system’s production configuration settings you would be fooled into thinking that we only need a simple configuration mechanism that supports a single configuration file. In production it’s always easier because things have settled down, but during testing is when the flexibility of your configuration mechanism really comes into play.

I work on distributed systems which naturally have quite a few moving parts and one of the biggest hurdles to development and maintenance in the past has been because the various components cannot be independently configured so that you can cherry-pick which services you run locally and which you draw from your integration/system test environment. Local (as in on your desktop) integration testing puts the biggest strain on your configuration mechanism as you probably can only afford to run a few of the services that you might need unless your company also provides those big iron boxes for developer workstations[#].

In the past I’ve found the need to override component settings using a variety of criteria and the following list is definitely not exhaustive, but gives the most common reasons I have encountered:-

Per-Environment

The most obvious candidate is environmental as there is usually a need to have multiple copies of the system running for different reasons. I would hazard a guess that most teams generally have separate DEV, TEST & PROD environments to cover each aspect of the classic software lifecycle. For small systems, or systems with top-notch test coverage the DEV & TEST environments may serve the same purpose. Conversely I have worked on a team that had 7 DEV environments (one per development stream), a couple of TEST environments and a number of other special environments used for regulatory purposes, all in addition to the single production one.

What often distinguishes these environments is the instances of the external services that you can use. Generally speaking all production environments are ring-fenced so that you only have PROD talking to PROD to ensure isolation. In some cases you may be lucky enough to have UAT talking to PROD, perhaps to support parallel running. But DEV environments are often in a sorry state and highly untrusted so are ring-fenced for the same reason as PROD, but this time for the stability of everyone else’s systems.

Where possible I like the non-production environments to be a true mirror of the production one, with the minimum changes required to work around environmental differences. Ideally we’d have infinite hardware so that we could deploy every continuous build to multiple environments configured for different purposes, such as stress testing, fault injection, DR failover etc. But we don’t. So we have to settle for continuous deployment to DEV to run through some basic scenarios, followed by promotion to UAT to provide some stability testing. What this means is that our inputs are often the same as for production, but naturally our outputs have to be different. But you don’t want to have to configure each output folder separately, so you need some variable-based mechanism to keep it manageable.

The Disaster Recovery (DR) environment is an interesting special case because it should look and smell just like production. A common technique for minimising configuration changes during a failover is to use DNS Common Names (CNAMEs) for the important servers, but that isn’t always foolproof. Kerberos delegation in combination with CNAMEs is a horribly complicated affair. And that’s when you have no control over the network infrastructure.

Per-Machine

Next up is machine specific settings. Even in a homogenous Windows environment you often have a mix of 64-bit and 32-bit hardware, slightly different hard disk partitioning, or different performance for different services. Big corporations love their “standard builds” which helps minimises the impact but even those change over time as the hardware and OS changes – just look at where user data has been stored in Windows over the last few releases. The ever changing security landscape also means that best practices change and these will, on occasion, have a knock-on effect on your system set up.

By far the biggest use for per-machine overrides though is during development, i.e. when running on the developers workstation. While unit testing makes a significant contribution to the overall testing process you still need the ability to easily cobble together a local sandbox in which you can do some integration testing. I believe the DEV environment cannot be a free-for-all and should be treated with almost the same respect as production because if the DEV environment is stable (and running the latest code) you can often reduce the setup time for your integration testing sandbox by drawing on the DEV services instead of running them locally.

Per-Process-Type

Virtually all processes in the system will probably share the same basic configuration, but certain processes will have specific tasks to do and so they may need to be reconfigured to work around transient problems. One of the reasons for using lots of processes (that share logic via libraries) is exactly to make configuration easier because you can use the process name as a “configuration variable”.

The command line is probably the default mechanism most people think of when you want to control the behaviour of a process, but I find it’s useful to distinguish between task specific parameters, which you’ll likely always be providing, and background parameters that remain largely static. This means that when you use the “--help” switch you are not inundated with pages of options. For example a process that always needs an input file will take that on the command line, as it might an optional output folder; but the database that provides all the background data will be defaulted using say an .ini file.

Per-User

My final category is down to the user (or service account) under which the process runs. I’m not talking about client-side behaviour which could well be entirely dynamic, but server-side where you often run all your services under one or more special accounts. There is often an element of crossover here with the environment as there may be separate DEV, TEST and PROD service accounts to help with isolation. Support is another scenario where the user account can come into play as I may want to enable/disable certain features to help avoid tainting the environment I’m inspecting, such as using a different logging configuration.

Getting permissions granted is one of those tasks that often gets forgotten until the last minute (unless DEV is treated liked PROD). Before you know it you switch from DEV (where everyone has way too many rights) to UAT and suddenly find things don’t work. A number of times in the past I’ve worked on systems where a developer’s account has been temporarily used to run a process in DEV or UAT to keep things moving whilst the underlying change requests bounce around the organisation. Naturally security is taken pretty seriously and so permissions changes always seem to need three times as many signatures as other requests.

Hierarchical Configuration

Although most configuration differences I’ve encountered tend to fall into one specific category per setting, there are some occasions where I’ve had cause to need to override the same setting based on two categories, say, environment and machine (or user and process). However because the hardware and software is itself partitioned (e.g. environment/user) it’s usually been the same as overriding on just the latter (e.g. machine/process).

What this has all naturally lead to is a hierarchical configuration mechanism, something like what .Net provides, but where <machine> does not mean all software on that machine, just my system. It may also take in multiple configuration providers, such as a database, .ini files, registry[*] etc. My current system only uses .config style[$] files at present and on start-up each process will go looking for them in the assembly folder in the following order:-

System.Global.config
System.<environment>.config
System.<machine>.config
System.<process>.config
System.<user>.config

Yes, this means that every process will hit the file-system looking for up to 5 files, but in the grand scheme of things the hit is minimal. In the past I have also allowed config settings and the config filename to be overridden on the command line by using a global command line handler that processes the common settings. This has been invaluable when you want to run the same process side-by-side during support or debugging and you need slightly different configurations, such as forcing them to write to different output folders.

Use Sensible Defaults

It might appear from this post that I’m configuration mad. On the contrary, I like the ability to override settings when it’s appropriate, but I don’t want to be forced to provide settings that have an obvious default. I don’t like seeing masses of configuration entries just because someone may need to use it one day – that’s what source code and documentation is for.

I once worked on a system where all configuration settings were explicit. This was intentional according to the lead developer because you then knew what settings were being used without having to rummage around source code or find some (probably out-of-date) documentation. I understand this desire but it made testing so much harder as there was a single massive configuration object to bootstrap before any testable code ran. I shouldn’t need to provide a valid setting for some obscure business rule when I’m trying to test changes to the messaging layer – it just obscures the test.

Configuration Formats

I’m a big fan of simple string key/value pairs for the configuration format – the old fashioned Windows .ini file still does it for me. Yes XML may be more flexible but it’s also far more verbose. Also, once you get into hierarchical configurations (such as .Net .config files), its behaviour becomes unintuitive as you have to question whether sections are merged at the section level, or individual entries within each section. These little things just make integration/systems testing harder.

I mentioned configuration variables earlier and they make a big difference during testing. You could specify, say, all your input folders individually, but when they are related that’s a real pain when it comes to environmental changes, e.g.

[Feeds]
SystemX=\\Server\PROD\Imports\SystemX
SystemY=\\Server\PROD\Imports\SystemY

One option is to generate your configuration from some sort of template, but I find that a little too invasive. It’s pretty easy to emulate the environment variable syntax so you only have one setting to change:-

[Variables]
SharedData=\\Server\PROD
FeedsRoot=%SharedData%\Imports

[Feeds]
SystemX=%FeedsRoot%\SystemX
SystemY=%FeedsRoot%\SystemY

You can even chain onto the environment variables collection so that you can use %TEMP% and %ProgramFiles% when necessary.

[#] Quite how anyone was ever expected to develop solid, reliable, multi-threaded services with a machine with only a single or dual hyper-threaded CPU is quite beyond me. I remember 10 years ago when we had 1 single dual-CPU box in the corner of the room which was used “for multi-threaded testing”. Things are better now, but sadly not by that much.

[*] Environment variables are great for controlling local processes but are unsuitable when it comes to Windows services because a machine reboot is required when they change. This is because the environment variables that a service process receives is inherited from the SCM (Service Control Manager), so you’d need to restart the SCM as well as the service (it doesn’t notice changes like the Explorer shell does). So, in this scenario I would favour using the Registry instead as you can get away with just bouncing the service.

[$] Rather than spend time up-front learning all about the .Net ConfigurationManager I just created a simple mechanism that happened to use files with a .config extension and that also happened to use the same XML format as for the <appSettings> section. The intention was always to switch over to the real .Net ConfigurationManager, but we haven’t needed to yet – even our common client-side WCF settings use our hierarchical mechanism.

The Public Interface of a Database

2011-05-11T00:49:00.001+01:00

The part of my recent ACCU London talk on database unit testing that generated the most chatter was the notion that there can be a formal public interface to it. Clearly we’re not talking about desktop databases such as Access, but the big iron products like SQL Server and Oracle. It is also in the Enterprise arena that this distinction is most sorely needed because it is too easy to bypass any Data Access Layer and directly hit the tables with tools like SQLCMD and BCP. I’m not suggesting that this is ever done with any malicious intent, on the contrary, it may well be done as a workaround or Tactical Fix[*] until a proper solution can be developed.

In any Enterprise there are often a multitude of in-house and external systems all connected and sharing data. Different consumers will transform that data into whatever form they need and successful systems can in turn find themselves becoming the publishers of data they only intended to consume because they can provide it in a more digestible form. The tricky part is putting some speed bumps in place to ensure that the development team can see when they have violated the design by opening up the internals to the outside world.

Building Abstractions

So what does this Public Interface look like? If you look at tables and stored procedures in a similar light to structs and functions in C, or classes and methods in C++/C# you naturally look for a way to stop direct access to any data members and this means stopping the caller performing a SELECT, INSERT, UPDATE or DELETE directly on the table. You also need to ensure that any implementation details remain private and do not leak out by clearly partitioning your code so that the clients can’t [accidentally] exploit them.

I’ll admit the analogy is far from perfect because there are significant differences between a table of data rows and a single instance of a class, but the point is to try and imagine what it would do for you if you could, and how you might be able to achieve some of the more desirable effects without sacrificing others such as productivity and/or performance. As you tighten the interface you will gain more flexibility in how you can implement a feature and more importantly open yourself up to the world of database refactoring (assuming you have a good unit test suite behind you) and even consider using TDD.

Stored Procedures

The most obvious conclusion to the encapsulation of logic is the blanket use of Stored Procedures and User Defined Functions so that all access is done using parameterised functions. These two constructs provide the most flexibility in the way that you design your abstractions because they allow for easy composition which becomes more essential as you start to acquire more and more logic. They also provide a good backdoor for when the unanticipated performance problems start to appear as you can often remediate a SQL query without disturbing the client.

Of course they are not a panacea, but they do provide a good fit for most circumstances and are a natural seam for writing unit tests and using TDD. The unit test aspect has probably been the biggest win for us because it has allowed us to develop and test the behaviour in isolation and then just plug it in knowing it works. It has also allowed us to refactor our data model more easily because we have the confidence that we are in control of it.

Views

Stored procedures are great when you have a very well defined piece of functionality to encapsulate, such as updating a record that has other side-effects that are inconsequential to the client, but they are more of a burden when it comes to querying data. If you try and encapsulate every read query using stored procedures you can easily end up with a sea of procedures all with subtly different names, or a single behemoth procedure that takes lots of nullable parameters and has a ‘where’ clause that no one wants to touch. Views solve this problem by giving the power back to client, but only insofar as to let them control the query predicates – the list of columns (and their names) should be carefully decided up front to avoid just exposing the base tables indirectly, as then you’re back to where you started.

User Defined Types

User defined types seem to have a bad name for themselves but I couldn’t find much literature on what the big issues really are[#][$]. I find them to be a very good way to create aliases for the primitive types in the same way as you would use ‘typedef’ in C/C++. A classic source of inconsistencies that I’ve seen in SQL code is where you see a varchar(100) used in some places and a varchar(25) in others as the developer just knows it has to be “big enough” and so picks an arbitrary size; it’s then left to the maintainer to puzzle the reason for the mismatch. UDTs allow your interface to be consistent in it’s use of types which makes comprehension easier and they can also be used by the client to ensure they pass compatible types across the boundary.

My current system started out on SQL Server 2005 which didn’t have the native Date and Time types so we created aliases for them. We managed to move to SQL Server 2008 before release and so could simply change the aliases. The system also has a number of different inputs and outputs that are not simple integers and so it’s important that we use the same scale and precision for the same type of value.

Schemas

Another tool in the box is Schemas. These are a way to partition your code, much like you would with namespaces (or packages), although they don’t provide the hierarchic behaviour that you get with, say, C# or C++. A very common database model is to separate the input (or staging) tables from the final ones whilst the data is validated and cleansed. One way to apply that might be to use the default ‘dbo’ schema for the production tables and use a separate ‘staging’ schema for the input tables. You can then overload table and object names without having to resort to ugly prefixes or suffixes.

You can of course go much further than this if you start out by treating the default ‘dbo’ schema as synonymous with ‘internal’ or ‘private’; then all objects in that default schema are considered as purely implementation details and not directly exposed to the client. You then create separate schemas for the major subsystems, such as a ‘backend’ schema for the server-side code and a ‘ui’ schema for the client. These will allow you to create multiple interfaces tailored for each subsystem that, if you use traditional functional composition, avoids duplicate code by just internally layering stored procedures. This natural layering also quickly highlights design problems, such as when you see an object in the ‘dbo’ schema referencing one in the ‘staging’ schema. There is often a natural flow, such as from staging to production, and schemas help you remain consistent about how you achieve that flow, i.e. push from staging to production or pull from staging by production.

Permissions

Ultimately if you want to deny access to something, just don’t grant it in the first place :-). Databases offer a fine-grained permissions system that allows you to grant access to specific objects such as individual stored procedures, functions and views. This means that your core definition of what constitutes the public interface is “anything you’ve been granted access to”. However it is not always obvious what may or may not be permissioned, and to what role, so the other mechanisms, such as schemas, can be used to provide a more visible means of scope.

Part of the permissions structure will be based around the roles the various clients play, such as server or UI, and so you will probably see a strong correlation between a role and the set of objects you then choose to expose in your various interfaces rather than just blindly exposing the same set for everyone. For example it is common to have a “Service Account” under which the back-end runs, this could be mapped to a specific Service Role which is then only granted permission to those objects it genuinely invokes which all exist in the specific Service schema. In contrast a Support Role may be granted access to all objects that only performs reads but have a bunch of special objects in a separate Support schema that can do some serious damage but are needed occasionally to fix certain problems. Different ‘public’ interfaces for different roles.

Oranges Are Not The Only Fruit

Jeff Atwood wrote a post for his Coding Horror blog way back in 2004 titled “Who Needs Stored Procedures, Anyways?” that rallies against the blanket use of stored procedures. One of his points is aimed at dispelling the performance myth of procedures, but as I wrote last year “Stored Procedures Are About More Than Just Performance”, but he’s right that embedded parameterised queries can perform equally well. Hopefully my comment about the use of views shows that I also feel the blanket use of procedures is not necessarily the best choice.

There are a lot of comments to his post, many of which promote the silver bullet that is an O/RM, such as Hibernate or Entity Framework. And that is why I was careful to describe the context in my opening paragraphs. If you’re working in a closed shop on an isolated system you can probably get away with a single Data Access Layer through which everything is channelled, but that is often not possible in a large corporation.

I would say that Jeff’s main point is that if you want to expose “A Service” then do just that, say, as a Web Service, so that you truly abstract away the persistent store. I agree with the sentiment, but corporate systems just don’t change technology stacks on a whim, they seem to try and drain every last ounce out of them. Hence I would question the real value in building (and more importantly maintaining) that proverbial “extra level of indirection” up front. Clearly one size does not fit all, but sadly many commenter's appear to fail to appreciate that. Heterogeneous systems are also wonderful in theory but when a company purposefully restricts itself to a small set of vendors and products what does it really buy you?

For me personally the comments that resonated most closely were the ones advocating the use of independent unit testing for the database code. I believe the ideal of using TDD to develop the database layer adds an interesting modern twist that should at least cause Jeff some pause for thought.

[*] The term tactical is almost always synonymous for quick-and-dirty, whereas strategic is seen as the right way to do it. However the number of tactical fixes and even entire tactical systems that have lived longer that many strategic solutions is huge. It just goes to show that growing systems organically is often a better way to approach the problem.

[#] The only real issue I’ve come across is that you can’t define a temporary table using UDTs on SQL Server – you have to add them to the model database or something which all sounds a bit ugly. I’ve not tried it out but I recently read that you can use “SELECT TOP(0) X,Y,Z INTO #Tmp FROM Table” or “SELECT X,Y,Z INTO #Tmp FROM Table WHERE 1 = 0” as a fast way of creating a temporary table based on a result set because the optimiser should know it doesn’t need to do any I/O. Describing your temporary table this way makes perfect sense as it avoids the need to specify any types or column names explicitly; so long as it incurs a minimal performance cost that is.

[$] Another more minor grumble seems to be to do with binding rules, in that they are effectively copied at table creation time instead of referencing the UDT master definition. You also can’t alias the XML type on SQL Server which would have been useful to us as we had performance concerns about using the XML type on a particular column. In the end we got close to what we wanted by using an alias to varchar(8000) with a rule that attempts a conversion of the value to XML – we could then just drop the rule if we noticed a performance problem in testing.

A StyleCop/FxCop For Databases

2011-05-04T21:33:00.001+01:00

My team all use the most excellent ReSharper during day-to-day C# coding as it performs some great on-the-fly static code analysis. We also occasionally run FxCop to provide some further insights into our C# code. However we didn’t really have anything for the database side of things, so a colleague of mine wrote one and called it ‘DbCop’. OK, so it’s clearly not in the same league as ReSharper but it has got me wondering if there are any commercial products out there that fill this space? I’ve not specifically hunted for one, but you still see things like this mentioned (if they’re worth anything) on sites like StackOverflow and personal blogs. I also dropped some not-so-subtle hints at the Red Gate stand at this years ACCU Conference but they didn’t give anything away if they do have anything like this in the pipeline...

Our tool is really nothing fancy, just some simple SQL scripts that spelunk the system tables and look at the schema metadata for some common mistakes and coding convention violations. It runs at the end of our database Continuous Integration build and generates a report; but it doesn’t fail the build if it finds a problem because we currently have no exclusion mechanism in place (besides manually hard-coding one into the script). We also have a separate schema/namespace called “build” for the DbCop objects so that they are not applied to integration test/system test/production databases by accident.

So far it only checks the following:-

No reserved words have been used for the names of our objects
The names of tables, columns and parameters adhere to our coding conventions
Table columns are defined using our UDT’s, not primitive types
Each table has a primary key
Each table has a clustered index

Clearly rules like the last two are made to be broken, but as a general rule they are pretty sound. Personally I tend to avoid a Primary Key rule violation by writing a unit test in the first place that ensures duplicates are disallowed when appropriate. It would be great to include spell checking into the process (just like FxCop) because I’m forever misspelling identifiers and with a database it’s much harder to change table and column names after they’ve gone into production and contain millions of rows.

PowerShell, Throwing Exceptions & Exit Codes

2011-05-03T21:03:00.001+01:00

[I raised a cut down version of this as a question via a comment to a related post on the Windows PowerShell blog back in March. I’ve not seen a response yet and doubt I ever will as it’s an old post and so I very much doubt anyone is monitoring it.]

I’ve got a bit of a love/hate relationship with PowerShell at the moment. Naturally whilst learning any new language the books steer you nicely towards the things that work, but as you start to “do your own thing” you step outside that comfort zone and the warts and inconsistencies start to appear. I should point out that this particular affliction affects more than just PowerShell[*] but it’s worse because it appears inconsistent in its behaviour and so appears to work – sometimes.

The Process Exit Code

Every process can return an exit code to signal to its caller something about the outcome of the task it was asked to perform. I don’t believe there is a formal definition anywhere about what constitutes “success” and “failure”[#] but the established convention is that zero means success and non-zero means unsuccessful. Of course what “unsuccessful” then means opens a whole new can of worms but if you’re writing a Windows batch file then the following construct is probably embedded in your head:-

<execute some process>
IF ERRORLEVEL 1 (
ECHO ERROR: <some error message>
EXIT /B 1
)

The ERRORLEVEL test is read as “if the last process exit code was greater than or equal to 1”. This assumes that the exit code will always be positive which is pretty much the norm, but it doesn’t have to be. So what does this have to do with PowerShell then?

The PowerShell EXIT & THROW Keywords

If you transliterate a Windows batch file into PowerShell you will probably end up writing code like this:-

<do something>
if ( <not some condition> )
{
write-output “something bad occurred”
exit 1
}

This is because “echo” is aliased as “write-output” and PowerShell still has an equivalent “exit” keyword to terminate the script[+]. So far so good, but PowerShell can do so much more and it also supports a “throw” keyword to allow you to use a more modern style of exception handling in your code that is especially useful when combined with functions. So you might expect that you could avoid the hard-coded exit code and use something like this instead (which is what I wanted to do):-

<do something>
if ( <not some condition> )
{
throw “something bad occurred”
}

Yes, I know that I’m using exceptions for error handling and that might be considered bad form, but in the places I was using this idiom the errors were truly non-recoverable and so the effect would be the same – the script should terminate and the caller be signalled that a fatal error occurred.

Unhandled Exceptions

The way that I test all my scripts and processes to ensure that they exit with a well formed result code is by using the following Windows batch file, which I name “RUN.CMD”:-

@ECHO OFF
CALL %*
ECHO.
ECHO ExitCode=[%ERRORLEVEL%]

So I ran my PowerShell script to test the error handling and I noticed that it always returned 0, irrespective of whether it terminated with a throw or not. The output showed the details of the exception as I expected, but with the exit code being 0 my calling parent batch scripts and job scheduler would not be able to detect a failure (without some really ugly scraping of the output streams). So I tried a few experiments with the throw construct. First an ‘inline’ command:-

C:\Temp>run PowerShell -command "throw 'my error'"
my error
At line:1 char:6
+ throw <<<< 'my error'
+ CategoryInfo : OperationStopped: (my error:String) [], RuntimeException
+ FullyQualifiedErrorId : my error

ExitCode=[1]

Great, an unhandled exception in a inline script causes PowerShell.exe to return a non-zero exit code. What about if I put the same one-liner in a .ps1 script file and execute it:-

C:\Temp>run PowerShell -file test.ps1
my error
At C:\Temp\test.ps1:1 char:6
+ throw <<<< 'my error'
+ CategoryInfo : OperationStopped: (my error:String) [], RuntimeException
+ FullyQualifiedErrorId : my error

ExitCode=[0]

Not so good. Yes we get the error message, but PowerShell.exe exited with a code that signals success. I have always specified the –File switch when running a script to avoid the need to do the whole .\ relative path thing. So what about running the same script file as a Command, surely that would be the same, wouldn’t it?

C:\Temp>run PowerShell -command .\test.ps1
my error
At C:\Temp\test.ps1:1 char:6
+ throw <<<< 'my error'
+ CategoryInfo : OperationStopped: (my error:String) [], RuntimeException
+ FullyQualifiedErrorId : my error

ExitCode=[1]

Meh? So, depending on whether I use the “–File” or “–Command” switch to execute the .ps1 script I get different behaviour. Is this a PowerShell bug or is there something fundamental about the execution model the differs between –File and –Command that I’ve yet to understand? Either way I wouldn’t want to rely on someone not being helpful and “fixing” the command line by switching the use of –Command to –File, especially as it affects error handling and we all know how hard people test their changes to verify the error handling still works as designed…

Trap To The Rescue

I have found a somewhat invasive workaround that at least ensures a sensible exit code at the expense of less pretty output. It relies on adding a Trap handler at the top of the script to catch all errors, output them and then manually exit the script:-

trap
{
write-output $_
exit 1
}

throw 'my error'

Here’s the output from it when run with the previously unhelpful “-File” switch:-

C:\Temp>run PowerShell -file test.ps1
my error
At C:\Temp\test.ps1:7 char:6
+ throw <<<< 'my error'
+ CategoryInfo : OperationStopped: (my error:String) [], RuntimeException
+ FullyQualifiedErrorId : my error

ExitCode=[1]

Now that’s better. The obvious change would be to use Write-Error as the output cmdlet but that has a side-effect when using redirection[+]. There is probably some way I can have my cake and eat it but my PowerShell skills are less than stellar at the moment and my Googling has turned up nothing positive so far either.

[*] Matthew Wilson wrote a Quality Matters column for one of the ACCU journals where he showed that C++, Java & C# all treated an unhandled exception in main() as a “successful” execution as far as reporting the process result code goes.

[#] In C/C++ you can use the two constants EXIT_SUCCESS & EXIT_FAILURE to avoid hard-coding a return code that is platform dependent. On Windows these equate to 0 and 1 respectively, although the latter could be any non-zero value. I seem to recall that these constants are defined by <stdlib.h>.

[+] Let’s ignore the fact that you might use “write-error” and are keeping to a similar model to cmd.exe. I have another post queued up that shows that the output mechanism is annoyingly broken in PowerShell when using file redirection if you’re considering using it to replace batch files that run under, say, a job scheduler.

Mock To Test the Outcome, Not the Implementation

2011-04-28T23:57:00.001+01:00

The other day a colleague came to me with a question about a problem they had trying to verify an expectation through a mock – they were trying to mock the invocation of an indexer property in C# with Rhino Mocks. Not knowing Rhino Mocks at all well I had to pass on the specific question, but it did raise other questions and it seems there was a ‘smell’ emanating from the test & code that my colleague had also picked up on…

The Feature

The code they were trying to test was a factory method that was going to be invoked by some 3rd party application framework. The method it seems has more than one responsibility forced upon it by the framework. The code looked something like this:-

public IView CreateView(IState state, string viewName,
string areaName)
{
var view = IoC.CreateObject<TView>();
var viewModel = IoC.CreateObject<TViewModel>();

viewModel.Load(state);
. . .
view.ViewModel = viewModel;
. . .
var area = ViewManager.Areas[areaName];
area.AddView(viewName, view);
. . .
return view;
}

Now, the code is already doing quite a bit more than I would expect a factory method to do, but that is down to the framework having what I consider excessive demands.

The Test

As I said above I’ve had very little experience with Rhino Mocks, and this post is not about that product which is undoubtedly very clever and very useful. No, it’s about what is being tested with it (the syntax is probably slightly wrong but I hope you get the idea):-

[Test]
public void testCreateView()
{
. . .
Expect.Call( viewModel.Load );
Expect.Call( ViewManager.Areas[areaName] );
. . .
var view = factory.CreateView(. . .);

Assert.That(view, Is.Not.Null);
. . .
}

Let’s put aside test naming conventions for now and focus on what’s inside the curly braces. The test is attempting to verify (amongst other things) the following two behaviours:-

The state is loaded into the ViewModel object
The view is registered with the view manager

The big problem I have with the test is that it is not verifying that the outcome of the test matches these expectations but that it is verifying exactly how those expectations are being met. In essence the feature specification appears to be saying this:-

The state is loaded into the ViewModel object by calling the Load() method
The view is registered with the view manager by associating the view with named area accessed via the indexer property

In short, why should the test prescribe the mechanism used to implement the feature?

Variations On a Theme

I’m going to discuss the “view registration” expectation first because I have another thing to say about the “state loading” that is more design related.

The expectation of the application framework is that the created view will already be registered with the “view manager” by the time the factory method returns. But we may be able to implement that in more than one way depending on what the interface to the view manager looks like. It’s not uncommon to augment a simple interface with different overloads (and extension methods) to allow the caller to write simpler code:-

viewManager.AddView(area, view);

viewManager[area] = view;

You may also choose to introduce your own facade over the abstraction and therefore invoke it indirectly:-

// Does all 3 necessary steps
myViewManagerFacade.Register(area, name, view);

I’ll leave it up to you to decide which you prefer, but my point is that your test should not care which of these many forms are used. In fact it should be possible to switch between them so long as they still adhere to the contract. By not being overly specific in your tests you stand a better chance of not writing brittle tests that break in the face of minor refactorings. One such refactoring that I’ve done in the past is to simplify an interface by introducing extension methods to add syntactic sugar so that the interface remains as simple as possible; should I change the expectation to now be on the extension method or the new interface method? I don’t believe that you can mock extension methods and so now your test code looks inconsistent with your production code. If you introduce a facade, that itself is mockable, do you verify that interface call or the underlying one? From the test’s perspective the outcome is the same and that is what ultimately matters.

So what would my test be? Well, if the real view manager type brings too much baggage to allow me to use it directly in the test I’ll create a mock that will implement the bits I need, i.e. record the results of the interaction:-

public Dictionary<…> m_views = new Dictionary<…>();

public void AddView(string area, IView view)
{
Views.Add(area, view);
}

In mocks I have absolutely no qualms about using public fields because the test is the important thing and the mock is a means to a different end than production code. Then in the test I just assert that the view was registered with the correct name:-

{
var viewManager = new MockViewManager();
var factory = new Factory(viewManager, . . .);
. . .
var view = factory.CreateView(state, . . .);

Assert.That(view, Is.Not.Null);
Assert.That(viewManager.m_views[area],
Is.EqualTo(view));
. . .
}

This is a manual mock and no doubt Rhino Mocks makes this kind of thing even easier to write if I just bothered to RTFM. The key point is that I’m only testing the outcome, not the means by which that outcome is achieved. I can change the factory implementation in the ways mentioned earlier and it in no way invalidates the test because from the application framework’s perspective I have met the contract.

Reinventing The Wheel

One thing that worries some people is that I’ve just re-implemented the ViewManager purely to test some other code and have therefore wasted my time re-inventing the wheel and created even more code that needs to be maintained. Well, yes and no. Note that I said I’d only do it if I couldn’t reuse the real ViewManager in my tests. Unit testing is about isolation and determinism – you don’t want a lot of dependencies making your tests brittle. If the real ViewManager cannot easily be plugged in by mocking its dependencies then yes I’d mock it, but if I can reuse it then no I wouldn’t.

When it comes to integration and system testing things get harder and at that point you may have to substitute more fully featured mocks to avoid taking a troublesome dependency on, say, an externally owned service. I may provide many mock implementations of the same service to suit the different types of automated tests being run.

Flexible Designs

So back to my other issue. The reason my colleague added the Load method was in part so that he could verify that the ViewModel object had been initialised from the persistent state. He suggested that in real world use the view model is unusable if this hasn’t happened and I get his point. But I would argue that it is not necessarily the responsibility of the factory to ensure that this happens. Once again we have to put ourselves in the shoes of the client (the application framework) and think how this would be observed – how can we interact with the resulting object in a way that depends on this interaction having being played out?

There are two ways to ensure that this orchestration happens:-

Explicitly, by the factory itself or some other mediator in the chain
Implicitly, as part of the ViewModel object’s construction

The first approach, which is the one my colleague had chosen, raises other questions such as what happens if someone calls Load a second time later? Or what behaviour can be relied upon between construction and Load being called? Two-phase construction is something I really try and avoid because when somebody does do the unexpected it can manifest as one of those hard to find bugs. And so I favour that latter, after all it’s called a constructor for a reason because it’s expected to construct an object :-).

The obvious follow-on question is “if I delegate responsibility of state loading to the ViewModel constructor then do I still need to test if the object returned from the factory is fully constructed?” Almost definitely “Yes”. There is a reason the method takes arguments and so you are probably expected to use them. If you have overloads of a factory method that do different things, say, one default constructs a view and the other constructs from an external resource you’d want to know that the objects were constructed in the appropriate way.

Now let’s add another common design choice into the mix – lazy loading. Does it matter if the state loading happens directly during construction, asynchronously in the background or even synchronously as some by-product of a method call that depends on the object having been initialised? The test should succeed either way because the object is either fully constructed up front or by the time the assertions attempt to verify the expectations.

Once again let’s compare the two approaches. The original test constrains the implementation into calling a specific method called Load, whereas the latter allows for a number of different implementations. It also allows the code to be refactored between the various implementations without affecting the tests.

Test Driven (Development|Design)

I think this kind of problem highlights the difference between the last word in TDD being interpreted as [D]evelopment or [D]esign. In the former you know what production code you’re going to write and therefore you bias the test towards verifying what you’re going to implement. But in the latter you don’t really know or care how it’s going to be implemented and so the test only verifies that the design allows for it to be done in some way. And the best way to achieve that is to only verify the results of any interactions and not the interactions themselves.

Using Extension Methods to Extend A C# Interface

2011-04-27T21:02:00.001+01:00

When I first read about extension methods being added to C# (which was a long time before I actually started using C#) I thought they were a liability. Superficially it seemed to be “an accident waiting to happen” because it’s hard enough trying to understand the responsibilities of a class or interface when you have the code in front of you. With extension methods there could be behaviours bolted-on by a third party that you just can’t see.

Once more the classic quote from Spider-Man “with great power comes great responsibility” rears it’s head. Used judiciously extension methods are a great way to graft on functionality that a class author hasn’t provided, or a way to provide a Fluent Interface to a particular problem domain. I would hazard a guess that the String class has had significantly more extension methods than any other as I personally have a written a few obvious ones such as IsEmpty and JoinUsing.

Extending concrete classes probably makes a lot of sense, but extension methods can also be used to extend interfaces. That sounds a but more surreal – how do you extend an abstraction in a concrete way that applies to all implementations of that abstraction? There is nothing magical it’s really just turning the idea of an abstract base class on it’s head…

An Initial Configuration Mechanism

All non-trivial systems end up with some common configuration mechanism and my current one is no exception. Whilst bootstrapping the initial components I didn’t want to get into the nitty-gritty of the .Net ConfigurationManager class so I introduced a simple interface:-

public IConfiguration
{
string Setting(string name);
string Setting(string name, string defaultValue);
}

The first overload will return the setting value or an empty string if there is no setting with that name[*]. The second overload returns the default value specified when there is no value to return. The natural implementation for the first overload then becomes:-

public string Setting(string name)
{
return Setting(name, “”);
}

I tossed and turned about leaving the first overload out as it forces the implementer of IConfiguration to implement something that is really just syntactic sugar. Initially the only way round this that I knew was to introduce a default implementation so that the implementer just had to derive from it to get the syntactic sugar method:-

public abstract class ConfigurationDefault
: IConfiguration
{
public string Setting(string name)
{
return Setting(name, “”);
}
}
. . .
public class XmlConfiguration : ConfigurationDefault
{
. . .
}

The use of abstract base classes is a tried-and-tested approach to reusing code in a polymorphic class hierarchy and so I was happy. At least until some new requirements came along.

Improvements to the Configuration Mechanism

The implementation served us well to start with but I knew from experience that there are certain behaviours I’ve found really useful, such as being able to specify environment variables like %ProgramFiles% and %TEMP% in file paths that are then expanded on demand. By now I had started getting to grips with extension methods and quickly realised that this was a behaviour that was also independent of any IConfiguration implementation, so I wrote it as an extension method:-

public static class IConfigurationExtensions
{
public string PathSetting(this IConfiguration
      configuration, string name, string defaultValue)
{
    var setting = Setting(name, defaultValue);

    // Do substitutions
    . . .
}
}

One Method to Serve Them All

After adding this support and a further behaviour to allow arbitrary substitution of other settings I found that it was all too easy to mess one up and get odd behaviour. We decided that the silent failure when doing substitutions was a bad idea and that instead we’d prefer to throw an exception if a setting or environment variable did not exist. But that caused a problem because the original interface was not designed to allow the absence of a setting to be detected, it just returned an empty string or the default value and I didn’t want to get into passing a random default value just to detect if the setting existed or not.

At first I thought that I should just extend the interface with a TryXxx() style method that would return true or false along with the value if it existed. But it was whilst thinking through the ramifications of this change that I realised that this was in fact the only method that I needed in the interface:-

public IConfiguration
{
bool TryGetSetting(string name, out string value);
}

All the other methods could be written as extension methods implemented in terms of this one interface method:-

public static class IConfigurationExtensions
{
public string Setting(this IConfiguration
                        configuration, string name)
{
    return Setting(name, “”);
}

public string Setting(this IConfiguration
      configuration, string name, string defaultValue)
{
    string value;

    if (configuration.TryGetSetting(name, out value))
      return value;

    return defaultValue;
}

public string PathSetting(this IConfiguration
      configuration, string name, string defaultValue)
{
    . . .
}
}

This also meant that I could drop the abstract base class that provided the default implementations of the syntactic sugar methods.

OK, so I haven’t exactly turned the abstract base class on its head, I’ve really swapped an abstract base class for a static helper class and with that I have lost the ability to re-implement the methods another way in the derived classes. But the whole point of the base class was not to enable polymorphism but to provide reusable helper methods – inheritance was just a means to a different end.

My original fears about extension methods lay unfounded and instead have provided a much cleaner solution to the problem of extending an interface to provide syntactic sugar (e.g. a fluent interface) where there previously was none. This revelation will no doubt be nothing new to experienced LINQ users because that’s exactly what it provides for IEnumerable, but for me it’s added a new dimension to how I think about and design my own interfaces.

[*] Yes I could have returned ‘null’ but my C++ roots were telling me that I should return a value not an empty reference (or NULL pointer in the C++ world). I still struggle with this duality of the String type – null vs “” – although as I start to use nullable value types more in the Data Access Layer I feel I’m getting to grips with it slowly.

You Write Your SQL Unit Tests in SQL?

2011-04-22T09:06:00.001+01:00

One of the questions that came up after my ACCU London talk back in January (xUnit Style Database Unit Testing) also came up last week after my ACCU Conference talk (Using xUnit As a Swiss Army Testing Toolkit):-

Why do you write your database unit tests in SQL?

My immediate and somewhat flippant response is:-

Why wouldn’t you write your SQL tests in SQL? You wouldn’t write your Python unit tests in C++ would you?

Now, if you swap the words Python and C++ around in that sentence it begins to sound more plausible. In fact it’s an idea that I’ve looked into in the past, but not so much at the unit test level, more at the component/integration level. Kevlin Henney, in his ACCU London talk last year, pointed out that you could use NUnit[*] to unit test C++ code via the magic of C++/CLI so using a different language to express your tests is by no means wrong but it comes with another cost…

Portability (People Skills)

Steve Love did a talk at the ACCU Conference back in 2009 about portability. Of course what most people think of when you say portability is about the source code and how it works across multiple platforms (e.g. Windows/Unix) or across multiple toolchains (e.g. Visual C++/GCC). But there is another aspect to it and that is portability across people – can you hire another person to do the same job?

I’ve worked in small teams where everyone has to do everything and also in larger teams where people specialise in a particular language or technology. It is hard enough to find good people as it is, adding another orthogonal requirement to the role only makes the search that much more difficult. OK, so many experienced T-SQL developers will probably also have some knowledge of C# and the Cool Kids will happily remind you how we should all be Polyglot Programmers and be able to work with multiple languages, but the organisations I work in don’t get to attract the superstar programmers. Sometimes you’re ‘given’ members of staff from another team, presumably because they can save money by avoiding firing one and hiring another in the mistaken belief that you will turn them into a superstar programmer.

Isolation (Tooling)

Even so, just because they might know C# doesn’t meant they feel comfortable using, say, Visual Studio as their main development tool. Our automated test runner currently uses SQLCMD and we (can) write our SQL code and tests using just SQL Server Management Studio (SSMS). This means that not only are the tests expressed in their language of choice, but the tooling is also the one they are likely most comfortable with. I’ve not tried SQL debugging with SSMS but I would have thought that it is far simpler if you’re not trying to debug across a high technology stack (it feels akin to debugging across RPC calls).

That last point sums up my approach to unit testing, which is that it is all about isolation. That doesn’t just apply at the code level – at the thing under test – but also to the toolchain that sits around it. That doesn’t mean that you should only use primitive tools like Notepad & SQLCMD to develop your code, but that you try and avoid having too many other layers (e.g. C#, ADO.NET) ‘above’ the code in your test scaffolding as it makes it unnecessarily complicated and can make it harder to do things like debugging.

The Right Tool For the Job

Deciding what the right tool is for any job is getting harder every day because new tools are springing up all the time that bring the best bits from other tools together to solve some other hybrid problem. I’m definitely sold on the idea of writing SQL unit tests in SQL as the language provides enough features to allow you to write fairly clear tests, but our Heat Robinson style test runner is clearly begging to be replaced by something better. I also need to look again at how Visual Studio studio approaches the problem because they have an army of developers [hopefully] much cleverer than me that should be able to balance these requirements. I also promised myself I’d got back and look at how TSQLUNIT was coming along as that is also more mature than our framework.

We never set out to write a SQL unit test framework, we just did what felt natural to bootstrap ourselves and that’s the way it ended up. Looking back I still think it feels the right way to go.

[*] Whereas other languages seem to have a few key unit test frameworks, C++ is blessed with a bucket load. At the ACCU Conference last week someone asked for a show of hands of those people that had written a unit test framework and there were quite a few hands. I’ve said before that I believe it’s somewhat of a Rite of Passage and writing a unit test framework seems to be common part of that voyage of discovery.

Database Enumerations – A Non-Performant Query

2011-04-22T09:02:00.001+01:00

Back in September last year I wrote about how we have implemented constants and enumerations in our SQL Server database (Implementing Constants & Enumerations in a Database). Essentially we have used parameterless User Defined Functions (UDFs) to act as the symbol for the constant or enumeration. At the time I said that I was not aware of any performance impact that this had, but that I would write it up when I found one. Well, I wrote a support query the other day and noticed that there was quite a difference between the query with and without the UDF. It looked something like this:-

SELECT *
FROM   Requests r
JOIN   Tasks t
ON     r.RequestId = t.RequestId
AND    t.TaskType = dbo.TaskType_Calculation()
WHERE r.ValueDate = . . .

The two tables involved in the join have millions of rows in them and although there is an index covering part of the query the selectivity is quite low in this case. I didn’t look at the execution plans because this is a template query that I use regularly and the only difference was the explicit filter on TaskType using the UDF. The query took 13 seconds to execute.

Given how often I run very similar queries to this I suspected this was the evidence I was looking for to show how a UDF could affect query performance, and so I did the obvious thing which was to substitute the call to the UDF with the relevant constant:-

AND t.TaskType = 2 -- dbo.TaskType_Calculation()

Lo and behold the query ran in just 2 seconds. I switched back and forth between the two forms to see if the performance was consistent and it was – 2s vs 13s. Somewhat perturbed now that our lovely idea may be an accident waiting to happen I checked the performance using an intermediate variable, which is effectively how we have used this idiom in our code (because you can’t always reference the UDF directly and it’s a bit verbose):-

DECLARE @calculationTask udt.TaskType_t
SET     @calculationTask = dbo.TaskType_Calculation()

SELECT *
FROM   Requests r
JOIN   Tasks t
ON     r.RequestId = t.RequestId
AND    t.TaskType = @calculationTask
WHERE r.ValueDate = . . .

Tentatively I pressed CTRL+E in SSMS to execute the query… and… it returned in just 2 seconds. Just to be sure I then alternated between all 3 forms but the intermediate variable version still performed exactly the same as the hard-coded constant. Phew! This is clearly not an exhaustive test but it does give me some continuing comfort that it is a sound idea but that we now know for sure that it can have a performance impact in some scenarios.

P.S. This was on SQL Server 2008 R1.

Describe the Problem, Don’t Prescribe the Solution

2011-04-09T00:15:00.001+01:00

The other day whilst scanning the backlog of change requests in JIRA I came across the following entry:-

Replace the use of nchar(x) with nvarchar(x)

And that’s all it said[#]. Now I knew instantly which tables the author was referring to because, like many teams, we only use a fixed width char(x) column for flag type fields such as Y/N or true fixed width strings like currency codes. The reasons why and how these couple of table definition slipped past the informal code review process is somewhat irrelevant, but suffice to say that we were days away from release and chose to take the ‘hit’ instead.

The fact that variable width text was being stored in a fixed width column was not causing any problems within the database or the back-end server code. Yes it was annoying when you cut-and-pasted result values from SSMS (SQL Server Management Studio) as you got a load of trailing spaces that you just had to manually trim:-

...WHERE t.Column = ‘VALUE ‘

So I put this one aside mentally filing it with the other issues of Technical Debt as there was nothing to fix per-se – the mechanism was ugly but it worked. There was also no reason to suspect it may cause any performance problems either as the tables contain relatively few rows of data.

Then a conversation some weeks later with the change request submitter brought up the subject of raising Technical Debt style change requests and I opined that now we were live the database schema is potentially a whole lot harder to change because we have data to migrate as well[+]. In this instance it wasn’t even vaguely in the area of rocket science, but nevertheless it was non-trivial and ultimately had to be paid for. And then the real requirement came to the surface and brought with it a new perspective…

These fixed width columns were not causing the back-end any problems because we never consumed them – we only every compared them to string literals or ran support queries. The impending GUI on the other hand had controls that displayed some of these fields and apparently the UI framework would bloat the size of the control to allow room for what it presumably thought were always going to be long textual values. My gut reaction was abhorrence at the thought of changing the database schema of a base table just to fix a layout issue in the GUI; that is the kind of nasty coupling that causes change paralysis because managers get worried about what will break. I understood it was a pain having to manually deal with the fallout in the GUI layer but there should be other much simpler solutions that didn’t involve writing SQL scripts to fix the table schema and migrate the data – namely trimming the string somewhere in the data access layer or the SQL query used to fetch the data[*].

Hold on a second I thought. What are you doing directly accessing base tables anyway? The rest of the public interface to the database is via stored procedures and views – you’ll need to abstract that access away behind one of those before going to production. More importantly that gives us another option for remediating this behaviour without touching the schema and data, and better yet, that would be inside the database public interface and so allow the refactoring to occur without changes to any clients. Of course this then forces the issue of what the public interface to this mechanism should really be and at that point it seems prudent to design it properly if the existing implementation is going to leak out badly. But if it doesn’t we can save that job for another day.

The important thing though is that we have identified the root cause and can therefore now begin to assign some value to the potential choices and costs. When the request implies changing the schema of persistent data purely for the sake of it, the risk seems high and costly and the reward seems low. But when faced with a path that allows for incremental change so that the full cost and risk is distributed over smaller steps it seems much more appealing as you can put your spade down at any point.

So how would I have initially phrased that change request? Well a few weeks ago I would have gone for something like this (I can’t remember what the exact issue with the GUI was so it still seems somewhat lame I’m afraid):-

Laying out the XYZ controls has to be done manually because the ABC data has extra whitespace padding due to the use of nchar(x) instead of nvarchar(x) for the FGH columns. These columns should probably be nvarchar(x) anyway as they contain variable width text.

However a recent article by Allan Kelly in the ACCU magazine Overload (The Agile 10 Steps Model) contained a sidebar titled “What’s the Story?” about User Stories. This described a common format for stories:- “As a [Role] I can [Action] So that [Reason]” that felt akin to the use of “[action] [should] [when]” that I follow when writing test case names (itself a variation of the “[given] [when] [then]” style). I felt that this would allow those of us less natural writers to still construct a succinct requirement even if it does sound somewhat robotic, but more importantly it would help bring the value of the request to the forefront. User Stories is something that I’m vaguely aware of but have largely glossed over because my requirements are nearly always technical in nature and tend to come through very informal channels so any connection with ‘End Users’ is usually missing; it just hadn’t clicked that “User Stories” could just as easily be read as “Developer Stories” or “Support Stories”. So this is my attempt at writing it this way:-

As a [developer] I can [trim the whitespace from the ABC data] so that [the XYZ controls are not laid out manually].

OK, so this reads very unnaturally because the story format implies a repeatable action rather than a one off event. A small and hopefully obvious change to the wording and order though should make this more natural:-

As a developer it would save time if the XYZ controls were laid out automatically by ensuring the ABC data contains no excess whitespace.

Do you think that reads better or not? I think the value, while still somewhat intangible as all technical requirements like this are, is clearer because you are forced to justify the action rather than providing it as an afterthought. The beneficiary of the requirement is also clearly stated which helps easily separate the functional from the non-functional.

Of course what the original submitter would probably have written would still be something like this:-

As a developer I need to replace the nchar(x) columns with nvarchar(x) columns so that the GUI layout is automatically laid out.

…because the [action] would likely still be the same, but one would hope that when writing the [reason] alarm bells should start ringing as the tight coupling between the database schema and GUI becomes apparent.

[#] This is also a common failing of n00b posters on sites like Stack Overflow. They have already decided on what their solution is and are trying to elicit an answer that helps implement that solution rather than taking the time to explain the underlying problem in case the reason they can’t see the proverbial wood is because there are a load of trees in the way.

[+] As you will see later our well defined SQL public interface and set of SQL unit test means that it is actually quite easy to refactor the schema.

[*] Eventually an O/RM such as Entity Framework or NHibernate will no doubt be in place to tackle this, and so it will be interesting to see how you would go about dealing with this kind of problem when you don’t immediately have access to either the SQL or the code that binds the result values to the DTO members. I’m sure it is easy, but how much out of the way would you have to go?

Friends Don’t Let Friends Use Varargs

2011-02-12T23:12:00.001Z

Whilst writing my previous post about being clever with strings and varargs functions I was reminded of another time that I had a run-in with them. Luckily I wasn’t the protagonist this time but a colleague instead. He decided that he wanted to make it easier to add an arbitrary set of strings to a particular class via the ctor. In modern day C++ you might do it via an iterator pair like this:-

{
const char* words[] = { “cat”, “mat”, “dog” }

Thingy thingy(words, words + ARRAYSIZE(words));
. . .
}

However this was many years ago and the project was MFC based so the use of STL idioms were very thin on the ground[$].

Another less obvious choice might be to define a vector, add them to that and define a ctor that takes a vector of strings. But nope, my colleague decided that he could avoid the ugly array definition and performance hit of the vector approach by declaring a varargs ctor like so:-

class Thingy
{
public:
Thingy(const char* word1, ...);
};

This meant that you could provide an arbitrary parameter list of strings and then terminate it with a NULL parameter:-

{
Thingy things(“cat”, “mat”, “dog”, NULL);
. . .
}

Naturally I suggested you don’t mess with varargs and that this was an accident waiting to happen[#]. I did agree that it was not an uncommon technique to define an array of items and use a NULL terminator at the end and that what he was effectively doing was this:-

{
const char* words[] = { “cat”, “mat”, “dog”, NULL }

Thingy thingy(words);
. . .
}

At the time we had a definite need to support up to three arguments which were actually all CStrings. I’m sure I suggested writing three ctor overloads that then call a common internal helper method instead for now:-

class Thingy
{
public:
Thingy(CString arg1);
Thingy(CString arg1, CString arg2);
Thingy(const CString& arg1, . . . arg2, . . . arg3);
};

But no, that was far too much work, his way was simpler and, after all, we were all experienced programmers so we don’t make those kind of silly mistakes…

Sadly I can’t remember when the first bug showed up or why it remained hidden for so long[+], but one did suddenly appear and guess what was missing? Yup, the NULL terminator. Of course now I had to check for any other occurrences of malformed ctor calls and I soon discovered one more. Clearly someone was not aware of this need to add a terminator so I did a little Software Archaeology and rooted around in our VCS to find out who needed a little educating… But of course, it was my esteemed colleague, the very same person who suggested that we wouldn’t make that kind of mistake!

So remember kids – Just Say No to Vargs Functions. Unless of course you’re doing Template Meta-Programming in which case you’ll be [ab]using them for a different reason.

[$] Due to it’s age MFC has its own containers and single iterator type and doesn’t adhere to the STL idioms that all all know and love now. Consequently if you spent your years knee deep in MFC code you could very easily to be unaware of them.

[#] Of course this was many years before I became aware of my own misfortunes with varargs function and anyway I’d read all the books by Scott Meyers, Herb Sutter, et al so that made me the fountain of all knowledge, right?

[+] It just goes to show that even with debugging techniques like the compiler using guard blocks and filling uninitialized stack space with a non-zero pattern it’s still possible for the stack to end up correctly formed by luck and not by judgement. That’s especially true when the stack variable is a ‘bool’.

A Not-So-Clever-Now String Implementation

2011-02-09T01:13:00.001Z

Back around 1995 when I first started using C++ commercially[+] I picked up an excellent book by the late Paul DiLascia titled “Windows++”. It was about writing Windows applications in C++ and the book developed a framework along similar lines to the then fledgling MFC. I decided that writing my own C++ framework would be a great way to learn C++ and I (somewhat foolishly) decided that I would use a similar code style to MFC so that I could easily reuse any of my framework code in my day job (after making it production ready of course).

Reinventing the Wheel

The heart of any C++ class library back then was the String class. C++ was still evolving and my work involved using MSVC 1.52c (aka Visual C++ 1.5) which had no template[^] support so naturally I created my own ‘classic’ class that managed an ANSI char buffer:-

class CString
{
private:
int bufsize;
char* buffer;
. . .
};

One of the problems with this class was that it was too easy to screw up[*] when using the printf() family of functions because it had a different memory layout to a plain char*, so if you forgot to invoke the moral equivalent of c_str() you could end up crashing the process because the bufsize member would be interpreted as a string pointer:-

{
CString str = “chris”;
printf(“hello %s”, str); // BOOM!
}

Swapping the two members around would probably work for the single-string case above, but not for N arguments as the trailing bufsize member would become the next string:-

{
CString first = “chris”, last = “oldwood”;
printf(“hello %s %s”, first, last); // Still BOOM!
}

Now, I felt that giving up the beauty of the printf() family was too much; I had just spent the last few years learning C and so I had the various format specifiers committed to memory. Windows also had its own variation, wsprintf(), that supported a few other useful extensions such as near/far pointers and the ANSI/Unicode build dependent variants of “%c” and “%s”.

Tempted by the Dark Side

One day whilst debugging, I stepped into some code that passed a MFC string directly into a printf() call and realised that it hadn’t gone bang. As I stepped into the printf() call I realised that the reason it had worked was because an MFC CString object had the same stack footprint as a raw char* pointer. When an object is passed into a varargs style function the object is passed by value, which in the case of a CString is effectively a single pointer. Also because the class only has a pointer based member it has no extra padding or alignment requirements – at least not on the x86 targets I was interested in.

I thought this was rather clever and set about changing my implementation so that it had similar behaviour. It hadn’t registered at first but I later realised the similarity with the implementation of the COM BSTR type which also stores the string length in an area preceding the string contents. And so I switched my implementation to this:-

class CString
{
private:
class CStringBuffer
{
      int   bufsize;
      char buffer[0];
};

char* string; // Really CStringBuffer.buffer
. . .
};

So now I allocate a single chunk of memory for the header (CStringBuffer) + contents (N * sizeof(char)), but instead of storing a pointer to the CStringBuffer header as a member, I slide along the memory block and store a pointer to the first character of the string, i.e. &CStringBuffer.buffer[0]. I used an interesting Microsoft extension- the zero element array. This beast used to be quite common in the Windows headers and is a useful way to create a placeholder for a variable length array that follows a structure in a contiguous piece of memory. The alternative is the standards compliant “char buffer[1]”, but then that makes your sizeof() arithmetic a little more complicated as you have padding to account for along with the extra byte in the array. Of course to access the header structure without littering the code with casts I wrapped the necessaries in an inline method:-

CStringBuffer* CString::GetBuffer()
{
return ((CStringBuffer*)string)-1;
}

All very ugly and switching from a C-style cast to a proper reinterpret_cast<> doesn’t make it any less unpleasant. Still, I happily lived in ignorant bliss for well over a decade until I decided to make a conscious effort to drop my CString class in favour of the standard string type std::string – at least for new libraries that is. The conduit was reading Matthew Wilson’s Imperfect C++ and coming across his STLsoft libraries. Suddenly I found a new impetus to deal with the impedance mismatch between MFC/ATL and Standard C++ (something I believe his new book may well be tackling).

A New Hope

The switchover went swimmingly well with my new Core and XML libraries showing the way forward whilst the old WCL, NCL and MDBL libraries became the epitome of the word ‘'legacy’. Where possible I added extra overloads to allow std:string’s to be used naturally but by-and-large the worlds were kept apart and the cost of the occasional implicit[#] conversion from a CString to a std::string was left for the profiler to find, should it ever show up.

Then I decided that where possible all new code, even in my legacy libraries, should adopt the proper string type and that I would slowly migrate code across. So I started making deeper and deeper refactorings (backed by my unit tests – where they existed) and still things looked good. Visual C++ 10 (aka VS2010) arrived on the scene and so a port was required to ensure my codebase was compatible and much to my surprise a unit test in my ANSI to Unicode conversion helper functions choked on an ASSERT inside the STL. The code was wrong, no doubt about it, but why hadn’t I see it before as I usually use STLport to keep me honest? I did a build with STLport 5.2 and it also whinged! So I fixed my iterator dereferencing bug and ran the entire suite to make sure I hadn’t entered into the murky waters of Undefined Behaviour anywhere else by accident and again it when BANG! One of my COM library tests was now failing! Running the test under the debugger revealed a more serious problem as the MS debug runtime started complaining about other matters…

Undefined Behaviour Strikes Back

It took a couple of debugging attempts but then I winced as I spotted my error. I had switched over a couple of helper functions from returning a CString to a std::string and the result of that function was in turn being passed to a varargs logging function like so:-

std::string SomeClass::toString()
{
return . . .;
}
. . .
void Elsewhere::DoSomething()
{
LOG(“Blah, blah, blah %s”, something.toString());
. . .
}

Yup, there was no call to c_str()!

Then the true cost of my foolish design choice started to dawn on me as I realised that there was no obvious way to catch the problem at compile time because varargs functions are effectively type-less once you extend into the ‘variable’ part of the parameter list. Yes, it might be possible to do something with templates and macros but that would be non-trivial. Add in the fact that I have used varargs style functions to do all my string formatting, often in my exception handling, and you can guarantee that there are plenty of time bombs in the shape of rarely executed exception handlers just waiting to go off.

Another New Hope

Maybe it was fate, but I had always been meaning to get my Core library and its unit test suite to build and run under more than just Visual C++. So I started investigating the alternatives such as GCC and Digital Mars (see “GCC for the Visually C++ Impaired”) and soon I had both Core and XML working under GCC 3.x and 4.x via the Code::Blocks IDE – albeit only with the standard warning level.

After being distracted for some months I finally got back to playing with GCC as I had always intended on compiling my code with the highest warning settings available to see how much static code analysis GCC can do over-and-above what VC++ already does – an awful lot it turns out (a topic for another day). And, guess what one of the behaviours that it singles out? Yup, the passing of non-primitive types to printf() style functions! I’ve yet to dig deeper and see if it is varargs functions in general or whether it’s functions like printf(). Either way it spots code like this:-

{
std::string text(“. . .”);
printf(“%s”, text);
}

Which is great because I now have a clear path forward for deprecating my CString type safely:-

Add a c_str() method to my CString type to provide compatibility with std::string
Start compiling[$] more of my codebase with GCC 4.x to identify and fix those bugs by appending a call to .c_str()
When a suitable refactoring moment arises switch code from my CString type to the standard string type
Test, test and test some more…

Epilogue

I often wonder whether it’s still worth maintaining my 15 year old C++ codebase of tools. Many of them were made obsolete years ago and so I have stopped fixing up the really old applications but the libraries live on – kept alive by each new utility that I write. I could start again from scratch but you don’t get that luxury in the real world so I’m inclined to try and refactor my way out. This way it remains a learning exercise – albeit one about refactoring legacy code – which sadly is what a lot of commercial C++ projects have become.

[+] I first used it at university some years earlier and at that time you used the ‘overload’ keyword to mark overloaded functions. I managed to get G++ (nay GCC) working on my Atari 1040 STE (with twin floppy drives) but it was not a pleasant experience. I already thought C was too bloated in comparison to Assembly Language and C++ was therefore even more pointless. Sooo naive…

[^] Not that I knew what a template was back then of course. I never really started to understand templates until the turn of the millennium and Template Meta-programming is still largely theory to me – that’s why there’s the Boost libraries!

[*] If you were lucky it was in a common code path and you got a UAE/GPF there and there; but often you weren’t and it became a time bomb if the code path was in an error handler that was called infrequently.

[#] Yes, I added a conversion operator to ‘const char*’ to avoid having two overloads on every text API method or littering my code with .c_str() style warts. Choose your poison…

[$] Sadly, although my Core and XML libraries link and I can therefore run the unit tests, I can’t do the same for my WCL, COM or MDBL libraries or projects dependent on them (i.e. everything). But that’s ok because it’s the static code analysis I’m after anyway.

My First Presentation

2011-01-28T08:25:00.002Z

Last Thursday[+] saw me popping my “presentation” cherry as I stood up in front of a small crowd of around 35 people and waffle on for 60 minutes about unit testing database code (SQL). The size of the audience and the number of questions at the end and in the pub afterwards makes me feel that it can probably be chalked up as a success. I certainly got a buzz out of it and more importantly I’m glad to get one under my belt before the ACCU Conference in April where I’ll be presenting “Using xUnit As a Swiss Army Testing Toolkit”.

It was pretty obvious from my twitter feed that I was more than a little bit nervous and even though I had a few calming replies from my fellow ACCU members I was still somewhat pensive. The ACCU is full of very smart people and the topic on which I was speaking was not what I would consider one of my core skills and so I was well aware that the room would contain people far more knowledgeable than me on the subject. The nightmare scenario was one of those people asking the kind of ‘obvious’ question that would invalidate the entire talk. The basis for this is some of the ‘heckles’ that I’ve seen at past conferences and talks. Fortunately they were unfounded… this time.

So what did I learn from the experience? Well, it’s kind of nice being the centre of attention for an hour or so :-) The biggest thing, and I’ve said this before about blogging, is that it makes you think even harder about your topic so you feel doubly sure you don’t say anything really stupid. When first asked by Allan Kelly to do a talk for ACCU London I reluctantly said ‘yes’ as I find originality a big stumbling block – I’m always assuming that I need to say something novel. Because of the ACCU’s heritage[*] I didn’t want to say anything C++ related, I don’t know anything about Java, the iPhone, Python etc and anyway those angles are already well covered by other prominent members so that pretty much left something C#/SQL based. I also didn’t want to do a shorter version of my upcoming conference talk so I looked back over my blog for inspiration and realised I had a few posts relating to database development that I could probably stitch together into a coherent piece And so the following blog posts were fused together to the form the meat of the session:-

xUnit Style Database Unit Testing
Simple Database Build & Deployment With SQLCMD

…with the following posts providing some additional minor content:-

Debug & Release Database Schemas
Implementing Constants & Enumerations in a Database

Naturally I had extravagant plans when planning it 3 months earlier. I was going to develop my own test runner and follow Paul Grenyer’s fine example by running actual tests with a live demo. I was also going to have a few vibrant slides and follow Martin Fowler’s lead by eschewing bullet point lists and unnecessary images. And then Xmas came and went and like any real student the deadline was looming and still I only had a rough outline.

My wife, who fortunately knows a thing or two about presenting[#], and more importantly knows a thing or two about her lazy husband, provided some sage advice to get me bootstrapped. It didn’t take long to realise I was never going to have time for the live demos (either beforehand or during the talk for that matter) and I also dropped the ‘clever’ slides in favour of creating what were essentially a giant bunch of queue cards. Once I had gone through the entire talk for real I decided I probably shouldn’t try and run before I can walk…

This may seem really obvious to everyone else but there is a world of difference between talking in your head and saying it out loud. I spent a number of train journeys on my commute planning the talk and then going through it in my head, i.e. actually talking in my head to practice what I was going to say – I didn’t think my fellow passengers would be that interested in the content and they might also find it a little repetitive :-). So when I came to practice it out loud I got to hear all the pauses and umm’s and errr’s and it was truly disheartening. I practiced it again over the next few evenings and although the talk become more fluent, I was still annoyed with myself for forgetting to say certain things. My wife stepped in once more with another ‘obvious’ piece of advice “The audience doesn’t have a script, so they won’t know what you did and didn’t mean to say”. She also said that if a point was truly important I wouldn’t forget it. Boy it annoys me how that women is always right! I still ditched one slide right at the last minute as I realised the content was effectively irrelevant, it also was hard to convey fluently and would help bring me back to nearer the 60 minute mark.

On the night it seemed to go pretty much according to plan, except for the technical problems getting my laptop and the projector to acknowledge each other’s presence. I only remember losing my train of thought once and I believe I got all the key points in just like my wife said I would :-). I felt the urge on a number of occasions to ad-lib as I thought of something new or different to say but I held fast and stuck to the content I’d been practicing. If I ever get to do the talk again I guess I can always ‘refactor’[^] it. It was also nice to be able to point out two members of the audience (Phil Nash & Steve Love) whilst acknowledging their ‘indirect’ contributions.

After I had finished rabbiting there were a number of questions ranging from the classic “how do you get your developers to write tests?” to one from fellow member Ed Sykes about whether it’s possible to get code coverage for SQL based tests? No, I don’t know. In the pub afterwards we got to chew the fat further and a few more ideas popped into my head. The question now is whether the alcohol killed off too many brain cells or whether I can still recall what they were for future blogging purposes…

All-in-all a wonderful experience and hopefully an excellent bit of practice in preparation for the main event - the ACCU conference in April.

FWIW Here is a link to the slides (PowerPoint).

[+] The photo was taken by fellow ACCU member Schalk Cronje.

[*] There are members who sit on the C++ panel! They know more than than just their onions…

[#] She’s a producer of live events and one of her many talents is to train and rehearse speakers.

[^] I guess that means “the message” stays the same but I say it in a different way.

Intriguing SCHTASKS Bug

2011-01-10T18:19:00.001Z

To get our system up and running quickly we chose to use the built-in Windows task scheduler to invoke our nightly batch. During initial development we added a couple of tasks by hand using the Scheduled Tasks UI, but as we got closer to our release date we created a batch file to script creation of the scheduled tasks so that we had a formal (and version controlled) deployment process, e.g.

C:\> SCHTASKS /CREATE /SC WEEKLY /D … /TN “RunBatch”

The script was written and tested on a Windows XP desktop machine, but when we came to test it on our Windows server the script failed with the following error:-

ERROR: The creation of the scheduled task failed.
Reason: The Task Name may not contain the characters: <, >, :, /, \,|.

This was most curious as the server was running Windows Server 2003 (which is the same family as XP) and the task names were all pretty simple using just a number, dash and short name, e.g. “01-EstablishValueDate” – there were no out-of-the-ordinary characters at all. Naturally I took the dash[*] out first – no luck. Then the numbers – nope. Finally I resorted to a binary search to try and pin down exactly which character(s) it was…

It turned out to be the letter ‘D’ at the start of the word “Date”! The first three tasks I tried all had the word “Date” in them somewhere and they all worked after just removing the letter ‘D’. To be more specific it was a capital letter D as a lower case ‘d’ worked! This became our temporarily workaround whilst I did a little Googling in the background as I just couldn’t entertain the idea that this was the real problem; I’ll always assume I’m doing something wrong until Occam's Razor tells me it’s not me :-). Fortunately I quickly discovered the following forum post “Error In Schtasks Command Line” where there author was having no joy in creating a scheduled task called “Defrag”. Sadly it doesn’t say they resorted to just calling their task “defrag” as a workaround. Anyway a short time later I managed to discover an MSDN article that appeared to confirm the behaviour. In the KB article it says the following:-

The Schtasks command verifies the task name by comparing the task name with an invalid character set. If the task name contains one or more characters from the invalid character set, the Schtasks command denies the task creation request.

However, in race conditions, the invalid character set may contain a valid character. Therefore, the Schtasks command cannot create some tasks even if the task names contain only valid characters.

My mind boggles at what the SCHTASKS code is doing to create such a race condition? And why just the letter D?

[*] A common source of error is pasting command lines out of Word documents or web pages into console windows; this often results in curious error messages from the application. The problem is the “clever” document editor converting a dash (en dash, minus sign, etc) into a proper hyphen (em dash) or equivalent Unicode character at a different code point from the ASCII one. It may look like a dash to you and me but the “computer says no”.

Exception Safe By Design

2010-11-27T10:28:00.001Z

I’ve recently gone back to my simple database query tool to add a few essential usability features and like any legacy codebase there were a couple of surprises in store. Now, I’m not making excuses for myself, but most of the code is the best part of 10 years old and like any journeyman I’ve learnt a lot since that was written. Also the recent episodes of Matthew Wilson’s Quality Matters column in Overload (one of the ACCU journals) focused on exception handing and so these two examples became even more prominent.

Resource Management

One of the new features I added was to properly support non-query type statements, i.e. INSERT/DELETE (it was originally written to be a query tool but it’s been useful to run ad-hoc SQL as well). This involved me digging into the ODBC facade and eventually I came across this piece of code:-

CSQLCursor* CODBCSource::ExecQuery(...)
{
CODBCCursor* pCursor = new CODBCCursor(*this);

try
{
    ExecQuery(pszQuery, *pCursor);
}
catch (const CODBCException&)
{
    delete pCursor;
    throw;
}

return pCursor;
}

Now clearly you’ll have to put aside your disgust of Hungarian Notation for a moment and focus on the real issue which is the shear verbosity of the code. It’s also wrong. On the one hand I’ve been a good boy and only caught what I can knowingly recover from (a specific CODBCException type exception) but I’ve also missed the point that any other exception type will cause a memory leak. Clearly I should have used catch(…) because the exception type is irrelevant in this case.

Only I shouldn’t, because the try/catch block is entirely unnecessary. Although now I’m firmly in the camp that believes RAII is probably the single most important idiom within C++, I obviously wasn’t then. I try really hard to write code that ensures there are no memory leaks and I use tools like the Debug CRT Heap in Visual C++ and BoundsChecker to point out my mistakes. And yet this piece of code highlights how hard it is to do that, especially in the face of exceptions; if you don’t have 100% code coverage in your testing it’s just too easy to miss something. What is all the more ridiculous though is that the correct solution is less work anyway:-

CSQLCursor* CODBCSource::ExecQuery(...)
{
Core::UniquePtr<CODBCCursor> pCursor(new CODBCCur…);

ExecQuery(pszQuery, *pCursor);

return pCursor.detach();
}

Assuming I can make no changes to the method signature this ticks the box of being exception safe by design. The Core::UniquePtr smart pointer class is analogous to boost::scoped_ptr which ensures single ownership until it’s given away at the end. But there’s still a bad code smell which is that the method returns a bald pointer so the callee could still leak it. What we should return instead is the smart pointer object so that we explicitly transfer ownership to the callee:-

Core::SharedPtr<CSQLCursor> CODBCSource::ExecQuery(..)
{
Core::SharedPtr<CODBCCursor> pCursor(new CODBCCur…);

ExecQuery(pszQuery, *pCursor);

return pCursor;
}

One unpleasant taste in the mouth is that because of the change in scope we cannot just return a Core::UniquePtr because that would require temporarily copying the object[*] so instead we use a smart pointer that allows shared ownership – Core::SharedPtr. Funnily enough this transfer of ownership from caller to callee is actually right up std::auto_ptr’s street and for the knowledgeable this would be preferable from a performance perspective. But sadly you end up worrying about the caller and whether they know the pitfalls of auto_ptr and so perhaps you go back to the safer option.

Side Effects

Another feature I wanted to added was to avoid requesting credentials when the connection doesn’t need them and this put me right into the client method responsible for opening a new database connection:-

void CAppCmds::Connect(. . .)
{
try
{
    OnDBDisconnect();
    . . .
    App.m_oConnection.Open(strConnection);
    . . .
}
catch(CSQLException& e) [#]
{
    App.AlertMsg(TXT("%s"), e.m_strError);
    OnDBDisconnect();
}
}

This piece of code I found puzzling because I couldn’t obviously see why it called OnDBDisconnect() both at the start of the method and also in the exception handler as the connection will either open successfully or throw, in which case we had already cleaned up earlier. So I looked at what OnDBDisconnect() did to see why it might be necessary:-

void CAppCmds::OnDBDisconnect()
{
if (App.m_oConnection.IsOpen())
App.m_oConnection.Close();

App.m_pCurrConn = NULL;
. . .
}

Nothing earth shattering there – just run-of-the-mill cleanup code. And so I went back to the Connect() method and looked more closely. And then I spotted it:-

OnDBDisconnect();
. . .
App.m_pCurrConn = App.m_apConConfigs[nConnection];
. . .
App.m_oConnection.Open(strConnection);

There are side-effects that occur between the point when the previous connection is closed and when we attempt to open a new one. A quick scoot back through the version control repository[+] to see what changes have been made to this method and I see that it was checked in as a fix for another problem after failing to open a connection. Clearly my mindset was “fix the broken exception handling” instead of “fix the poorly written method”. The first rule of writing exception safe code is to perform all side-effects off to one side and then use exception safe mechanisms like std::swap()[^] to “commit” the changes.

So, for my example code above we need to introduce some temporaries to hold the unstable state, perform the Open and then persist the new state:-

OnDBDisconnect();
. . .
Core::SharedPtr<…> pConfig(App.m_apConConfigs[…]);
Core::SharedPtr<…> pConnection(new CODBCConnection);
. . .
pConnection.Open(strConnection);
. . .
App.m_oConnection.swap(pConnection);
App.m_pCurrConn.swap(pConfig);

Taking a cue from the preceding section about resource management I have used my Core::SharedPtr[$] smart pointer type as the connection will live on after this method returns. The catch handler remains but only to report the error message to the user, no cleanup now needs to be performed.

But we can do even better. If we take a closer look at what happens as a result of using swap() we see that the call to OnDBDisconnect() may actually be superfluous. In fact it may be better not to do it all. The call to swap() effectively means that the new connection will live on as App.m_oConnection and the old one will get destroyed at the end of the scope (the try/catch block). Now if we look at what OnDBDisconnect() does:-

void CAppCmds::OnDBDisconnect()
{
App.m_oConnection.Close();
App.m_pCurrConn = NULL;

UpdateScriptsMenu();
UpdateUI();
}

It closes the existing connection and resets m_pCurrConn (these both now happen automatically near the end of Connect()) and it resets the UI. If we look at the tail end of the Connect() method we see this:-

UpdateScriptsMenu();
UpdateUI();

What this all means is that if an exception is thrown then the user loses their current connection as well. If we perform all the work off to the side and don’t modify any state (i.e. don’t call OnDBDisconnect()) up front then the user at least gets to keep their current connection as a consolation prize. If the new connection does work then we just save the new state and let the old one disappear at the end of the try/catch block. Is that just being clever or an attempt to write the simplest code possible? A topic for another day…

[*] I need to go back and read up on the new std::unique_ptr type in C++0X as I’m sure this was designed for this kind of scenario – and also the std::vector<std::auto_ptr<T>> gotcha.

[#] Using an exception to signal failure when opening a database connection falls right into the “Using Exceptions For Flow Control” category in Matthew Wilsons Quality Matters column and would almost certainly fall into the real of “Not Exceptional” for Kernighan & Pike too. In this case I’m inclined to agree and will be adding a TryOpen() method at the next opportunity.

[+] Lone developers need Source Configuration Management (SCM/VCS) too. At least, if you want to perform any software archaeology to avoid regressions, that is.

[^] It would be great if std::swap() had a ‘nothrow’ guarantee as that would make it excellent for this idiom. But I’m not sure it is guaranteed by the standard - is it just a convention like not throwing exceptions from destructors? My Google skills found lots of chatter but no quote citing chapter & verse.

[$] Naturally you wouldn’t use the full type name everywhere but use a typedef instead. Given how prevalent the use of the SharedPtr type is (especially with types stored in containers) I tend to add a typedef right after a new entity class declaration for the smart pointer wrapper, e.g. class Xxx { . . . }; typedef Core::SharedPtr<Xxx> XxxPtr;

Reacquainting Myself With Sed & Awk

2010-11-23T23:07:00.001Z

I’ve always felt somewhat uncomfortable using Visual Studio as my main text editor because there seems to be legions of programmers out there that swear by vi and emacs’ text processing prowess. For me (a C++/C# Windows server-side programmer) I find that day-to-day all I need is syntax highlighting, cursor key movement and basic search & replace; plus the basic code navigation aid “Go to definition”. But that’s pretty much it. So I posed a question on accu-general to find out if I’m some kind of programming Neanderthal or whether all these supposed fancy text processing aids are actually used day-to-day. My straw poll suggests not and so I don’t feel quite so pathetic now.

Still, I had one of those moments the other day when I needed to perform a really simple text processing task – I needed to add 6 tab characters to the end of every line in a large (10’s of MB) text file. This kind of thing (modifying[*] a file that large) is somewhat unusual for me. Instinctively I opened the file in my (already running) Visual Studio and tried to work out how I was going to do it. The two options I would normally use are Find & Replace and a macro (which I then run repeatedly by holding the ‘run macro’ key down :-). Given the size of the file I didn’t have all day for the macro to run so it had to be a replace using a regex. The one I came up with replaced the \n character with \t\t\t\t\t\t\n and then I waited for VS to do its thing…

Whilst waiting I thought to myself how much easier that probably would have been with Sed and/or Awk. I could even hear the sniggers of those Unix programmers around me. But I haven’t used either tool in 20 years and I don’t have a copy installed by default because I’m running Windows of course. I remember trying ports of the major Unix tools way back when I started programming professionally and it was painful because of the fundamental difference between who is responsible for handling wildcard expansion (the shell in Unix vs the program in DOS). Plus the memory constraints of DOS meant they weren’t altogether reliable[#]. Time has obviously marched on but I’m still jaded by the experience of the very distant past and I’ve also discovered various alternatives that have made up the shortfall in many respects.

And then I ran into another classic issue – trying to work out which SQL scripts had been saved in ASCII and which in Unicode (SQL Server Management Studio annoyingly uses Unicode by default). This was causing my colleague’s (Cygwin[^] based) grep not to match various files. Once again I remembered the old Unix ‘file’ tool that attempts to detect what format a file is and whist searching for a Windows port of that I came across the UnxUtils project on SourceForge. This looked to be a native port of the most commonly found Unix tools such as awk, sed, grep, egrep etc. It didn’t contain a version of ‘file’ but I got one of those from somewhere else. Anyway, I unzipped the UnxUtils package, added it to my PATH, and then waited for a moment to use them.

That moment came sooner than I expected. I needed to analyse our on disk trade repository and provide some idea of its size and daily growth. My instinct this time was somehow to use Excel as that is pretty much what the financial world uses for all its analysis and reporting. But we were running an old version that still has the 65K row limit and our repository was well over 100K files per day. So I Googled for an Awk based solution which (adapted for Windows) initially became:-

c:\> dir /s *.trade | gawk “{ sum += $3 } END { print sum }”

This was pretty close, but I needed to strip the chod out of the dir listing for the daily comparison. Also, recursively searching the repository whilst I was getting this right was going to be way too slow so I captured the output from dir once and applied a sprinkling of Sed to remove the rubbish:-

c:\> dir /s *.trade > filelist.txt
c:\> sed –n /^[0-9][0-9]/p < filelist.txt | gawk “{ sum += $3 } END { print sum }”

And that was it for the current repository size. The day-on-day growth just involved extra calls to Sort and Diff to trim out the common items - it was a textbook example of pipeline processing!

Of course I got stuck for ages not knowing that Sed has a limited regex capability or that the ‘END’ in the gawk script had to be in uppercase. But it inspired me to put the effort in to learn the tools properly and so I popped onto Amazon and found a second hand copy of Sed & Awk (2nd Edition) by Dougherty & Robbins for under £10 and also picked up the O’Reilly Sed & Awk Pocket Reference for a couple of quid to leave in my drawer at work.

In the last couple of months I’ve read the majority of the book so that I feel more equipped to notice when this pair of tools would be a better choice. One of the most useful features I discovered about Sed (that I don’t believe VS can do easily) is to apply a transformation to repeating sections of a file (which are specified by an ‘address range’, i.e. a pair of regex’s that denote the start and end of the region). I’ve also found it easier to use Awk to generate, say, SQL statements where I have the output from a query that then needs to be turned back into a bunch of stored procedure calls with the previous result set as the parameters. This is doable in VS with regex based search & replacing but it’s probably not the most efficient use of my time.

The only real problem I have is that the Sed regex syntax is a subset of the normal POSIX one, so some common features are not available. It’s also another regex syntax to learn (along with Visual Studio’s own warped variation).

By this point I suspect that I’m preaching to the converted and there is much nodding of heads. There has also probably been the quiet whispering of “the right tool for the job” and all that. So yes, I’m incredibly late to the party, but I’m most pleased with my re-acquaintance. I don’t expect to ever be writing multi-line scripts but even if it makes the one-liners easier it will have been ten quid well spent.

[*] I search and analyze much larger log files every day with BareTail, BareGrep and LogParser. It’s just that I don’t need to modify them or generate other files from them.

[#] When it came to searching my entire (400 MB) hard disk it was actually quicker to close Windows 3.1, reboot into Linux (0.91pl15 IIRC) and use find + grep, save the results and then reboot back into 16-bit Windows than it was to do the search directly in Windows (or via a DOS prompt) in the first place.

[^] I want my tools to behave naturally on Windows - I have no desire to simulate a Unix environment.

My ACCU 2011 Session Proposal - Using xUnit As a Swiss Army Testing Toolkit

2010-09-27T23:12:00.001+01:00

I’ve decided to be brave and submit a session proposal for next year’s ACCU Conference. I know if I get accepted that means I’ll have to curtail the relentless late nights in the bar – at least until I’ve given my presentation. Anyway, the catalyst for my proposal was my blog back in March titled Integration Testing with NUnit. Not long after posting that I watched Paul Grenyer give a presentation to the London branch of the ACCU that touched on similar ground. My session is intended to take the idea further still and cover use of the xUnit style in other areas where perhaps custom test scripts and tools might have been written such as API exploration and verifying external systems.

Session Summary

Title: Using xUnit As a Swiss Army Testing Toolkit
Type: Tutorial/Case Study
Duration: 90 minutes
Speaker: Chris Oldwood

Speaker Biography

Chris started out as a bedroom coder in the 80s writing assembler on 8-bit micros; these days it’s C++ and C# on Windows in big plush corporate offices. His career has covered both shrink wrapped applications and in-house systems with the past 5 years focusing on grid-based distributed systems in the Finance industry. When not attached to a keyboard and screen he has a wife and four children to entertain, dips his toe in the local swimming pool and provides the commentary for the annual Godmanchester Gala Day Duck Race.

Session Description

Modern Unit Testing practices act as a conduit for improved software designs that are more amenable to change and can be easily backed by automation for fast feedback on quality assurance. The necessity of reducing external dependencies forces us to design our modules with minimum coupling which can then be leveraged both at the module, component and subsystem levels in our testing. As we start to integrate our units into larger blocks and interface our resulting components with external systems we find ourselves switching nomenclature as we progress from Unit to Integration testing. But is a change in mindset and tooling really required?

The xUnit testing framework is commonly perceived as an aid to Unit Testing but the constraints that it imposes on the architecture mean that it is an excellent mechanism for invoking arbitrary code in a restricted context. Tests can be partitioned by categorisation at the test and fixture level and through physical packaging leading to a flexible test code structure. Throw in its huge popularity and you have a simplified learning curve for expressing more that just unit tests.

Using scenarios from his current system Chris aims to show how you can use a similar format and tooling for unit, component and integration level tests; albeit with a few liberties taken to work around the inherent differences with each methodology.

Implementing Constants & Enumerations in a Database

2010-09-10T07:46:00.001+01:00

How often have you come across this kind of code in a stored procedure:-

select abc
from xyz
where Status = 3

…or if you’re lucky (or perhaps unlucky if the comment is out-of-date):-

select abc
from xyz
where Status = 3 --Failed

One of the downsides of SQL (at least with T-SQL) is a lack of support for constants, and by extension enumerations.

No Magic Numbers

It’s long been a best practice that you avoid the use of so called Magic Numbers like the ‘3’ in the examples above in favour if some more symbolic name. This aids both readability and maintainability because you don’t have to go and lookup what the number represents (in the case of an enumeration) and if the enumeration value needs to change you only have to change the relationship with the symbol and not the code where the symbol is referenced.

Local Constants

If you only need a constant for use within a single stored procedure then you can just use a normal variable:-

declare @Failed int
set @Failed = 3
. . .
select abc
from xyz
where Status = @Failed

This will make the latter SQL more readable and hopefully more maintainable - it should be far easier to grep for “Failed” than “3” if the numeric value does need to change[*].

Global Constants

Sadly the need for one-off constants is pretty rare. What is more likely is the need for a globally defined constant so that we don’t violate the DRY/SPOT principle by defining local constants in every procedure where it’s used. The best solution I could come up with[#] was to use a User Defined Function:-

create function Failed()
returns int
as
return 3
go

This means that you can use it like so:-

select abc
from xyz
where Status = Failed()

However it’s not always possible to use a UDF in the same place that you can a constant or variable (I’ve yet to read up on why) and so sometimes in a stored procedure you still need to use a local variable as well:-

declare @Failed int
set @Failed = Failed()
. . .
exec SomeOtherProc @Status = @Failed

Lack of Namespaces

Although some databases have support for schemas, they are often quite simplistic and not designed for nesting like you would in C++ and C#. This means that to avoid conflicts we need to resort to the old style of using some form of prefix or suffix on the name:-

select abc
from xyz
where Status = TaskStatus_Failed()

This becomes a little less distracting with enumerations as in C# you are forced to prefix the enumeration value with it’s type name anyway (unlike C++, although may people nest C++ enumerations within a separate struct to get the same effect) so it’s not too unnatural to read. Of course you could still create a separate namespace, called say ‘constants’, in which you put all these UDFs if you felt that it helped matters.

An Alternative - Lookup Tables

The main thing I like about the UDF solution is that the function name means the symbol appears as code rather than data. One alternative I’ve seen is to use a small lookup table that has the integer value and a string for the symbol. This makes the numbers less ‘magic’ but it means you have invoke a query to get the enumeration value:-

declare @Failed int
select @Failed = EnumValue from TaskStatusEnum
where EnumSymbol = ‘Failed’

I’ll be honest and admit that I’ve done no performance comparisons between this and the UDF idea. It also still feels to me like you’re expressing the enumeration with data rather than code though but that’s probably nonsense.

Client-Side Use

The other benefit of using a UDF is that you can also use it client side should you happen to need to embed queries within your code so your magic numbers don’t leak further afield. We use proper enumerations in our C# code that are kept in sync manually but tested automatically as part of a build. Ideally we should generate the UDF’s and C# enum’s from a single source, but for the moment the automated tests acts as a nice safety net.

[*] Of course in the real world you would never rely on the consistent use constants like this and would actually search for all references to the table and/or column name instead, but hey this is theory :-)

[#] Remembering that I’m not a SQL expert by any stretch of the imagination.

What’s Exceptional Depends on Context

2010-09-08T18:12:00.001+01:00

The latest edition of Overload (one of the journals from the ACCU) has another episode of Matthew Wilson’s excellent Quality Matters column. The latest topic is exception handling in which he makes a reference to The Practice of Programming by Kernighan & Pike – a book which I’ve only ever skimmed so far. Hence I thought I should to read the relevant chapter and somewhat surprisingly I found the following statement:-

“Exceptions are often overused. Because they distort the flow of control, they can lead to convoluted constructions that are prone to bugs. It is hardly exceptional to fail to open a file; generating an exception in this case strikes us as over-engineering. Exceptions are best reserved for truly unexpected events, such as file systems filling up or floating-point errors.”

My gut reaction to that was “No it’s not – it depends on context”…

External Resources Always Fail

Messer’s Kernighan & Pike are obviously far more knowledgeable and experienced than me so why did they make such a bold statement? Generalising somewhat, I presume that they were suggesting that any interaction with an external resource should be handled with due care as you should expect it to fail – after all we always encounter problems with files, sockets, databases etc. And as a general rule this makes perfect sense.

Full Environmental Control

But… In the world of in-house server-side software you are often completely in control of the entire environment in which your code runs. Even for in-house desktop software the environment may be a little more hostile but you can still ensure certain invariants. Taking the “opening a file” example again and given the following conditions, would failing to open a file still be considered unexceptional?

The file exists (say we just created it and control its lifetime)
The permissions on the folder are correct (our server installation script set them)
The server has plenty of internal resources (memory, handles etc)
The process has not suffered any unrecoverable errors

In this situation I would be on the side of it being exceptional. In my experience failures under these kinds of scenarios usually point to a system configuration error [1 or 2], which means it would never have worked, or the server/process itself has become unstable [3 or 4] due to some earlier (and hence unexpected) issue.

Recoverability

Of course the issue is perhaps moot as all errors fall into two categories – recoverable and unrecoverable. In the scenario described above the recovery is effectively manual[*] – fix the system configuration or restart the process/server. In the real world I can’t see how you could justify the cost of designing a system to automatically cope with some/all of those potential edge cases[+]. Even the last one (process instability) can be hard to handle fully via process recycling because too much badly written code masks nasty errors like Out Of Memory or Access Violations by sinking everything so you don’t get to react to it.

In contrast the times where I can see you might be able to recover from failing to open a file are when the choice of file is controlled by the user, such as in a desktop application, or when the contents of the file can be considered optional, such as user settings.

Applying the issue wider afield to databases and remote services we end up at clustering/replication as the obvious recovery option. Even so, given the general reliability of in-house networks and (non-blade) hardware these days, I would still consider it pretty exceptional for a failure to occur. By-and-large misconfiguration is the most common reason I see for failures involving resources.

Do Or Do Not (Or There May Be a Try)

The chapter of The Practice of Programming in question was about designing interfaces so do we assume that this message is aimed more at the writers of libraries rather than applications? Given that you don’t know the context in which your code is being called do you have to choose between error codes or exceptions? Would your interfaces be better or worse if you somehow supported both so that the caller could decide? I’m sure there are many developers out there right now wincing as they stare at their legacy COM heavy codebases that have a mixture of wrappers generated by Visual C++’s #import feature where some methods throw and others return HRESULTs, some have a Chk prefix and others don’t. What I’m suggesting has more moderation to it…

In the .Net world there is a common pattern where methods by default succeed or throw an exception and a parallel set of methods named TryXxx that return a bool and use an output parameter instead. The latter ‘overloads’ allow you to avoid the cost[#] of handling an exception such as when parsing numbers and DateTimes that you can expect to be iffy (format wise) but they will still throw other exceptions which are orthogonal to the task such as Out of Memory.

Was It Just a Poke At Java?

Or maybe I’m just being too literal? The book was written over 10 years ago (back when exceptions where still not exactly mainstream) and it follows an example in Java so perhaps they were just having a pop at the choice Java had made. After all, at that time Java was still popular as a client-side technology – even outside the browser.

The fact that people like Matthew Wilson are still regurgitating this argument a decade later means there is still much to learn. I think the TryXxx pattern is a nice compromise as you get the flexibility of handling a domain failure without resorting to an unnecessary try/catch block and you also don’t accidentally mask orthogonal failures by ignoring the return code or catching an exception of too generic a type. I wonder if this also satisfies Kernighan & Pike or do they find it muddies the waters even further and still stand by their original statement?

[*] I say manual because what normally seems to happen is that the system monitoring software gets bombarded with errors and someone has to make sense of them. Once that’s done it usually involves bouncing something…

[+] In the embedded world, where lives may be at risk, there are forces way outside the normal risk/reward trade offs of an in-house corporate system.

[#] I’m referring to both the performance and maintenance costs of using exceptions.

IDisposable’s Should Assert In Their Finalizer

2010-08-18T08:02:00.002+01:00

Paradoxically[*] I’m finding that C# makes resource management much harder than C++. In C++ life is so much easier because you always have to deal with resource management and RAII is the tool that makes it a no-brainer – you either use stack based variables or heap allocated variables and a smart-pointer class such as scoped_ptr/shared_ptr. C# on the other hand makes resource management ‘optional’ through the use of the IDisposable interface[%].

What Is and Isn’t Disposable?

And that is my first problem, you don’t always know what does and doesn’t need disposing – you have to go and find out. Of course IntelliSense can help you out here a little but it still means checking every object to see if it has a Dispose() method[#]. The other alternative is to hope you get it right and rely on a static analysis tool like FxCop to point out those ‘occasional’ mistakes. Personally I’ve yet to get anything really useful out of FxCop outside the usual stylistic faux pas’ which seems to be more the domain of StyleCop.

IDisposable is Viral

OK, that’s a little harsh on FxCop as I’m still learning to use it effectively. But after years of using C++ tools like Lint and BoundsChecker to watch my back I was more than a little disappointed. It does seem to point out if I aggregate a type that needs disposing and I haven’t implemented the Dispose pattern, which is nice. However Dispose() is like ‘const correctness’ in C++ - it’s viral - once you start correctly applying IDisposable to your types it then mushrooms and you now need to fix the types that aggregated those and so on.

Should Interfaces Inherit IDisposable?

This leads me to my first question – should interfaces inherit from IDisposable if you know that at least one implementation needs it? On the face of it the answer seems to be no as disposing is purely an implementation detail; but the whole point of interfaces is to avoid ‘programming to an implementation’. If the answer is no then the moment you cast down to an interface you hide the disposing behaviour. COM essentially has to deal with the same problem and its solution is to make AddRef() and Release() a fundamental requirement of every interface. Of course C# has RTTI built in through the use of the ‘as’ and ‘is’ keywords and so you can always attempt a cast to IDisposable from any other interface. However surely this way lies madness as your code would be littered with seemingly random ‘usings’ just in case an implementation later needed it. Here’s an example where this issues has cropped up most often to date…

We are using the Gateway Pattern extensively in our middle tier services to talk to other systems and so the gateway implementation often requires a WCF proxy which requires calling Close() (or a Socket, database connection etc). So, do I expose the disposing requirement through the gateway interface?

public interface ITradeGateway : IDisposable
{
. . .
}

public class TheirBackEnd : ITradeGateway
{
. . .
}

…or just implement IDisposable on the concrete type?

public interface ITradeGateway
{
. . .
}

public class TheirBackEnd : ITradeGateway, IDisposable
{
. . .
}

In my mind the first is more ‘discoverable’ than the second and it gives any static code analysis tools a fighting chance in pointing out where you might have forgotten to call Dispose(). Some might argue that at the point of creation you know the answer anyway as you have the concrete type so why does it matter? Well, I tend to wrap the creation of these kinds of services behind a factory method that returns the object via the intended interface so that you are not inclined to rely on the concrete type unnecessarily:-

public static TradeGatewayFactory
{
    public ITradeGateway CreateTradeGateway()
    {
        return new TheirBackEnd();
    }
}

Most of our factory methods are not quite this simple as they tend to take a configuration object that further controls the construction so that they can hide whether the ‘service’ is hosted in-proc (which is useful for testing and debugging) or remotely via a proxy[+].

Does It Matter If You Forget?

I mostly work on distributed systems where scalability and reliability are major concerns and perhaps I’m being overly pessimistic about the memory consumption of my services but I think it’s important that for certain kinds of resources that their lifetime is managed optimally[$]. At the moment I’m dealing with a managed wrapper over an in-house native library that is used to manipulate the key custom data container that the organisation traffics in. The underlying native implementation uses reference-counted smart pointers for efficiency and naturally this has leaked out into the managed wrappers so that many aspects of the API return objects that implement IDisposable. In fact it’s all too easy to use one of the helper methods (such as an index property) and find yourself leaking a temporary that you didn’t know about and bang goes your careful attempts to control the lifetime of the container, e.g.

// Good.
using (var section = container.GetSection(“Wibble”))
{
var entry = section.Value;
. . .
}

// Leaky.
var entry = container[“Wibble”].Value;
. . .

I definitely think this scenario should be picked up by a static analysis tool and if I’ve read the blurb on FxCop 10.0 (that ships with VS2010) correctly then I have high hopes it will watch more of my back.

Assert In The Finalizer

So can we do anything else than rely on tooling? I think we can and that would be to put a Debug.Assert in the Finalizer - after all if the object is being consumed correctly then you should honour the contract and call Dispose() at the relevant point in time. I think it’s safe to say that the Garbage Collector does a pretty good job of hiding most mistakes by running frequently enough, but as Raymond Chen points out on his blog last week (which is “CLR Week”) - you should not rely on the Garbage Collector running at all.

For my own types that don’t manage any native resources themselves it could be implemented like this:-

public class ResourceManager : IDisposable
{
#ifdef DEBUG
    ~ResourceManager()
    {
        Debug.Assert(false);
    }
#endif
    . . .
    public void Dispose()
    {
        m_member.Dispose();

#ifdef DEBUG
        GC.SuppressFinalize(this);
#endif
    }
}

So basically we’re saying that if Dispose() is not invoked, then, when a Garbage Collection does finally occur at least we’ll know we forgot to do it. Sadly we can’t rely on being able to inspect the members in the debugger to work out which instance of an object was forgotten because finalizers can be run in any order; but maybe we’ll get lucky.

If you start from a clean slate then you can write a unit or integration test that forces a full garbage collection right after exercising your code to ensure any errant finalizers run and get instant feedback about your mistake:-

[Test]
public void should_not_leak_resources()
{
    var consumer = new ResourceConsumer();

    consumer.consumeResources();

    GC.Collect();
    GC.WaitForPendingFinalizers();
}

I’ll be honest and point out that I’ve put off actually trying this out in earnest until I have had time to investigate how to tap into the Asserting mechanism so that I can avoid hanging the test runner with a message box unless I’m running under the debugger. I’ve done this plenty of times with the Debug MS CRT (_CrtSetReportHook) so I’m sure there must be a way (I’ve only scratched the surface of the TraceListener class but I’m guessing it plays a part).

Debug Builds – Not The Done Thing?

Back in an earlier post Debug & Release Database Schemas I suggested there must be times when a debug build is used in the C#/.Net world. Unlike the C++ world, this beast does not seem to be at all prevalent. In fact I’ve yet to come across any 3rd party (or in-house teams) promoting a debug build. Visual Studio and C# supports the concept, but I wonder if teams only expect it to be used for internal testing? Jeffrey Richter briefly mentioned “Managed Debugging Assistants” in his book CLR via C# but I’ve yet to read up on how you use them effectively, i.e. tap into the mechanism programmatically so that I can log these failures whenever the services are running unattended; not just under the debugger.

[*] It’s not really a paradox as 15 years C++ vs 1 year C# isn’t exactly a fair comparison.

[%] Optional in the sense that not every type requires it.

[#] or Close() in the case of the thread synchronization types which is a nice inconsistency.

[+] I’m still not convinced by the use of an off-the-shelf Inversion of Control (IoC) framework as it only seems to save the odd line or two of code at the expense of managing another 3rd party dependency. I also much prefer creating immutable types that are fully constructed via the ctor than providing multi-phase construction via mutators which IoC frameworks seem to require. Maybe I just don’t work on the kind of systems they’re aimed at?

[$] The obvious question here I suppose is “Why are you using C# then?”. And the answer [for now] is “because we are”. I was expecting this to to scale-up further that it has, but we can still scale-out further if needs be.

Stored Procedures Are About More Than Just Performance

2010-08-09T19:15:00.001+01:00

A colleague who is new to the team and working on a new GUI project asked me whether we should be using an ORM tool like Entity Framework for the impending GUI Data Access Layer. Our existing services all invoke stored procedures via manually[*] crafted ADO .Net code which is largely due to our inexperience with ORM tools, but also because we hardly have any DB code so it’s not a burden – unlike what the GUI is likely to need. This in turn led to a question about why we use stored procedures in the first place when the performance of modern SQL query optimisers is such that the benefits are less considerable. What I found interesting about this question was that we had never chose to communicate with the database through stored procedures for performance reasons; on the contrary it has always been about creating an abstraction over the database to ease maintenance.

OO Data Models

I have been practising OO for long enough that I naturally see rows in a table as instances of objects and stored procedures as their methods. You can use OO techniques with procedural languages like C and SQL, it just takes a little artistic license in the naming scheme to work around some of the inherent limitations - although simulating virtual functions is a little trickier :-). There are a number of patterns available to deal with the ORM problem (the oldest ones I know of were in James Rumbaugh’s book Object-Orientated Modelling and Design from 1991) and in particular the problem of mapping polymorphic types to tables but I find the vast majority of cases fall into the simple Table == Type category.

A classic example would be a table of customers with ‘methods’ to add a new customer or find an existing one by its unique ID:-

create table Customer
Id int not null,
Name varchar(100) not null

exec Customer_Insert 1234, ‘name’
exec Customer_FindById 1234

The most common scheme I’ve seen is to just prefix procedures with the ‘type’ name, which can seem a little odd to long term SQL’ists and so sometimes it’s feels more natural to them to treat the prefix as a namespace and repeat the ‘type’ in the ‘method’ name:-

exec Customer_InsertCustomer 1234, ‘name’
exec Customer_FindCustomerById 123

This prefix also has the nice side-effect of causing related procedures to be grouped together within the file-system/VCS and database IDEs like SQL Server Management Studio (SSMS).

Of course for simple INSERTs and SELECTs this probably feels like overkill as all you’ve done is to wrap a one-line query and make more work for yourself. But where it starts to pay off is when your one-liner turns into a two or three-liner, requires tuning because the query optimiser needs help or you want to weave in other aspects such as debug code (which I covered recently in Debug & Release Database Schemas).

Where’s the Public Interface?

In another post about database development, xUnit Style Database Unit Testing, I briefly referred to the ‘Public Interface’ of the database and what might constitute it. Obviously in SQL there is no direct equivalent of ‘public’ and ‘private’ that you can use to restrict access to objects on a logical level, but there is a real security mechanism instead that you can leverage. By following the “Principle of Least Privilege” and only granting access (e.g. EXECUTE permissions) on those objects that the client needs to use you can enforce a real public/private barrier.

Sadly this is not altogether intuitive from a maintenance perspective and can easily lead to the erroneous exposure of private procedures without some other convention to highlight the intended accessibility of the code. Even if you also follow the principle that you only write unit tests that target the public interface, it’s still not enough. One option would be to adorn the names of private procedures with another prefix or suffix, e.g.

exec __Customer_VerifyState @current, @intended
exec Customer_VerifyState_private @current, @intended
exec Customer_internal_VerifyState @current, @intended

Using a prefix disrupts the nice grouping in the IDE/file-system, whilst a suffix may be invisible if it’s too subtle. Placing it between the ‘Type’ and ‘Method’ names adds a little speed bump that could provide enough of a clue without adversely affecting readability, although it feels more intrusive than the suffix. If you are purely relying on naming conventions then a more intrusive adornment may be better as you can then use a tool like grep to find violations in your sever and client code.

The only other option I see available would be to use schemas, e.g.

exec private.Customer_VerifyState @current, @intended

I’ve not played much with schemas and on my current project we have already used them to distinguish between the core production code and other procedures that aid in unit testing (via a ‘test’ schema) or support utilities (via a ‘util’ schema). Our inexperience with schemas has also meant that we’ve run into a few minor permissioning issues with temporary tables and user-defined types that makes us a little wary of sprinkling them too liberally.

SQL Maintenance

All the teams I’ve worked in during the last dozen or so years have had developers whose main role is to maintain the database code. That doesn’t mean that they didn’t have other skills as well, but effectively they were hired to maintain the SQL side of the codebase. That also doesn’t mean that other developers did not contribute to the SQL code (although in some cases a more rigid structure was in place) as SQL is often a core skill, but the quality of SQL generated by the generalists may not be sufficiently high enough for the volume of data being manipulated.

I definitely put myself in the category of developers that can write basic SQL queries and stored procedures and can handle many table joins, but when it comes to concepts like nested sub-queries I start to question whether I’m writing an efficient query or not. I’ve also read around the subject enough to know that cursors have their place and it’s not usually in the kinds of queries I write so having a SQL expert around to turn my functional query into an efficient one really helps.

Of course the query optimisers built into SQL databases now are way more advanced that when I first started using them, but there still seems to be a need for SQL experts who can tweak a query to correct the optimisers bad judgment calls and give orders of magnitude performance gains, or add some hints to avoid deadlocks caused by lock escalation. This kind of maintenance is so much easier if the SQL is contained within stored procedures as you can leverage the faster deployment mechanism.

[*] Naturally much of the boiler-plate code has been refactored out by wrapping the underlying SQL classes and using Execute Around Method so that it has minimal excess baggage.

Where’s the PowerShell/Python/IYFSLH*?

2010-07-27T07:52:00.001+01:00

An obvious question to many might be “why are you still writing batch files when PowerShell superseded it years ago?”. Even VBScript is way more powerful than cmd.exe and then there’s everyone’s pretender to the sysadmin throne – Python (and by extension IronPython in the Windows world).

Not Corporate Policy, Again

I’ve been following PowerShell since it was called Monad, and IronPython too during its pre DLR days (especially after a pretty cool demo by Michael Foord at the ACCU Conference in 2008). Once again the problem in the corporate world is that Application Servers are often very tightly controlled and organisations generally don’t like teams jumping on every bandwagon when it has the potential to create conflicts and therefore downtime; I’m sure it was Raymond Chen[#] that said “If the solution starts with ‘First Install X’; then now you have two problems”. PowerShell, Python, Ruby, PERL etc. all involve installing runtimes and libraries which may themselves rely on other shared 3rd party components – DLL Hell is still a problem today. Given that many large organisations only recently (i.e. the last couple of years) migrated to XP on the desktop and Windows Server 2003 for their servers you can see why the uptake of PowerShell has been slow – it’s just not been readily available in this environment; plus there’s a mountain of legacy COM and VBScript which still does the job perfectly well.

Drive-By Programming

As a freelance developer I also feel I have the need to act responsibility which means I can’t go off writing code in whatever takes my fancy and then leave my client with a load of stuff they can’t support; of course some see this kind of behaviour as job security… For me to see a technology as worthy of production use means that there must be a critical mass of people (ideally within the team) who can support it. I’ve worked in a medium sized team where we prayed the two production PERL scripts didn’t fail because no one really knew what they did. I’ve also seen an intern write some production code during his 6 month rotation in PowerShell when there was little to no knowledge of it within the team. I raised the obvious question about who would be able to support it when his rotation ended which prompted the usual discussion about code reviews. It also prompted a deeper one about whether we should try and standardise on one ‘glue’ language going forward and where (in the development process) it would be useful to introduce it to allow time to gain the necessary skills for its eventual promotion to production status. One alternative is for the management to be made aware of the impending cost and stump up the training to acquire the critical mass; but training often seems to be the first to go when money gets tight.

Ok, that sounds overly dramatic I know because anyone can perform simple maintenance, such as replacing database connection strings, file-system paths, server names etc. But there is a very real problem there if something more serious goes wrong as it’s highly likely that there won’t be any tests in place to guide any significant development. You also have to factor in that full-time experienced developers often write this kind of code initially but as the team grows and a separate support team comes on board it is them that are faced with picking up this maintenance burden.

The Right Tool For the Job

We discussed right back at the start of my current project what we wanted to pick as our ‘auxiliary’ languages to make sure that we all had the same hymn sheet. The server and client side coding was largely to be done in C#, but we were also going to need a language to act as the ‘glue’ to bind all the disparate Batch Processing tools together; plus we thought we might need another language to aid in development & testing. One again PowerShell and Python seemed to be at the top of the list. But whereas on a previous project it was Python’s interop with COM and C++ that made it look attractive, it was PowerShell’s interop with .Net (due to the C# bias) that made it favourable this time and it was also part of the standard app server and desktop build now so we had no deployment concerns. Somewhat interestingly a few months later the department did a fact-finding mission to see how many people had Python skills as it seems that they wanted to promote the Python/IronPython approach to the development teams instead. I’m looking forward to seeing how this initiative pans out as we’ve only just started on this area of our system so we have very little to throw away.

Proving a Point

I fired up a command prompt on this 9 month old netbook which is fully patched to XP SP3 to knock up some PowerShell equivalents to the cmd shell examples I gave last time and what do you know… PowerShell isn’t installed!

[*] IYFSLH = Insert Your Favourite Scripting Language Here

[#] I’m sure it was him, but my Google Fu is unable to locate this quote.