xUnit Style Database Unit Testing

Like many small teams I find myself in the position of not only writing framework and application code but also occasionally for the back-end database. In most teams I’ve worked in there’s generally been a developer that tends to do far more of the core database work than the others, such as writing triggers, handling indexes (or is that indices :-) and optimising our poorly written stored procedures. During my last assignment I started getting more involved in the development process of the database code as I wanted to try and help streamline it to iron out some of the integration and deployment issues we were having. My thoughts weren’t exactly greeted with open arms by the DBAs, but in my new role I’m finding that I have an opportunity to test out my ideas as they are being taken on board by the team with an interest in developing them further.

The Traditional Development Process

Database development has always appeared to be treated differently to normal application development. With application code you have to write the source, compile it, test it and then deploy it to the application servers (let’s ignore dynamic languages for now). With SQL you can write the source, deploy it very simply and then test it. What’s even easier is that you can extract the source from the database, make a change and redeploy without even saving the file anywhere else. Of course just because you can work this way, doesn’t mean you should. Some of the issues I’ve seen previously involved code which was running on the database but was not in the Version Control System (VCS) and regressions caused by code being written to work with the current data set instead of to a “specification”.

In my current role I wanted to see if we could adopt a more formalised process for database development. Essentially I want to see if we can treat database code almost exactly the same way as the application code:-

• The VCS acts as the master for all source code - you always start from what’s in the repository not a database
• Unit tests exist for the public interfaces
• Automated running of the unit tests on the build machine
• Automated packaging of the scripts on the build machine ready for deployment
• Automated deployment of the database code to a test environment

The last three should be relatively simple – just a bit of scripting, but what does it mean to unit test database code? When considering stored procedures and user-defined functions the answer is straightforward because they map onto traditional unit testing of subroutines & functions. But what about triggers, views and constraints? Also how should the running of the test suite fit into the wider context of the whole database development process? Is a more structured development process too detrimental because fights against traditional practices and tooling?

The Public Interface

When unit testing object-orientated code it is often asked if you should test private methods. I don’t believe so because they are implementation details and should be tested only as a by-product of exercising the public interface. If they are not invoked via the public interface, then what purpose do they serve (Reflection and Active Objects notwithstanding)? Trying to test private methods also leads you to exposing them or writing hacks to facilitate this goal which only ends up leading to the need to refactor tests when the implementation changes which seems anathema to the cause.

So what constitutes the Public Interface of a database? Obviously Stored Procedures and User Defined Functions fall into this category, but what about tables? This depends on your access model. If you’re allowing your client code full access to your tables then your schema changes can directly affect your clients, whereas if you follow the model of all access via Stored Procedures then they’re not public. Views are also public as they’re often created with the very purpose of encapsulating the base tables they draw from.

We have chosen this latter model and have agreed that all implementation details such as Constraints and Triggers should only be tested as a by-product of testing the Stored Procedures that drive them. For example, if the specification says that the SubmitOrder() procedure should default the order status to ‘Pending’, then it matters not whether that happens explicitly in the sp code itself, by a default value on the table or via an insert trigger – they all achieve the same externally visible goal.

The xUnit Model

With the decision made about what to test, the next step is finding a way to do it. The most popular unit testing model these days seems to be xUnit. In an OO language you implement a test as a class method that includes an Assert to perform a comparison after invoking the required behaviour. Each test only tests one thing, is not dependent on any other, should be free from side-effects and is carefully named to reflect the intended behaviour (nay specification). The model also supports the ability to define common pre and post-test code in methods called SetUp() and TearDown() at both the Test and Fixture (set of related tests) levels to remove repetitive code and make the test intent clearer.

So how can we map these concepts onto database code? The side-effect free behaviour is a no-brainer as we can just use transactions. Databases do not support a declarative model that allows you to use Attributes to indicate tests, fixtures etc, but they do have their own internal tables that can be used in a reflection-like manner to invoke procedures based on their names. For example you could use a prefix such as “test” or “utsp_” to separate unit test code from production code. Then to distinguish between the actual tests and the initialisation and cleanup functions you could use “<sp>_Should<expectation>” as an indicator of a test, and “<sp>_SetUp” and “<sp>_TearDown” for the other two. The fact that all test procedures take no arguments and return no values means that they can be invoked easily by reflection.

This leads to a SQL unit test script structured something like so:-

CREATE PROCEDURE testSubmitOrder_SetUp
AS
    INSERT Customer(CustomerId, CustomerName)
        VALUES(1, ‘TestCustomer’)
GO

CREATE PROCEDURE testSubmitOrder_TearDown
    . . .
GO

CREATE PROCEDURE testSubmitOrder_ShouldSetStatusPending
AS
    DECLARE @OrderId int
    EXEC SubmitOrder ‘TestCustomer’, ‘widget’, 2,
        @OrderId OUTPUT
    IF NOT EXIST (SELECT 1 FROM Order
                  WHERE OrderId = @OrderId
                  AND OrderStatus = ‘Pending’)
        RAISERROR (‘Order should be inserted with Pending status’, 16, 1)
GO

CREATE PROCEDURE testSubmitOrder_ShouldUpdateInventory
    . . .
GO

This leads you to a physical layout where one .sql file contains all the test code for a single Stored Procedure, much like a single source file containing all the tests for a class.

I’ll be honest and admit that the details of mocking and supporting a Fluent Interface style for asserting far exceeds my knowledge of SQL and databases. I know (because I’ve done it by mistake in the past!) that procedures of the same name can belong to different users, so perhaps this mechanism can be exploited? For the moment the style above serves our purpose.

Executing The Test Suite

My initial thoughts about the way that the test suite would be executed were somewhat grandiose. I thought that the Test Fixture could be used to only load the database objects needed to run the tests and therefore also provide a mechanism though which mocks could be injected. I fear that this is a maintenance nightmare in the making if the dependencies are not automated. The simpler option is to recreate the database from scratch and then run the unit test scripts on that. Because each unit test is responsible for its own test data we can avoid taking the “restore” option which ensures that we are guaranteed a truly clean database - one free from developer detritus.

The wonderful knock-on effect of formulating this process is that it can be executed on the build machine as part of the Continuous Integration process. SQL Server Express is just the ticket for the database role as it gives us all the features of it’s bigger brother, but can easily cope with the trivial workload we throw at it. This also feeds into the daily database development cycle as I can use a local installation of SQL Server Express as my sandbox where I only require the schema, objects and minimal test data because I’m focusing on the functional aspects of the problem, not performance testing.

This technique also gives us a path to the goal of continuous integration testing of database and application code and then continuous deployment to the system test environment. Here we can automatically rebuild a database from scratch, populate it with real data, deploy the code to the application servers and push the big green button. If this is done during the night, then by the morning we will have exercised a significant number of code paths and have a high degree of confidence that the environment is stable and ready for further integration/system testing.

Scalability

One of the conventions in unit testing is to reduce your dependencies so that the tests run really quickly, then you have no excuse not to run them at every opportunity. Unfortunately, due to the nature of the toolchain (an out-of-process database) running tests is not as instantaneous as with application code. Running the individual tests isn’t too expensive, but rebuilding the database from scratch can take a while so I would keep that for when you do a “Clean Build” prior to check-in.

TSQLUnit

As I mentioned in a previous post “Company-Wide IRC Style Chat & Messaging” I first started thinking about this some years ago, and a colleague at the time pointed me to TSQLUnit. I had a very brief look back then and it seemed pretty useful as it followed a similar model. I haven’t had time to go back and look at it in more detail. None of what I’ve said is rocket science though, if anything the hardest part is writing the Windows Batch files to trawl the VCS repository and invoke the relevant .sql scripts – a whole “FOR /R %%I” loop with a “SQLCMD –i %%I” body. Even PowerShell seems overkill for this job :-)

 

Phew, I didn’t expect this post to turn into War and Peace! At the moment this is working out really well for my current team, but I’m aware that as a greenfield project we currently don’t have a very large schema or number of objects. However, it’s going to be a large system in the end so it will be really interesting to see how all this scales up.

Coders at Work – Jamie Zawinski

I’ll be perfectly honest and admit that until Joel Spolsky’s “Duct Tape Programmer” blog, I’d never heard of Jamie Zawinski. Whilst reading Ayende Rahien’s blog the other day I find that he was also attributed with that memorable quote about Regular Expressions:-

Some people, when confronted with a problem, think "I know, I’ll use regular expressions." Now they have two problems.

Luckily my mum, ever in search of something to buy me for my birthday, checked my Amazon wish list and pulled an item from it - the book Coders at Work by Peter Seibel. The blurb states that this is a collection of interviews with some of the top programmers of our time. Now I consider myself to be pretty well read and yet I could only tell you who 4 of the 15 interviewees were, and one of those was only because his name came up at the Stack Overflow Dev Day last October. For the record Donald Knuth, Joe Armstrong & Simon Peyton Jones are the names I immediately recognised and Peter Norvig was the one name-checked during the Dev Day. Anyway I’m sure looking forward to finding out who these other “prominent” programmers are and maybe I’ll be able to work out why their names didn’t register.

My impression of Jamie from the blog posts by Joel Spolsky and Uncle Bob wasn’t at all favourable. I felt he was portrayed as a reckless hacker who paid little attention to the maintenance aspects of software development. I had a vision of a loner coder sat in an Ivory Tower churning out Netscape. Ok, maybe I’m exaggerating now :-) Of course soon into the book I realised this picture was far from the truth, in fact the more I read the more I noticed how much I have in common with him…

I started out working on shrink-wrap software and have since spent more time working in the enterprise arena. As Jamie points out these two worlds are very different and that can have a significant knock-on effect on the way that software is developed. In the dog-eat-dog shrink-wrap world your customer likely has a number of choices and if you don’t deliver someone else will. In stark contrast is the closed world of in-house software where your customer has only one choice – you - they can’t reasonably go anywhere else. In Jamie’s words:-

I know it’s kind of a cliché but it comes back to worse is better. If you spend the time to build the perfect framework…release 1.0 is going to take you three years to ship and your competitor is going to ship their 1.0 in six months and now you’re out of the game. You never shipped your 1.0 because someone else ate your lunch. Your competitor’s six-month 1.0 has crap code and they’re going to have to rewrite it in two years but, guess what: they can rewrite it because you don’t have a job anymore.

These statements put his later comments about unit testing (at Netscape) into a whole different light. I’m sure Uncle Bob would cite Yoda and use the quote “No! No different! Only different in your mind”, and in theory he is probably right. After my recent post “My [Unit] Testing Epiphany” it would be churlish of me to disagree entirely with Bob, but I can also see where Jamie is coming from.

I also agree with his sentiments about education. It’s extremely rare these days to come across a programmer that didn’t go to University and get a degree in Computer Science or some form of Engineering (I have a degree in Electronic System Engineering - but I sucked big time on the analogue side!). In the couple of cases I have come across where a decent candidate lacked a degree they fitted the mould of the classic 80’s bedroom coder and have themselves described how hard it is to be noticed when you don’t have a degree on your CV. I rated both these candidates highly for the type of work I do because they come from a time where resource management needed careful consideration. In the interview (pp 47) Siebel questions whether we can leave those days behind in favour of a more abstract view of resource management. Given the current attitude to Scaling-Out (more hardware) instead of Scaling-Up (optimised code) it seems that way, but as the need to reduce power consumption within the data centres (and therefore the cloud) becomes more important we may see greater importance placed on those skills.

The one part of the interview that did surprise me was his attitude towards Design Patterns. He said:-

Then there was another book that everybody thought was the greatest thing ever in that same period – Design Patterns - which I just thought was crap. It was just like, programming via cut and paste…Then in meetings they’d be tossing around all this terminology they got out of that book.

I honestly think that he has completely missed the point (or maybe I’m misunderstanding his answer). I’m sure that some developers do try and design software in a cookie cutter fashion and attempt to use every one of the GoF patterns, but by-and-large the single biggest contribution they have made is the introduction of a common vocabulary. They have also been instrumental in promoting the notion of “Program to an interface, not an implementation”. Maybe he uses common terms like Composite, Factory, Mediator, Proxy etc but doesn’t attribute their wide acceptance or visual connotations to the GoF. Fair enough, it doesn’t take a rocket scientist to guess what each of those terms mean, but there are a number of others that are less obvious and yet are also incredibly common, such as Double-Checked Lock, Scoped Guard, Monitor, Context Object and Data Mapper. I’d sure be interested to know how he avoids this aspect of Patterns.

I’ve only read one this one chapter so far but it has achieved it’s job of painting a far richer picture of the man that ended up becoming the centre of a row about software quality. It has also reminded me that there are different forces driving the software development process, like shrink-wrap vs enterprise and application vs framework, which pretty much guarantees that one size never fits all.

Process Explorer Failed to Display DLLs or Handles

If this was Mark Russinovich he would probably have titled this post “The Curious Case of Process Explorer…”, but I’m not him and it’s not nearly as interesting or challenging as the kinds of issues he blogs about…

The other day I was investigating an issue where our new C# based grid service was failing to run on any engines because of what appeared to be an issue with assembly paths. As a n00b to .Net and having done a little reading into AppDomains and how assembly paths and native paths are used I thought I’d have a look at the grid engine surrogate process with Process Explorer to inspect its Environment Block and also what DLLs it already has loaded. This was on a 64-bit server running Windows Server 2003. After firing up Process Explorer I duly opened the lower pain and switched it to DLL mode only to be presented with an empty list. I humoured myself by checking the sizes of the columns and switching to Handle mode, but the pane was definitely empty. I even clicked on a few other processes like the svchosts’s, but the result was the always the same - nought.

Then, purely by accident, I clicked on explorer.exe and it worked. Realising that this was one of the processes running in my Windows session I added the “Username” column to the top pane and sure enough the only processes I could inspect where my own. Most of the machines I’ve dealt with in the past have been 32-bit boxes except for a few Big Iron 64-bit application servers. Not sure If I’d run into a limitation of Process Explorer on 64-bit Windows, I decided to go old school and run the command-line based Handle tool instead. This time I got a more interesting result:-

Error loading driver: Access is denied

I thought they had granted me admin rights on the box, so again I thought I’d humour myself and check by running:-

C:\> NET LOCALGROUP Administrators

Sure enough I was an administrator. Time to hit the Search Engines…

I didn’t find anything that exactly matched my issue with Process Explorer (PE), but there were some posts about the Handle error message. One post said to check whether you have the SeLoadDriverPrivilege granted in any process. I checked PE itself, and no I didn’t have this in my token. The post then went on to describe how you can check if you should have this privilege by using secpol. I did and the Administrators group is indeed granted this right.

Casting my search wider I started searching for info on SeLoadDriverPrivilege in the hope that I might discover where this has affected other similar low-level diagnostic tools. What I found was lots of discussion about rootkits and other malware. Given the environment I was working in I found this scenario extremely unlikely, but at the same time I could imagine how the security software that was running on the server could also be using similar techniques exactly to stop this kind of infection.

Fortunately along the way I came across another post that mentioned running Process Explorer under the SYSTEM account – the post was actually about Vista and UAC, but the idea seemed worth a go. To do this you can use another Sysinternals tool “psexec” with the “–s” and “–i” switches. I must confess that in all the years I’ve been using psexec I’ve never once needed to use these switches. As the command-line help will tell you, –s runs the process under the SYSTEM account and –i allows it to interact with the desktop. You also need to remember to fully qualify the path to the child process you’re invoking if it’s not going to be on the SYSTEM account PATH, e.g.

C:\>C:\Tools\psexec –s –i C:\Tools\procexp.exe

Lo and behold it did the trick, I could see everything again in Process Explorer. Naturally I’d like to know if the whole SeLoadDriverPrivilege thing was a red herring or not, and if it isn’t, get some closure on why the privilege is missing in the first place – now that’s the kind of job Mark Russinovich would tackle…

My [Unit] Testing Epiphany

In my recent post “TFD vs TLD – What’s the Definition of ‘Later’?” I alluded to a state of arrogance that was subsequently punctuated by a fall from grace and a sudden realisation that my testing strategy was in fact fatally flawed. The outcome was a new found appreciation for Unit Testing in its modern guise. Since then I’ve tried to guide other developers along the righteous path in the hope that they could be spared the pain of my embarrassment, but I’ve come to believe that it is a Rite of Passage because the benefits are largely downstream which makes the upfront costs feel excessive, especially when the clock is ticking.

Anyway, so how did I reach a level of complacence and arrogance that caused me to dismiss writing formal unit tests as a waste of time?

My professional career started out in the early 90’s at a small software house writing graphics based desktop applications. The great thing about graphical software is that you can see whether something works or not. Of course, off-by-one pixel and colour errors are difficult to spot, but by and large you can test the application manually and get good results. The culture at the company was strong on reading and we used to have copies of DDJ (Dr Dobbs Journal), MSJ (now MDSN Magazine) and WDJ (Windows Developer Journal) on rotation. I was also encouraged to read Steve McGuire's Writing Solid Code and Steve McConnell's Code Complete – two books which I found awe inspiring and I think others in the company did too. There was already a strong desire to produce quality software and many of the techniques in the books were adopted to ensure an even higher level of workmanship*.

This is a list of some of the core techniques that we adopted and that I’ve pretty much carried on to this day:-

• Compile with the highest possible warning level and report warnings as errors. Then, where possible, fix the code and not the compiler to ensure it builds cleanly.
• Use ASSERT to verify pre-conditions, post-conditions and invariants in functions to highlight bugs as soon as possible.
• Don’t code defensively. Fail fast and loudly to give yourself the best chance of finding and fixing bugs.
• Step through code changes in the debugger to verify that it does what you think it should, paying special attention to edge cases like loop termination conditions and branches to ensure your testing gives good code coverage.

The one item most relevant to this post is probably the last one - using the debugger to verify what you’ve written. I’ve followed this religiously as it made perfect sense to me. Why do a dry-run mentally or use print statements when you you can do a live run and inspect the state with the debugger? The most significant side-effect of this practice though is the need to be able to invoke relevant code in the debugger with ease so that the code/build/test cycle is as short as possible. This in turn often lead to the creation of test harnesses and diagnostic tools with a narrow focus that also really helps with support issues. So, if this was proving to be pretty successful, why change? Because the key thing missing is repeatability, and this is where I came unstuck...

Fast forward a few years and I was writing a very simple but fast XML parser and as a by-product I knocked up a noddy GUI which not only allowed us to edit the kind of simple XML data fragments we were dealing with, it also acted as a nice test harness because all the libraries' features could be exercised via the various GUI functions. One day I did a little refactoring, and after making some minor changes and running through some tests, I checked the edits into the source repository, safe in the knowledge that I’d improved the quality of the codebase a little. Some days later a colleague (who had been introduced to unit testing at a previous company and was trying to spread the word) announced to the team that his “unit tests were failing”. These tests were for a library that no one had touched in ages, and due to there being no Continuous Integration or automatic running of this test suite, it wasn’t as simple as picking on the last few check-ins. A quick spot of debugging later and the finger was duly pointed - at yours truly…

In my ‘supposedly’ low-risk spell of refactoring I had managed to replace a line something like this:-

return Decode(m_reader->ReadTag(blah, . . ., blah));

with one that no longer called the Decode() method which parsed and replace entity references like &amp; and &apos; :-

return m_reader->ReadTag(blah, . . ., blah, blah);

This should of course have been caught in my testing, but it wasn’t; because I was foolishly under the impression that my refactoring couldn’t have had any impact on the handling of entity references. Secondly I also apply a policy of diff’ing every single change before checking it in to ensure that I don’t mistakenly check-in temporary/commented out code and to give me a final pass to sanity check that I’ve not missed something crucial. My expected change was at the end of the line, but somehow I’d missed another accidental change at the start. It was during the subsequent discussion with my colleague that the light-bulb went on and I realised that I was (a) far more fallible than I had come to believe and (b) that I could in fact make subsequent changes faster and more accurately because of the scaffolding that would already be in place. Although the failing unit tests were not for my XML parser (he was just using it to load test cases and one of them had an “&amp;” in it) I fixed the code and quickly re-ran his test cases to verify my fix and allow him to continue his work.

Actually he had explained to me some months before about xUnit style unit testing and even showed me how to use CppUnit. My objection at the time was the volume of code you appeared to need to write just to test a simple function. Quite frankly I thought it was laughable that a Senior Software Engineer should need to resort to such verbose tactics to test their code – I felt I could test far more efficiently my way. Granted you don’t get some of the ease of use with C++ and CppUnit that you do with the Java and C# variants but the time spent physically writing the test ‘shell’ is probably marginal in comparison to that spent designing and writing both the code under test and the test itself.

I guess my mistake was being unaware of the true costs of maintenance – especially on large systems that have a regular turnover of staff/contractors. Yes, I may have written the code initially and will likely perform the first maintenance due to my familiarity with it, but longer term someone else will take over. It also greatly benefits the schedule if anyone can just dive into any code. Often there is little documentation so you have only the source to go on and unit tests provide a way for you to roll your specification into your codebase. The future maintenance programmer will then not only have some sort of spec to work from but will also have a purpose built regression test suite on tap to which he only has to write any additional tests or maybe refactor some existing ones. What they absolutely do not have to do is start from a blank sheet of paper. Now throw into the mix the fact that computers are a far better tool for performing mundane tasks like regression testing, and as long as you follow the advice about ensuring your tests run quickly, you can get excellent test coverage and feedback within a very short period of time. Automated building and running of test suites is de-rigour nowadays and wrapped up inside the concept of Continuous Integration. Roy Osherove (author of The Art of Unit Testing and the excellent ISerializable blog) covered Automated building and Continuous Integration just recently in his blog.

I’m sure some would argue that my example above wouldn’t have been found with a unit test, but would be classified as a component-level test; the definition of “the smallest testable part” depends to a degree on how much your design allows for fine grained testing. Either way the code did not require any expensive external resources, like a database or network connection, and so could have been subjected to a barrage of tests that would run in the blink of an eye, even on hardware of that era. I would then have known almost instantly of my mistake. Better yet, the suite of tests would also have provided my teammates the courage needed to allow them to perform similar surgery in the future.

 

 

*Unfortunately it seems that the magazine reviewers cared little about quality, or never tested the products significantly enough to notice poor workmanship. The evidence of this was the market leader at the time getting high marks for a product that was buggy and unreliable. Yes it had lots of features but you had better save your work regularly. This was during the ‘90s and whilst GST is no longer in business** the market leader continues to this day – so much for taking pride in ones work…

**What was originally Timeworks Publisher, and then PressWorks is now I believe called Greenstreet Publisher. I worked mostly on it’s sister product DesignWorks - a line-art drawing package - and the bundled utilities SnapShot and KeyPad.

WCF Weirdy – NetTcpBinding, EndpointIdentity and Programmatic Configuration

We are now using WCF for our inter-process communication (IPC) mechanism, and as you might expect the development and testing on a single machine worked a treat. However once we deployed it to our test environment in a proper remote client/server scenario it stopped working. I wasn’t involved in the main work, but as it appeared to be security related and I’ve done a fair bit in the past with DCOM and security I thought I’d see if I could lend a hand.

The architecture is (as Juval Lowry describes in his WCF Programming book) a classic Intranet setup, i.e. a client and server talking using the WCF NetTcpBinding and with all settings at their defaults. This meant that in our environment I expected it to pretty much mirror the run-of-the-mill DCOM model, except that we hadn’t got quite as far as running the service end under a “system” account yet – both client and server were still running under the same user account. The error we got from the client end was:-

“A call to SSPI failed…The target principal name is incorrect”

The initial Google search for this error threw up a very useful post from Scott Klueppel. The only problem was that we weren’t using configuration files, we were configuring the endpoints programmatically. Whilst I was googling my colleague ploughed through the MSDN documentation and found that using the constructor overload of EndpointAddress that takes an EndpointIdentity also seemed to do the trick:-

EndpointAddress endpoint = new EndpointAddress(uri, EndpointIdentity.CreateSpnIdentity("MACHINE\ACCOUNT"));

This tied up nicely with what Scott said is his post about needing an identity, but not entirely, because he mentioned that an empty identity will suffice. However the MSDN docs and many other Internet posts go intro great detail about crafting a service name string and using SetSPN to publish it to Active Directory. Naturally he followed the official documentation and it worked. To humour ourselves we also tried an empty string and that worked too! In fact it didn’t matter what we passed as the string to CreateSpnIdentity() – it worked. This just seemed way too random for my liking so I dug a little deeper to see if I could find out why this was happening. After all, if the contents of the string don’t matter, why bother passing it? And although it may not matter now, will it later down the line when we’ve all forgotten about this episode? That’s not the kind of maintenance nightmare I want to be responsible for.

Fortunately I discovered the answer via a couple of MSDN articles about WCF and Authentication – Overriding the Identity of a Service for Authentication and Service Identity and Authentication. The latter contained the following crucial note:-

“When you use NT LanMan (NTLM) for authentication, the service identity is not checked because, under NTLM, the client is unable to authenticate the server. NTLM is used when computers are part of a Windows workgroup, or when running an older version of Windows that does not support Kerberos authentication.”

I’ve yet to work in an Enterprise that relies on Kerberos across the board. I’ve seen it used in some parts of my clients’ network* but by-and-large NTLM still lives. I did search for a utility that might help confirm my suspicions, but I didn’t find anything. I know it’s famous last words, but I don’t see any reason to doubt this as the answer.

Long term the whole SetSPN thing seems like the way to go, but it will complicate our deployment procedure so I wanted to see if the articles had any other useful nuggets that might guide us towards some middle ground. As a footnote to Scott’s advice the MDSN articles have this to say about empty Identity strings:-

When SPN or UPN Equals the Empty String

If you set the SPN or UPN equal to an empty string, a number of different things happen, depending on the security level and authentication mode being used:

• If you are using transport level security, NT LanMan (NTLM) authentication is chosen.
• If you are using message level security, authentication may fail, depending on the authentication mode:
• If you are using spnego mode and the AllowNtlm attribute is set to false, authentication fail.
• If you are using spnego mode and the AllowNtlm attribute is set to true, authentication fails if the UPN is empty, but succeeds if the SPN is empty.
• If you are using Kerberos direct (also known as "one-shot"), authentication fails.

We’ve temporarily gone for the first option and filed a JIRA task to revisit this before our deployment procedures need finalising. This also gives me more time to read up on the subject of WCF authentication. I own a copy of Keith Browns Programming Windows Security which has served me well, but it’s finally starting to look a little dated.

 

* It’s a good way of find out if your clock is in sync or not. I once investigated a bunch of authentication failures on some of our VMware test boxes and it turned out to be because the clocks had drifted outside the 5 minute window that Kerberos allows.

TFD vs TLD – What’s The Definition of “Later”?

The fallout from Joel Spolsky’s “The Duct Tape Programmer” post still seems to be reverberating around the blogosphere. The dust has largely settled and Uncle Bob has resumed normal service on his usual insightful postings about testing etc. For a moment I thought he had gone way over the top and become excessively evangelical - even for him. He seemed to suddenly become a one man crusade to restore balance to the force and recruit legions to the cause of TDD. I saw him in person for the first time at the ACCU Conference last year and found him a truly compelling speaker.

As often occurs with a controversial issue such as this I discovered lots of useful discussion in the comments to his post, which I personally lapped up because I have only really been able to dabble in Unit Testing and TDD due to working on legacy systems with massive coupling. Also there were plenty of TLA’s flying about such as TFD (Test First Development), TLD (Test Later Development) and TDD (Test Driven Design/Development) along with BDD (Behaviour Driven Development) and DDD (Domain Driven Design) thrown in for good measure to keep any idle spectators on their toes.

The most useful post for me came a few days later when Uncle Bob (Echoes from the Stone Age) started to discuss the types of problems that you cannot tackle with TDD. This was hugely enlightening because this list pretty much mirrors a large part of what I do day-to-day. I felt somewhat relieved that I haven’t been “not getting the point” so much as attempting to squeeze the proverbial square peg into a round hole. And before you ask, it’s not that I haven’t mocked my interfaces to reduce my dependencies etc, it’s just that I don’t write a lot of what you might traditionally call Business Logic. The logic I do write is often heavily dependent on I/O, threads or 3rd party systems which pushes one more towards the ‘Functional’ rather than ‘Unit’ side of the testing spectrum to find real value.

Anyway, the one part of the discussion which I never felt was truly resolved, or that even the warring parties seemed to clarify their position on was what is the definition of “later” in TLD - Test Later Development. I got the general impression that some of the commentators had visions of old-school Waterfall style development where the “later” meant weeks, months or even years after coding had been completed. Does anyone really work like that these days? Did anyone really work like that? Certainly in my career to date the notion of “later” has pretty much meant seconds to minutes, with an hour on a rare occasion. I cannot imagine ever writing a non-trivial piece of code and not testing it in some way immediately. Yes, my tests may not have been truly exhaustive, but I do know that when a piece of code I wrote failed it was probably due to some fairly non-obvious scenario. I realise that sounds incredibly arrogant and I’m going to document my fall from grace in a follow-up post…

What was clear from the comments was that everyone in the discussion agreed that Unit Testing is A Good Thing – for some definition of unit testing. However it did also highlight that there is still much confusion about the various different forms of testing and what people call them. Some did not distinguish between TDD and Unit Testing which raised the ire of a few commentators. I myself have found the definition of “Unit” to be incredibly vague especially when trying to apply it to other development disciplines such as database scripts. I worked on a legacy system once where, due to the high level of coupling, the smallest testable unit was an entire .exe component.

I think I’m now beginning to “get” the TDD philosophy, but I’m not entirely sure it fits my style of working for two reasons. The first is that I have always found very lightweight modelling (in the way that Martin Fowler describes it as sketching in UML Distilled) to be incredibly useful for visualising a design first. The second is that I have 15+ years of experience to draw upon which means that many aspects of a design are in the realms of boiler-plate code. I find it very disruptive context switching between the boiler-plate and test code when I just want to rattle out the bog standard stuff first and use TLD to add the unit tests after. I can therefore save TFD for the new territory even though I often have a pretty good idea of where I’m heading because that’s why I sketched out the design in the first place. I’m sure it’s a fatally flawed argument and maybe I need to let go of the up front sketches and trust that I’ll still find my way via TDD and discover other benefits along the way. Old dogs can learn new tricks - you just have to tempt them with the right sort of biscuits.

Maybe I’m being too literal (it’s one of my more annoying traits) and perhaps we should just avoid the use of the TLD acronym and instead use TMLD (Test Moments-Later Development) and TMLD (Test Much-Later Development). Oh, wait, that won’t work… How about TRAID (Test Right After Implementing Development) and TAYFHIWD (Test After You’ve Forgotten How It Works Development). Hopefully there will be another TFD vs TLD flame war real soon and I’ll get the chance to fill in some more blanks and gain a further understanding of the Software Development Process. Heck, why am I waiting? Surely I can bait Uncle Bob with what I’ve just written… Now then, where’s his blog hosted again…