Kevlin Henney: Rethinking Unit Testing in C++
For the most part I think the C++ suffix in the presentation’s title was a little superfluous. Yes, he was demonstrating with examples in C++, but much of the message was clearly language agnostic. The first half illustrated the evolution of unit testing from a bunch of random chunks of ‘test’ code through to an organised set of tests that had a significant air of “Formal Specification” about it. The initial tests were really just a stream of consciousness laid out sequentially in a single scope with no attention paid to structure and consequently future maintenance costs. This was then evolved into a procedural style whereby the tests were partitioned around the (member) functions under test. One of the main issues with this style is that it’s impossible not to use the same functions in other test methods, which makes the idea of isolated tests and order independence somewhat moot.
The next progression was coined ‘contextual’ whereby the partitioning was biased towards ‘behaviours’ or ‘features’ rather than the implementation. By this point the names of the tests started to follow natural language rather than the more terse method naming we use daily and Kevlin described how some developers find this transition difficult because they’re not used to such expression. Although not as rigid, the style was reminiscent of the BDD Given/When/Then form with the test name resembling a proposition rather than an exclamation. The judicious application of macros meant that the source file was taking on a more formal shape with a SPECIFICATION at the top, followed by a series of PROPOSITIONs. But, not only did the code appear more formal, the output from the tests had a more intention revealing feel about it. Nat Pryce and Steve Freeman gave a talk a few months back about TDD in which they highlighted the importance of the test diagnostics and these ideas certainly enhanced that message.
A brief diversion into unit testing C++ code with NUnit & C++/CLI then followed. Once again the aid of a few macros helped to hide away the test method attributes and static Assert invocations so that visually the code appeared just like the native versions. Kevlin admitted that there are limitations to this technique, but it was a good first approximation that showed a nice meshing of the two ‘meta’ techniques – .Net reflection and C/C++ pre-processing.
For his grand finale Kevlin showed how he could capture the expression being asserted. However this was not only in textual form, but also in programmatic form so that the output for a test failure would display the failed expression and the values of the variables used within that expression. Naturally the audience wanted to see behind the curtain and we quickly found ourselves staring into Expression Template territory. Kevlin only had a partial implementation due to time constraints, but there was already a significant body of template code. The most interesting aspect was the choice of operator used to allow the code to work its magic by ‘capturing’ the leftmost part of the assert expression – it needed to be overloadable, practically unused and higher than those you would normally find inside assert expressions. The proverbial rabbit that he pulled out of the hat was the ‘->*’ operator which I readily confess to not knowing, but promptly Googled when I got home.
Not that I departed for home the moment the bell went as there was still the matter of post-discussion beer which was satisfied by The Slaughtered Lamb. Fortunately we had a few of the non-members join us in the pub afterwards which gave us ample opportunity to slip The King’s Shilling into their pint…