Tag Archives: testing practices

main is just another function

Last week I talked about code that isn’t unit-testable, at least not by my definition of what a unit test is. In keeping with that, this blog post will talk about testing code that has side-effects.

Recently I’d come to accept a defeatist attitude where I couldn’t think of any other way to test that passing certain command-line options to a console binary had a certain effect. I mean, the whole point is to test that running the app differently will have different consequences. As a result I ended up only ever doing end-to-end testing. And… that’s simply not where I want to be.

Then it dawned on me: main is just another function. Granted, it has a special status that makes it so you can’t call it directly from a test, but nearly all my main functions lately have looked like this:

int main(string[] args) {
    try {
        doStuff(args);
        return 0;
    } catch(Exception ex) {
        stderr.writeln(ex.msg);
        return 1;
    }
}

It should be easy enough to translate this to the equivalent C++ in your head. With main so conveniently delegating to a function that does real work, I can now easily write integration tests. After all, is there really any difference between:

doStuff(["myapp", "--option", "arg1", "arg2"]);
// assert stuff happened

And (in, say, a shell script):

./myapp --option arg1 arg2
# assert stuff happened

I’d say no. This way I have one end-to-end test for sanity’s sake, and everything else being tested from the same binary by calling the “real” main function directly.

If your main doesn’t look like the one above, and you happen to be writing C or C++, there’s another technique: use the preprocessor to rename main to something else and call it from your integration/component test. And then, as they say, Bob’s your uncle.

Happy testing!

Advertisements
Tagged , , ,

Neves’s Laws of Testing

I got into a heated argument over testing practices at work the other day. That led me to think about the subject quite a bit, and have come up with a few laws of testing based on my experience:

  1. Only mock what’s needed to make the test deterministic and fast; otherwise call the real code
  2. A flaky test is worse than no test at all
  3. Good tests verify behaviour; bad tests verify an implementation

The worst thing about law number 3 is that it’s just stating the commonly known knowledge that coupling is bad and should de reduced, but for some reason some people don’t realise they’re increasing it when they write tests for the implementation.

Law number 1 needs some explaining. Mostly I’m only interested in mocking/stubbing/etc. functions that make a unit test no be “pure”. Usual suspects are file system operations, networking, databases, etc. I’ve seen a few examples that don’t fit the “traditional” examples but make good canditates. One that particularly opened my eyes was using a mock log object to state that error messages were logged when calling a function with certain arguments. Another situation is that in some embedded contexts it might be hard to even get a file to compile outside of a specialised environment. Mocking can make sense there too. Basically, whatever makes testing difficult should be mocked. Or to paraphrase Einstein, “All code should be mocked as little as possible, but not less”.

What are your testing laws? How do you disagree with mine?

 

Tagged