The main function should be shunned

The main function (in languages that have it) is…. special. It’s the entry point of the program by convention, there can only be one of them in all the object files being linked, and you can’t run a program without it. And it’s inflexible.

Its presence means that the final output has to be an executable. It’s likely however, that the executable in question might have code that others might rather reuse than rewrite, but they won’t be able to use it in their own executables. There’s already a main function in there. Before clang nobody seemed to stumble on the idea that a compiler as a library would be a great idea. And yet…

This is why I’m now advocating for always putting the main function of an executable in its own file, all by itself. And also that it do the least amount of work possible for maximum flexibility. This way, any executable project is one excluded file away in the build system from being used as a library. This is how I’d start a, say, C++ executable project from scratch today:

#include "runtime.hpp"
#include <iostream>
#include <stdexcept>

int main(int argc, const char* argv[]) {
    try {
        run(argc, argv); // "real" main
        return 0;
    } catch(const std::exception& ex) {
        std::cout << "Oops: " << ex.what() << std::endl;
        return 1;
    }
}

In fact, I think I’ll go write an Emacs snippet for that right now.

Tagged ,

API clarity with types

API design is hard. Really hard. It’s one of the reasons I like TDD – it forces you to use the API as a regular client and it usually comes out all the better for it. At a previous job we’d design APIs as C headers, review them without implementation and call it done. Not one of those didn’t have to change as soon as we tried implementing them.

The Win32 API is rife with examples of what not to do: functions with 12 parameters aren’t uncommon. Another API no-no is several parameters of the same type – which means which? This is ok:

auto p = Point(2, 3);

It’s obvious that 2 is the x coordinate and 3 is y. But:

foo("foo", "bar", "baz", "quux", true);

Sure, the actual strings passed don’t help – but what does true mean in this context? Languages like Python get around this by naming arguments at the call site, but that’s not a feature of most curly brace/semicolon languages.

I semi-recently forked and extended the D wrapper for nanomsg. The original C API copies the Berkely sockets API, for reasons I don’t quite understand. That means that a socket must be created, then bound or connect to another socket. In an OOP-ish language we’d like to just have a contructor deal with that for us. Unfortunately, there’s no way to disambiguate if we want to connect to an address or bind to it – in both cases a string is passed. My first attempt was to follow in Java’s footsteps and use static methods for creation (simplified for the blog post):

struct NanoSocket {
    static NanoSocket createBound(string uri) { /* ... */ }
    static NanoSocket createConnected(string uri) { /* ... */ }
    private this() { /* ... */ } // constructor
}

I never did feel comfortable: object creation shouldn’t look *weird*. But I think Haskell has forever changed by brain, so types to the rescue:

struct NanoSocket {
    this(ConnectTo connectTo) { /* ... */ }
    this(BindTo bindTo) { /* ... */ }
}

struct ConnectTo {
    string uri;
}

struct BindTo {
    string uri;
}

I encountered something similar when I implemented a method on NanoSocket called trySend. It takes two durations: a total time to try for, and an interval to wait to try again. Most people would write it like so:

void trySend(ubyte[] data, 
             Duration totalDuration, 
             Duration retryDuration);

At the call site clients might get confused about which order the durations are in. I think this is much better, since there’s no way to get it wrong:

void trySend(ubyte[] data, 
             TotalDuration totalDuration, 
             RetryDuration retryDuration);

struct TotalDuration {
    Duration duration;
}

struct RetryDuration {
    Duration duration;
}

What do you think?

Tagged , , , , , , , ,

Don’t hoard code

For me, the two most important principles in programming are, in order, DRY and YAGNI. Most of my coding decisions ends up respecting one or the other. For some reason YAGNI seems to be less well known. In my experience one tends to get less pushback for DRY – it’s the accepted best practice. But YAGNI seems to need more persuasion, and I’m not entirely sure why.

I’m converted: I love red diffs. I don’t even look at the red sections during code review. Do the tests still pass? Ship it! The thing is that, despite me being a programmer and my “one job” (not really, but you know what I mean) being to write code, I hate code and want the least of it in my project. I mean it.

Code that doesn’t exist is excellent. It doesn’t have to be read, and therefore doesn’t need to be understood, which means it can’t confuse anyone. It doesn’t have bugs. It doesn’t need to be tested. What’s not to like?

And yet, in project after project, one sees code commented out for mostly no good reason. My personal “favourite” (by which I mean I froth at the mouth) is C or  C++ code with #if 0 / #endif pairs. In one project there were even multiple of those, and nested to boot.

Maybe it has to do with not trusting version control. If all you’ve ever used is one of those ancient paid-for systems (not naming any names but you can guess) and have never felt the bliss that is working with git or Mercurial then maybe it’s understandable. Because it might actually be hard to go look at the history and find when you deleted something or why. But these days? No excuse: git grep that_thing_that_I_remember_that_isn’t here_anymore.

And never mind that, in my experience at least, the times anybody goes code spelunking for deleted code are so few and far between that the trade-off is obvious. Code that hasn’t but should be deleted gets in the way. That’s a real cost, paid every day, and for… what? Because someone someday might need that snippet and it takes them an extra minute to find it?

YAGNI. Delete and move on.

Arch Linux – why use a Docker image when you can create your own?

It seems silly in retrospect. I’d never have considered building an Arch Linux based Docker container in any other way but starting with one from the registry and a Dockerfile. But… it’s Arch Linux, you install this distro into a directory and chroot into it. Why settle for someone else’s old installation?

The script used to bootstrap an Arch install, pacstrap, even lets you exclude some packages from the default install or add things you need. So I ended up with a bash script that installs Arch into a directory, chroots into it and runs commands as required, then bundles the whole thing into a docker container. Repeatable, checked in to version control, and not wasting layers of AUFS.

Who needs a Dockerfile?

C is not magically fast, part 2

I wrote a blog post before about how C is not magically fast, but the sentiment that C has properties lacking in other languages that make it so is still widespread. It was with no surprise at all then that a colleague mentioned something resembling that recently at lunch break, and I attempted to tell him why it wasn’t (at least always) true.

He asked for an example where C++ would be faster, and I resorted to the old sort classic: C++ sort is faster than C’s qsort because of templates and inlining. He then asked me if I’d ever measured it myself, and since I hadn’t, I did just that after lunch. I included D as well because, well, it’s my favourite language. Taking the minimum time after ten runs each to sort a random array of 10M simple structs on my laptop yielded the results below:

  • D: 1.147s
  • C++: 1.723s
  • C: 1.789s

I expected  C++ to be faster than C, I didn’t expect the difference to be so small. I expected D to be the same speed as C++, but for some reason it’s faster. I haven’t investigated the reason why for lack of interest, but maybe because of how strings are handled?

I used the same compiler backend for all 3 so that wouldn’t be an influence: LLVM. I also seeded all of them with the same number and used the same random number generator: the awful srand from C’s standard library. It’s terrible, but it’s the only easy way to do it in standard C and the same function is available from the other two languages. I also only timed the sort, not counting init code.

The code for all 3 implementations:

// sort.c
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/time.h>
#include <sys/resource.h>

typedef struct {
    int i;
    char* s;
} Foo;

double get_time() {
    struct timeval t;
    struct timezone tzp;
    gettimeofday(&t, &tzp);
    return t.tv_sec + t.tv_usec*1e-6;
}

int comp(const void* lhs_, const void* rhs_) {
    const Foo *lhs = (const Foo*)lhs_;
    const Foo *rhs = (const Foo*)rhs_;
    if(lhs->i < rhs->i) return -1;
    if(lhs->i > rhs->i) return 1;
    return strcmp(lhs->s, rhs->s);
}

int main(int argc, char* argv[]) {
    if(argc < 2) {
        fprintf(stderr, "Must pass in number of elements\n");
        return 1;
    }

    srand(1337);
    const int size = atoi(argv[1]);
    Foo* foos = malloc(size * sizeof(Foo));
    for(int i = 0; i < size; ++i) {
        foos[i].i = rand() % size;
        foos[i].s = malloc(100);
        sprintf(foos[i].s, "foo%dfoo", foos[i].i);
    }

    const double start = get_time();
    qsort(foos, size, sizeof(Foo), comp);
    const double end = get_time();
    printf("Sort time: %lf\n", end - start);

    free(foos);
    return 0;
}


// sort.cpp
#include <iostream>
#include <algorithm>
#include <string>
#include <vector>
#include <chrono>
#include <cstring>

using namespace std;
using namespace chrono;

struct Foo {
    int i;
    string s;

    bool operator<(const Foo& other) const noexcept {
        if(i < other.i) return true;
        if(i > other.i) return false;
        return s < other.s;
    }

};


template<typename CLOCK, typename START>
static double getElapsedSeconds(CLOCK clock, const START start) {
    //cast to ms first to get fractional amount of seconds
    return duration_cast<milliseconds>(clock.now() - start).count() / 1000.0;
}

#include <type_traits>
int main(int argc, char* argv[]) {
    if(argc < 2) {
        cerr << "Must pass in number of elements" << endl;
        return 1;
    }

    srand(1337);
    const int size = stoi(argv[1]);
    vector<Foo> foos(size);
    for(auto& foo: foos) {
        foo.i = rand() % size;
        foo.s = "foo"s + to_string(foo.i) + "foo"s;
    }

    high_resolution_clock clock;
    const auto start = clock.now();
    sort(foos.begin(), foos.end());
    cout << "Sort time: " << getElapsedSeconds(clock, start) << endl;
}


// sort.d
import std.stdio;
import std.exception;
import std.datetime;
import std.algorithm;
import std.conv;
import core.stdc.stdlib;


struct Foo {
    int i;
    string s;

    int opCmp(ref Foo other) const @safe pure nothrow {
        if(i < other.i) return -1;
        if(i > other.i) return 1;
        return s < other.s
            ? -1
            : (s > other.s ? 1 : 0);
    }
}

void main(string[] args) {
    enforce(args.length > 1, "Must pass in number of elements");
    srand(1337);
    immutable size = args[1].to!int;
    auto foos = new Foo[size];
    foreach(ref foo; foos) {
        foo.i = rand % size;
        foo.s = "foo" ~ foo.i.to!string ~ "foo";
    }

    auto sw = StopWatch();
    sw.start;
    sort(foos);
    sw.stop;
    writeln("Elapsed: ", cast(Duration)sw.peek);
}



Tagged , ,

Dipping my toes in the property based testing pool

I’ve heard a lot about property-based testing in the last 2 years but haven’t really had a chance to try it out. I first heard about it when I was learning Haskell, but at the time I thought regular bog-standard unit testing was a better option. I didn’t want to learn a new language (and one notoriously difficult at that) and a new way of testing at the same time. Since then I haven’t written anything else in Haskell for multiple reasons and it’s always been something I’ve been wanting to try out.

I decided that the best way to do it is just to implement property-based testing myself, and I’ve started with preliminary support in my unit-threaded library for basic types and arrays thereof. The main issue was knowing how to write the new tests. It wasn’t clear at all and if you’re in the same situation I highly recommend this talk on the subject. Fortunately, one of the examples in those slides was serialization, and since I wrote a library for that too, I immediately started transitioning some pre-existing unit tests. I have to say that I believe the new tests are much, much better. Here’s one test “on paper” but that actually runs 100 random examples for each of the 17 types mentioned in the code, checking that serializing then deserializing should yield the same value:

@Types!(bool, byte, ubyte, short, ushort, int, uint, long, ulong,
        float, double,
        char, wchar, dchar,
        ubyte[], ushort[], int[])
void testEncodeDecodeProperty(T)() {
    check!((T val) {
        auto enc = Cerealiser();
        enc ~= val;
        auto dec = Decerealiser(enc.bytes);
        return dec.value!T == val;
    });
}

I think that’s a pretty good test/SLOC ratio. Now I just have to find more applications for this.

Am I a Mockist now?

I’ve always considered myself on the classicist side of using test doubles. It just clicked with me I guess, and it didn’t help that I’ve not had good experiences with using mocking frameworks. The first time I tried it in Python I ended up asserting different functions were being called in such a way that my test started mirroring the production code, and at that point I stopped. It was clearly the wrong direction, but I at least I noticed. I had the displeasure once to review a test like that once and had to convince the authors it was a bad idea.

I recently bought and read Jay Fields’s Working Effectively With Unit Tests and was finally exposed to how a mockist goes about writing their tests. I was a bit of an eye-openener: the examples and explanations actually made sense, and I started thinking that maybe mockist thinking wasn’t so bad. What was really surprising to me was that using mocks was described as testing behaviour instead of state: I’d always thought that the downsides of mocks were that they tested implementation instead of behaviour. I started realising that well-written mockist tests just get to the same destination by a different way. I’d just never seen well written tests with mocks before.

Coincidentally, just after I’d finished that book I listened to a podcast interview with J.B. Rainsberger of Integrated Tests are a Scam fame. I heard him say something I’d never heard before: “Don’t mock types you don’t own”. That was also eye-opening: everything that made mocking bad in my eyes suddenly disappeared. If you’re only ever mocking types/functions under your control, then testing implementation isn’t such a bad thing, you can always refactor the implementation of the code you own without tests breaking. No more brittle tests and no more writing test doubles by hand.

I tried it out recently at work and I was actually pleased with the result. Am I a mockist now? I don’t know. Even though I used mocks I ended up with a hybrid approach for some tests, so I guess I’m just trying to use the right tool for the job. Which is always a good thing, right?

Prototyping is useful after all

Python is said to be a great prototyping language, I guess because types don’t have to be written down explicitly. I’ve never really gotten why someone would want to do that. Even after reading Code Complete I felt like it sounded like a good idea but also maybe… a waste of time?

I think I’ve just changed my mind. I’ve been struggling at work for a couple of weeks now with some new C code. It’s not just that it’s in C, there’s also some infra I’d never used before to work out as well. I felt like my head was clouded by implementation details that I had to consider at every step, so I tried something different: write it all, from scratch, in D.

Why D? Two reasons. First, it’s my favourite language. Second, the syntax is similar enough that a translation of the solution to C should be straightforward. One hour later I’d not only implemented everything I wanted, but now knew exactly what I had to do in C-land. Which, of course, will take much longer than an hour.

Still, I’m now wondering why I took so long to prototype my solution. It could’ve saved me two weeks of mostly unproductive work. The problem is too hard to solve in your current environment? Solve it somewhere else where it’s easy and translate later.

Write custom assertions whenever possible

I’ve been very interested in readable tests with great error messages recently. Mostly because they kept failing and I wanted the most information possible in order to quickly identify the cause. This is another reason why I like TDD: you see the test failing first, so if the error message isn’t great you’ll know straight away instead of months later.

The good testing frameworks provide a way of writing your own custom assertions. I’d never really looked into them that much before, but now I realize the error of my ways. Recently I wrote a test that contained this line:

fileName.exists.shouldBeTrue;

Readable, right? The problem is when it fails:

foo.d:42 - Expected: true
foo.d:42 -      Got: false

And now you have to go read the test and figure out what went wrong. It’s a lot better to get the information that a file was supposed to exist instead right away. So I wrote a custom assertion and was then ready to write this:

fileName.shouldExist;

With the corresponding failure message:

foo.d:42 - Expected /tmp/foo.txt to exist but it didn't

Now it’s a lot easier to pinpoint where the problem is. For starters, you would probably want to start checking the contents of the surrounding directory, having saved the time you would have had to spend figuring out what exactly was false.

Tagged ,

The Reader monad is just an object

I found myself writing some code like this a week or two ago:

string testPath = ...;
func1(testPath, ...);
func2(testPath, ...);
...

I thought it was annoying that I had to keep repeating the same argument to every function call and started thinking about ways that I could automatically “thread” it. By just thinking that word I was taken to the world of Haskell and thought that the Reader monad would be great here. I even thought of implementing it in D, until I realised…  that’s just an immutable object! I don’t need a monad, just a struct or a class:

struct Foo {
    string testPath;
    func1(...) const;
    func2(...) const;
}

In most OOP languages I don’t need to pass anything when func1 or func2 are called, and they both implicitly have access to all the instance variables, which is exactly what I wanted to do anyway. In D I can even get something resembling Haskell do notation by writing the client code like this:

with(immutable Foo()) {
    func1(...);
    func2(...);
}

(as opposed to):

auto foo = immutable Foo();
foo.func1(...);
foo.func2(...);

I can’t believe it took me this long to realise it, but the Reader, Writer and State monads are just… objects. Reader is immutable, but I just showed how to get the same effect in D. It’d be similar in C++ with a const object, and the lack of the with keyword would make it more verbose, but in the end it’s very similar.