Haskell: unexpectedly making it harder to unit test

Surprised by the title? So was I.

Haskell is a pure functional language devoid of side-effects that gets work done by instead describing those side-effects. All of the code that deals with the outside world is tainted by IO in the type system, forever to be shunned, a pariah with regards to the rest of the program. This is touted as one Haskell’s strengths, and rightly so. But recently, it got in my way.

My go-to task to learn a new programming language now is to implement an MQTT broker. That was in fact, the subject of my very first post. So that’s how I attacked learning Haskell. Some time later, I had a nasty bug and concluded that my problem had been too much logic in the IO monad. I started refactoring like mad, writing new unit tests and making sure most of the program logic was in pure code. It took a while to realise… I couldn’t.

I went fancy in my testing endeavours and almost wrote a blog post about it. I abstracted away how bytes were to be acquired and wrote a function with this signature:

replyStream :: (Monad m) => a -> (a -> m BS.ByteString) -> Subscriptions a -> m [Response a]

The reason for that ‘a’ type parameter is that in production ‘a’ is a Handle to read from, and in tests, ‘a’ can be any type I want, just to check the logic (I used Int). The part I was proud of was the second parameter: a monadic function that maps a handle type to a bytestring in a given monad. In production, the monad would be IO, in tests, I used the State monad and fed it the “packets” I wanted. My idea was to construct an infinite lazy byte stream for each client, feed it to a function that would produce an infinite lazy list of MQTT messages by calculating where the byte boundaries were and in turn give it to replyStream, which would get more bytes from the passed-in function. Code reuse! Abstraction! No IO!

But then… I tried making this work with the existing networking code, at which point I realised I couldn’t. Not if I wanted the server to work as it should, anyway. My approach would have worked if only there wasn’t any global mutable state in the MQTT protocol, but there is. What one client sends affects what the other clients might receive. Which means I can’t just use State to thread what the subscriptions are from message to message in the stream, since every message might have to change the global state, which alters how the server processes messages from different clients. The streams aren’t independent.

What’s a Haskell programmer to do? As far as I know, the Haskell options for global mutable state are IORef, MVar and TVar. If there are other options, please let me know. And here’s the kicker: no matter which one is chosen, they all require the IO monad to get any work done. Which meant whatever code I wrote to process messages would be IO-tainted and therefore, not testable in a pure way.

I’m sure there’s more than one way to skin this particular cat. I considered using TVar and two monads for replyStream so that in production it could be STM returning IO, but that just seems unnecessarily complicated. I already dislike the fact that I made my functions generic on the handle type, and using two monads, monad transformers or what-have-you is just too much complication and abstraction in production code and all “just” for testing. I gave up, moved the logic to the IO portion and called it a day.

For once, this would have been easier in run-of-the-mill imperative language. Handle type? Just cast from int in tests. Unit testing without side-effects? Easily done in the D version. How? Regular global mutable state. No hoops, no monads, just easily readable code.

Haskell: you’ve let me down. It may well be there’s still an idiomatic way to accomplish what I wanted to do. Even then, it isn’t obvious at all and that would also be a failure.


6 thoughts on “Haskell: unexpectedly making it harder to unit test

  1. Paul says:

    The obvious solution is to make your broker a monad transformer with its own private state.

  2. mightybyte says:

    Haskell is not making it harder to test. You are making it harder to test by refusing to use Haskell’s equivalent to the things that you are doing in D. The fact that those things might need to run in the IO monad is irrelevant.

    • atilaneves says:

      You may disagree, but I don’t think it’s irrelevant. It may well be that the best way is to write unit tests for code in the IO monad that doesn’t actually do any IO. If that’s the case, I don’t think it’s ideal.

      • When I want to test an API that lives in IO. I stop and break apart the operations that I actually want into a class.

        class Monad m => MyOperations m where
        whateverImperativeOperation :: Int -> Double -> Thingy -> m Result

        and describe the laws I’d expect to hold between those operations.

        I can then make an instance for IO that lets my program run as usual.

        Then I can write a monad that implements this API but which does ‘mocking’ or which is a pure semantic model. Both of those are useful for testing.

        Along the way I usually discover that I can lift MyOperations over monad transformers so my code becomes much more general as a consequence.

        So in the end I get a mocking framework, a semantic model, a fast IO implementation, and the ability to lift over monad transformers and I can go about my day with a great deal more surety that my model is sound. If it isn’t one of those steps usually finds the problem long before I get to the unit tests!

        Alternately, if all of that is too much work for you there are things like http://hackage.haskell.org/package/QuickCheck-2.8.1/docs/Test-QuickCheck-Monadic.html that can be used directly for monadic testing.

      • mightybyte says:

        It’s not ideal, but literally everything you do in any other mainstream language is way worse. It’s not fair to Haskell to be in purist mode when you’re talking about Haskell but pragmatist mode when you’re talking about any other language. But that is what you’re doing here when you say “Haskell: unexpectedly making it harder to unit test” and “Easily done in the D version”. I’m fine with either perspective, but you should apply the perspectives evenly across the board.

  3. […] Haskell: Unexpectedly Making It Harder To Unit Test […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: