Testing Coffeescript Cake Tasks

At Startup Weekend, we have a few cake tasks for aggregating data and statistics that we run nightly. Figuring out how to test these cake tasks was an interesting conversation, and eventually we ended up running a child process to execute our cake task. Here's an example of what our mocha tests looked like:

{exec} = require 'child_process'

describe "Cake tasks", ->
  describe "dashboard:events-metrics", ->
    beforeEach (done) ->
      fixtures.load "../fixtures/events", ->
        exec "NODE_ENV=test cake dashboard:events-metrics", (err, stdout) ->
          done()

    it "does something", (done) ->
      …

This approach immediately worked and was easy to debug. Another aspect I liked about this approach was that it most directly mimicked what happens when the cake task was invoked from the command line. The downside was that each exec call took between 3 and 4 seconds. This was much longer than any other test took to execute, but at least everything worked as expected.

I've recently taken a moment to re-think how we should be testing these tasks, especially to make them run faster. I came up with an approach that didn't use child_process. This approach required adding some test helper functions. Here is what part of our test_helpers.coffee file looks like:

cakeTasks = {}
module.exports =
  task: (name, description, action, done) ->
    [action, description] = [description, action] unless action
    cakeTasks[name] = {name, description, action}

  runTask: (name, cb) ->
    throw "No Task Found" unless (action = cakeTasks[name]?.action)
    done = ->
      cb()
    console.log action.length
    if action.length > 0
      action(done)
    else
      action()
      cb()

Let's go back to our test file and import these functions:

{task, runTask} = require '../test_helpers'

# require our tasks
require('../../lib/tasks/dashboard')(task)

If your look at lib/tasks/dashboard, you'd see us defining tasks like this:

module.exports = exports = (task) ->
  task 'dashboard:events-metrics', "calculate events data", ->
    # do something

Then our Cakefile simply requires these tasks:

require("./lib/tasks/dashboard")(task)

Now, back to our tests. Change the exec call to use the new runTask function:

beforeEach (done) ->
  fixtures.load "../fixtures/events", ->
    # exec "NODE_ENV=test cake dashboard:events-metrics", (err, stdout) ->
    runTask "dashboard:events-metrics", ->
      done()

The biggest catch is that, now, our asynchronous cake tasks won't finish completing before the callback fires. We have to change our task definition to accept a done function, to tell our task runner when the task has finished.

task 'dashboard:events-metrics', "calculate events data", (done) ->
   … do something
   done?()

The reason for calling done?() and not done() is because when this task is invoked with cake, the done method argument will be undefined.

After changing our test suite to call runTask instead of exec, the performance boost is unbelievable. Before, each test took about 4 seconds to run, which added about a minute to the time it took to run our whole test suite. After switching to the new testing method, each test dropped down to about 40 milliseconds. As you can imagine, this 10,000% speed boost is extremely welcome.