At Startup Weekend, we have a few cake tasks for aggregating data and statistics that we run nightly. Figuring out how to test these cake tasks was an interesting conversation, and eventually we ended up running a child process to execute our cake task. Here's an example of what our mocha tests looked like:
{exec} = require 'child_process'
describe "Cake tasks", ->
describe "dashboard:events-metrics", ->
beforeEach (done) ->
fixtures.load "../fixtures/events", ->
exec "NODE_ENV=test cake dashboard:events-metrics", (err, stdout) ->
done()
it "does something", (done) ->
…
This approach immediately worked and was easy to debug. Another aspect I liked about this approach was that it most directly mimicked what happens when the cake task was invoked from the command line. The downside was that each exec
call took between 3 and 4 seconds. This was much longer than any other test took to execute, but at least everything worked as expected.
I've recently taken a moment to re-think how we should be testing these tasks, especially to make them run faster. I came up with an approach that didn't use child_process
. This approach required adding some test helper functions. Here is what part of our test_helpers.coffee
file looks like:
cakeTasks = {}
module.exports =
task: (name, description, action, done) ->
[action, description] = [description, action] unless action
cakeTasks[name] = {name, description, action}
runTask: (name, cb) ->
throw "No Task Found" unless (action = cakeTasks[name]?.action)
done = ->
cb()
console.log action.length
if action.length > 0
action(done)
else
action()
cb()
Let's go back to our test file and import these functions:
{task, runTask} = require '../test_helpers'
# require our tasks
require('../../lib/tasks/dashboard')(task)
If your look at lib/tasks/dashboard
, you'd see us defining tasks like this:
module.exports = exports = (task) ->
task 'dashboard:events-metrics', "calculate events data", ->
# do something
Then our Cakefile
simply requires these tasks:
require("./lib/tasks/dashboard")(task)
Now, back to our tests. Change the exec
call to use the new runTask
function:
beforeEach (done) ->
fixtures.load "../fixtures/events", ->
# exec "NODE_ENV=test cake dashboard:events-metrics", (err, stdout) ->
runTask "dashboard:events-metrics", ->
done()
The biggest catch is that, now, our asynchronous cake tasks won't finish completing before the callback fires. We have to change our task definition to accept a done
function, to tell our task runner when the task has finished.
task 'dashboard:events-metrics', "calculate events data", (done) ->
… do something
done?()
The reason for calling done?()
and not done()
is because when this task is invoked with cake
, the done
method argument will be undefined
.
After changing our test suite to call runTask
instead of exec
, the performance boost is unbelievable. Before, each test took about 4 seconds to run, which added about a minute to the time it took to run our whole test suite. After switching to the new testing method, each test dropped down to about 40 milliseconds. As you can imagine, this 10,000% speed boost is extremely welcome.