← til

Partitioning parallel tests on Heroku CI

February 24, 2020
ruby

Running tests in parallel on Heroku CI can greatly reduce test suite execution time. Spinning up multiple nodes is as simple as changing app.json to something like this:

"environments": {
  "test": {
    "formation": {
      "test": {
        "quantity": 5
      }
    }
  }
}

The next step after adding multiple nodes is deciding which tests to run on each node. Heroku's official documentation recommends to use either ci_queue by Shopify or Knapsack.

Setting up each of them is pretty complicated, and in case you want something really simple, here's a Ruby script for partitioning tests.

#!/usr/bin/env ruby

tests = Dir["test/**/*_test.rb"].
          sort.
          select.
          with_index do |_, index|
            index % ENV["NUMBER_OF_NODES"].to_i ==
              ENV["CI_NODE_INDEX"].to_i
          end

exec "rails test #{tests.join(" ")}"

This doesn't split the tests evenly by execution time, like Knapsack does, but it provides simple partitioning without too much fuss.


Did you like this article? You can follow me on Twitter or subscribe to the RSS feed if you'd like to follow along.