Running tests in parallel on Heroku CI can greatly reduce
test suite execution time. Spinning up multiple nodes is as simple as changing app.json
to something like this:
"environments": {
"test": {
"formation": {
"test": {
"quantity": 5
}
}
}
}
The next step after adding multiple nodes is deciding which tests to run on each node. Heroku's official documentation recommends to use either ci_queue by Shopify or Knapsack.
Setting up each of them is pretty complicated, and in case you want something really simple, here's a Ruby script for partitioning tests.
#!/usr/bin/env ruby
tests = Dir["test/**/*_test.rb"].
sort.
select.
with_index do |_, index|
index % ENV["NUMBER_OF_NODES"].to_i ==
ENV["CI_NODE_INDEX"].to_i
end
exec "rails test #{tests.join(" ")}"
This doesn't split the tests evenly by execution time, like Knapsack does, but it provides simple partitioning without too much fuss.