The ideal number of workers in Jest

08 October 2018   0 comments   Python, ReactJS

tl;dr; Use --runInBand when running jest in CI and use --maxWorkers=3 on your laptop.

Running out of memory on CircleCI

We have a test suite that covers 236 tests across 68 suites and runs mainly a bunch of enzyme rendering of React component but also some plain old JavaScript function tests. We hit a problem where tests utterly failed in CircleCI due to running out of memory. Several individual tests, before it gave up or failed, reported to take up to 45 seconds.
Turns out, jest tried to use 36 workers because the Docker OS it was running was reporting 36 CPUs.

> circleci@9e4c489cf76b:~/repo$ node
> var os = require('os')
> os.cpus().length

After forcibly setting --maxWorkers=2 to the jest command, the tests passed and it took 20 seconds. Yay!

But that got me thinking, what is the ideal number of workers when I'm running the suite here on my laptop? To find out, I wrote a Python script that would wrap the call CI=true yarn run test --maxWorkers=%(WORKERS) repeatedly and report which number is ideal for my laptop.

After leaving it running for a while it spits out this result:

3 8.47s
4 8.59s
6 9.12s
5 9.18s
2 9.51s
7 10.14s
8 10.59s
1 13.80s

The conclusion is vague. There is some benefit to using some small number greater than 1. If you attempt a bigger number it might backfire and take longer than necessary and if you do do that your laptop is likely to crawl and cough.

Notes and conclusions


Your email will never ever be published

Related posts

Inline scripts in create-react-app 2.0 and CSP hashes 05 October 2018
Fancy linkifying of text with Bleach and domain checks (with Python) 10 October 2018
Related by Keyword:
Experimenting with Nginx worker_processes 14 February 2019
Best EXPLAIN ANALYZE benchmark script 19 April 2018
Fastest *local* cache backend possible for Django 04 August 2017
How to do performance micro benchmarks in Python 24 June 2017
Why didn't I know about machma?! 07 June 2017