Benchmarking Node.js Worker Threads

NodeJS official documentation states that there is no real benefit of using worker threads for I/O, but wanted to benchmark it to understand the difference.

Setup

We used Benchmark.js for benchmarking and piscina as a pool for worker threads. benchmark.js was used to run the same code in 2 scenarios - one using a single thread and one using the piscina pool. The degree of parallelism was passed to the program via an environment variable. The test code is present in worker.js in both the cases.

These tests were run on a Macbook Pro (13-inch, 2020, Intel CPU) with 2.3 GHz Quad-Core Intel Core i7 (8 CPU cores) and 16GB of memory. The tests were run from an embedded terminal in VSCode. No other foreground processes were running.

Httpbin was used for testing I/O. This has the disadvantage of being slow vs a locally hosted mock server but reduces noise as I didn't want a competing server process sharing the same resources.

Copycopy code to clipboard
// benchmark.js
const path = require("path"),
  Benchmark = require("benchmark"),
  suite = new Benchmark.Suite(),
  Piscina = require("piscina"),
  pool = new Piscina({
    idleTimeout: 5000,
    filename: path.resolve(__dirname, "./worker.js"),
  }),
  method = require("./worker");

const parallelism = parseInt(process.env.P);

suite
  .add("single thread", {
    defer: true,
    fn: async function (deferred) {
      const promises = [];
      for (let i = 0; i < parallelism; i++) {
        promises.push(method());
      }
      await Promise.all(promises);
      deferred.resolve();
    },
  })
  .add("worker threads", {
    defer: true,
    fn: async function (deferred) {
      const promises = [];
      for (let i = 0; i < parallelism; i++) {
        promises.push(pool.run());
      }
      await Promise.all(promises);
      deferred.resolve();
    },
  })
  .on("cycle", function (event) {
    console.log(String(event.target));
  })
  .on("complete", function () {
    console.log("Done");
  })
  .run();

Default options for piscina.

Copycopy code to clipboard
{
	"filename": "worker.js",
	"name": "default",
	"minThreads": 4,
	"maxThreads": 12,
	"idleTimeout": 5000,
	"maxQueue": null,
	"concurrentTasksPerWorker": 1,
	"useAtomics": true,
	"taskQueue": {
		"tasks": []
	},
	"niceIncrement": 0,
	"trackUnmanagedFds": true
}

Only I/O

Send an HTTP request to an endpoint.

Copycopy code to clipboard
// worker.js
const request = require('request-promise');

module.exports = () => {
  return request('https://httpbin.org/get');
};

Observations

• No significant performance benefits over using a single thread.
• Multi-threaded performance starts to degrade as parallelism increases beyond maxThreads.
• Single thread performance also takes a hit as parallelism increases, but owing to large standard deviation, this could be because of server performance too.

CPU and I/O

Send an HTTP request to an endpoint after calculating fibbonacci recursively.

Copycopy code to clipboard
// worker.js
const request = require('request-promise');

function fibonacci(n) {
    if (n < 2)
      return 1;
    else
      return fibonacci(n - 2) + fibonacci(n - 1);
}

module.exports = async () => {
    fibonacci(20);
    await request('https://httpbin.org/get');
};

Observations

• I/O trumps CPU work, maybe a larger fibbonacci number could've provided different results.
• Using worker threads is slightly better when parallelism is less than maxThreads but beyond that no advantage.

Only CPU

Calculate fibbonacci recursively.

Copycopy code to clipboard
// worker.js
function fibonacci(n) {
    if (n < 2)
      return 1;
    else
      return fibonacci(n - 2) + fibonacci(n - 1);
}

module.exports = async () => {
  fibonacci(20);
};

Observations

• For CPU intensive work, use worker threads.

Conclusion

• Worker threads for pure I/O based work do not provide any significant performance improvements. At higher parallelism, it performs worse than a single thread.
• Worker threads provide significant performance benefits for CPU intensive work.
• For mixed workloads, YMMV. There could be a minor performance bump as the CPU intensive work is offloaded to threads, but it depends on the time spent in CPU vs I/O.
• Worker threads work well when parallelism is less than the number of CPU cores on the machine. Beyond that, performance starts to dip as the pool starts to queue work.