How to scale a Node.js server: isolate expensive tasks in forked process (Technical)

Author: | Created on: Feb 27th 2019 | Last edited on: 17 days ago


Async

This is a practical tutorial for web developers who are using or about to use a Node server in production. In this tutorial, you will write an expensive task that runs in a main and forked process. You will see how the main process gets blocked and stops responding to incoming requests. And how the forked process solves this problem by keeping the main process unblocked.

This tutorial assumes some basic knowledge of HTTP and Express. For those of you who like to run the app locally, here is the code launch .

  • Clone repo to your local machine with:
    git clone git@github.com:builderbook/builderbook.git
  • Inside the 6-end-forked-process folder, run yarn or npm install to install all packages listed in 6-end-forked-process/package.json.

For those of you who don’t have time to run app locally, I deployed the app at https://forked-process.builderbook.org launch .


Node’s blessing and curse

On a high level, Node’s strength and weakness comes from the same fact: the main process is single-threaded launch :

  • The advantage of a single-threaded design is higher performance, since multiple threads consume more CPU, memory, space and require context switching launch . With a single thread, more of your server’s resources are devoted to processing requests from the client.
  • The disadvantage of a single-threaded design is that the main process may get blocked by an expensive task that takes a long time to execute. In other words, Node handles a high number of non-expensive tasks well but gets blocked by expensive tasks.
    In this tutorial, we will run the same expensive task inside a main and a forked process. To make the data flow more realistic, we will create a page with two buttons. Clicking the first button will run an expensive task inside the main process. Clicking the second button will run an expensive task inside the forked process.

When a user clicks either button, the following happens:

  • The client sends a GET request to the server’s API endpoint.
  • An Express route handles this request and calls a method that executes an expensive task.
  • This expensive computational task runs either inside the main process or inside the forked process.

Set up an internal API for this tutorial

The app for this tutorial has only one page. To see the this page, either start the app launch locally with yarn dev or navigate to https://forked-process.builderbook.org launch .

The only page in the app looks like this.
Click buttons to execute an expensive task in either main or forked Node process.

Async

Let’s discuss the actual code.


Page

I wrote the Index page of the app with React and Material-UI. I won’t explain how the page was written, but feel free to inspect the actual code launch . I will also skip explaining how to write the API method sendRequest from lib/api/public.js. sendRequest method uses the fetch method to send a request and receive a response. I discussed both of these topics in my previous tutorial at freeCodeCamp launch and in our book launch .

When a user clicks on the grey button that says Execute task in main Node process, the app calls the mainProcessButton function that, in turn, calls and waits for the sendRequestToMainProcess method.

When a user clicks on the grey button that says Execute task in forked Node process, the app calls the forkedProcessButton function that, in turn, calls and waits for the sendRequestToForkedProcess method.

Excerpt from pages/index.js:


mainProcessButton = async () => {
  try {
    const sum = await sendRequestToMainProcess();
    alert(`Calculated by main process: ${sum}`); //eslint-disable-line
  } catch (err) {
    console.log(err); //eslint-disable-line
  }
};

forkedProcessButton = async () => {
  try {
    const sum = await sendRequestToForkedProcess();
    alert(`Calculated by forked process: ${sum}`); //eslint-disable-line
  } catch (err) {
    console.log(err); //eslint-disable-line
  }
};

<Button variant="raised" color="secondary" onClick={this.mainProcessButton}>
  Execute task in main Node process
</Button>

<Button variant="raised" color="primary" onClick={this.forkedProcessButton}>
  Execute task in forked Node process
</Button>

You may notice from the above code that when the client receives a response from the server — an alert modal (API method alert) will show up and show the result of our expensive calculation.
Alert modal indicates that response from server was successful.

Async

API methods

The method sendRequestToMainProcess uses sendRequest from lib/api/public.js to send a GET request to the /api/v1/public/main-process API endpoint and then wait for a response.

Similarly, the method sendRequestToForkedProcess uses sendRequest from lib/api/public.js to send a GET request to a different API endpoint, /api/v1/public/main-process, and also wait for response.

API methods responsible for sending a request to the server (defined at lib/api/public.js):

export const sendRequestToMainProcess = () =>
  sendRequest('/api/v1/public/main-process', {
    method: 'GET',
  });

export const sendRequestToForkedProcess = () =>
  sendRequest('/api/v1/public/forked-process', {
    method: 'GET',

Express routes

On the server, I wrote two Express routes to handle requests for each API method from above. The first Express route simply passes the variable limit to a longComputation function and waits for this function to return sum. When sum is returned, the route sends a response to the client with res.json(sum). I defined the function longComputation at server/longComputation.js:

export default function longComputation(limit) {
  let sum = 0;
  for (let i = 0; i < limit; i += 1) {
    sum += i;
  }
  return sum;
}

Both Express routes from server/app.js:


server.get('/api/v1/public/main-process', async (req, res) => {
  try {
    const limit = 3e9;
    const sum = await longComputation(limit);
    res.json(sum);
    // console.log('sent sum to client');
  } catch (err) {
    res.json({ error: err.message || err.toString() });
  }
});

server.get('/api/v1/public/forked-process', async (req, res) => {
  try {
    const forked = fork('server/forked-longComputation.js');
    const limit = 3e9;
    forked.send(limit);
    forked.on('message', (sum) => {
      res.json(sum);
      // console.log('sent sum to client');
      forked.kill();
    });
  } catch (err) {
    res.json({ error: err.message || err.toString() });
  }
});

The second Express routes does not pass limit to the longComputation function. Instead, this route creates a forked process using fork from the child_processes module of Node. Check out server/app.js( link launch ), find this import line:

import { fork } from 'child_process';

In the next section, let’s have a deeper look at what’s going on in the second Express route.


Main process creates forked process, sends data to it, receives data from it, and kills it

Let’s discuss in detail what we did inside the Express route with the API endpoint /api/v1/public/forked-process.

Our code does the following:

  • creates forked forked/child process with child_process.fork('modulePath’):

    import { fork } from 'child_process';
    const forked = fork('./server/forked-longComputation.js');

    The method child_process.fork() creates a new Node process (called forked process, a type of child process) and establishes an inter-process communication channel (IPC) between this forked process and main process. For a deeper dive, check out Node docs launch .

  • sends limit to the forked process using subprocess.send():

    forked.send(limit);

    The method subprocess.send(message) sends messages from the main/parent process to the forked/child process. In our case, we send a number, limit. See Node docs launch to learn about the subprocess.send() method.

  • helps the main process receive sum from the forked process via subprocess.on(eventName, listener):

    forked.on('message', (sum) => {
    // some code
    });

The main process receives sum from the forked process via forked.on('message'). The forked (forked) process is an emitter of events.

When the forked process uses process.send(sum) (code at server/forked-longComputation.js, discussed below) to send sum to the main process, an event message is emitted. To receive and handle message from the forked process, the main process listens for this message event and executes some function via forked.on('message', (sum) => ...). This is somewhat similar to an Express route, server.get('path', (req, res) => ...), but the event is message instead of path.

Check out Node docs for event message ( link launch ) and the emitter.on() method ( link launch ).

  • once the main process receives sum from the forked process, the main process sends a response with sum to the client with res.json(). After that, the main process kills the forked process with subprocess.kill():
    forked.on('message', (sum) => {
    res.json(sum);
    forked.kill();
    });

Check Node docs launch that describe how the subprocess.kill() method works.

Now we’ve discussed all code inside the main process. In the next section, we will discuss code that runs inside the forked process.


Forked process gets data from main process, computes sum and sends data back to main process

So far we only looked into code that runs inside main process. The code below runs inside the forked process and is located at server/forked-longComputation.js:

const longComputation = (limit) => {
  let sum = 0;
  for (let i = 0; i < limit; i += 1) {
    sum += i;
  }
  return sum;
};

process.on('message', async (limit) => {
  const sum = await longComputation(limit);
  process.send(sum);
});

The forked process, similar to function inside server/longComputation.js, computes sum. As you just learned, the forked process is an event emitter. The main process is also an event emitter. When the main process sends data to the forked process with forked.send(limit), the event message is emitted. The forked process can receive data and handle it by listening to the message event from process emitter via:

process.on('message', async (limit) => {
  const sum = await longComputation(limit);
  process.send(sum);
});

We are done with our code setup!

To learn about all methods of child processes in Node, check out the official documentation at https://nodejs.org/api/child_process.html launch . I also recommend this overview launch of Node child processes by Samer Buna.


Testing forked process

Start the app with yarn dev. Let’s count the number of node processes. Run the following command in a new terminal:

ps -e|grep node

You will see 4 Node processes: it’s our main process plus background processes from V8 launch to manage garbage collection, compiling, and other background tasks.

Navigate to http://localhost:8000 launch , click the blue button that says Execute task in forked Node process. Before the alert modal shows up with sum, run ps -e|grep node in your terminal. Now you will see 5 Node processes! The extra one is our forked process.

The number of processes goes back to 4 after your browser receives a response that contains sum, since we kill the forked process with forked.kill().

Here is a list of Node processes before I clicked the button (4 processes), right after (5 processes), and after the alert modal shows up (4 processes).
The extra Node process is our newly created process. Main process kills forked process after sending response.

Async

Let’s see how an expensive task blocks the main process and makes the server unavailable to incoming requests. Make sure your app is running or navigate to https://forked-process.builderbook.org launch .
Click button to execute expensive task in either main or forked process

Async
  • Click on the grey button that says Execute task in main Node process.
  • Before the alert modal shows up, click the hyperlink Open page in new tab.
  • While the main process executes the expensive task, the main process is blocked and you cannot load the app’s page! The server is unavailable for incoming requests:
    You can’t load page when main process is blocked.
    Async

Repeat the above three steps but with the blue button that says Execute task in forked Node process. While the forked process executes expensive task, you are able to load the app’s page! The main process is not blocked, since the expensive task runs inside the forked process.
You can load application’s page since main process is not blocked.

Async

Important notes

  • Be careful not to make your server a fork bomb launch (creating too many forked processes that deplete the server’s resources).
  • Check out how we used forked process for expensive tasks in our production app launch .

If you found this article useful, check out our Github repo launch and our book launch , where we cover this and many other topics in detail.

If you are building a software product, check up our SaaS boilerplate launch and Async launch .