An article to understand how Node handles CPU-intensive tasks

Background introduction

We have heard more or less the following words in our daily work:

Node is a 非阻塞I/O (non-blocking I/O) and 事件驱动 (event-driven) JavaScript运行环境 (runtime), so it's great for building I/O intensive applications, such as web services, etc.

I don’t know if you will have the same doubts as me when you hear similar words: 单线程的Node为什么适合用来开发I/O密集型应用？ it reasonable to say that languages that support multithreading (such as Java and Golang) are more advantageous to do these jobs? ?

To understand the above problem, we need to know what Node's single thread refers to.

Node is not single threaded

In fact, when we say that Node is single-threaded, we only mean that our JavaScript代码 is running in the same thread (we can call it 主线程 ), 而不是说Node只有一个线程在工作 . In fact, the bottom layer of Node will use libuv's 多线程能力 to put part of the work (basically I/O related operations) in some 主线程之外 threads to execute, when these tasks are completed, 回调函数 returns the result to the JavaScript execution environment of the main thread. Take a look at the schematic:

Note: The above picture is a simplified version of Node 事件循环 (Event Loop). In fact, the complete event loop will have more stages such as timers.

Node is suitable for I/O intensive applications

From the above analysis, we know that Node will distribute all I/O operations to different threads through libuv's multi-threading capability, and the rest of the operations are performed in the main thread. So why is this approach more suitable for I/O intensive applications than other languages such as Java or Golang? Let's take the development of web services as an example, mainstream back-end programming languages such as Java and Golang are 并发模型是基于线程 (Thread-Based), which means that they will create a 单独的线程 for each network request. 单独的线程 to deal with. But for web applications, it is mainly for 数据库的增删改查，或者请求其它外部服务等网络I/O操作 , and these operations are finally handed over to 操作系统的系统调用来处理的(无需应用线程参与)，并且十分缓慢(相对于CPU时钟周期来说) , so most of the time the created thread is 无事可做 and our service also bears additional 线程切换 overhead. Unlike these languages, Node does not create a thread for each request, 所有请求的处理 all happens in the main thread, so there is no 线程切换 overhead, and it will pass 线程池 process these I/O operations asynchronously, and then tell the main thread the result in the form of an event to avoid blocking the execution of the main thread, so it 理论上 is more efficient of. It's worth noting here that I'm just saying that Node 理论上 is faster, not necessarily. This is because in reality, the performance of a service will be affected by many aspects. We only consider one factor here 并发模型 , and other factors such as runtime consumption will also affect the performance of the service, for example , JavaScript is a dynamic language, the data type needs to be inferred at runtime, while Golang and Java are static languages. Their data types are set at compile time That's for sure, so they might actually execute faster and use less memory.

Node is not suitable for CPU-intensive tasks

We mentioned above that Node's operations other than I/O-related operations will be executed in the main thread, so when Node needs to process some tasks CPU密集型 , the main thread will be blocked. Let's look at an example of a CPU-intensive task:

 // node/cpu_intensive.js

const http = require('http')
const url = require('url')

const hardWork = () => {
  // 100亿次毫无意义的计算
  for (let i = 0; i < 10000000000; i++) {}
}

const server = http.createServer((req, resp) => {
  const urlParsed = url.parse(req.url, true)

  if (urlParsed.pathname === '/hard_work') {
    hardWork()
    resp.write('hard work')
    resp.end()
  } else if (urlParsed.pathname === '/easy_work') {
    resp.write('easy work')
    resp.end()
  } else {
    resp.end()
  }
})

server.listen(8080, () => {
  console.log('server is up...')
})

在上面的代码中我们实现了拥有两个接口的HTTP服务： /hard_work CPU密集型接口 ， hardWork CPU密集型 function, and /easy_work this interface is very simple, just return a string directly to the client. Why do you say hardWork function is CPU密集型 ? This is because it performs arithmetic operations on i in the CPU 运算器 without any I/O operations. After starting our Node service, we try to call the /hard_word interface:

We can see that the /hard_work interface will be stuck, because it needs a lot of CPU calculation, so it will take a long time to complete. And at this time, let's take a look again /easy_work this interface has any effect:

We found that after /hard_work took up CPU resources, the innocent /easy_work interface was also stuck. The reason is that the hardWork function blocks the main thread of Node and the logic of /easy_work will not be executed. It is worth mentioning here that only a single-threaded execution environment based on an event loop such as Node will have this problem, and Thread-Based languages such as Java and Golang will not have this problem. So what if our service really needs to run the CPU密集型 task? Can't change languages? What about All in JavaScript ? Don't worry, for processing CPU密集型任务 , Node has prepared many solutions for us. Next, let me introduce three commonly used solutions, they are: Cluster Module , Child Process and Worker Thread .

Cluster Module

Concept introduction

Node introduced the Cluster module very early (v0.8). The role of this module is through 一个父进程启动一群子进程来对网络请求进行负载均衡 . Due to the space limitation of the article, we will not discuss in detail what APIs the Cluster module has. Interested readers can look at the official documentation later. Here we directly look at how to use the Cluster module to optimize the above CPU-intensive scenarios:

 // node/cluster.js

const cluster = require('cluster')
const http = require('http')
const url = require('url')

// 获取CPU核数
const numCPUs = require('os').cpus().length

const hardWork = () => {
  // 100亿次毫无意义的计算
  for (let i = 0; i < 10000000000; i++) {}
}

// 判断当前是否是主进程
if (cluster.isMaster) {
  // 根据当前机器的CPU核数创建同等数量的工作进程
  for (var i = 0; i < numCPUs; i++) {
    cluster.fork()
  }

  cluster.on('online', (worker) => {
    console.log(`worker ${worker.process.pid} is online`)
  })

  cluster.on('exit', (worker, code, signal) => {
    // 某个工作进程挂了之后，我们需要立马启动另外一个工作进程来替代
    console.log(`worker ${worker.process.pid} exited with code ${code}, and signal ${signal}, start a new one...`)
    cluster.fork()
  })
} else {
  // 工作进程启动一个HTTP服务器
  const server = http.createServer((req, resp) => {
    const urlParsed = url.parse(req.url, true)
  
    if (urlParsed.pathname === '/hard_work') {
      hardWork()
      resp.write('hard work')
      resp.end()
    } else if (urlParsed.pathname === '/easy_work') {
      resp.write('easy work')
      resp.end()
    } else {
      resp.end()
    }
  })
  
  // 所有的工作进程都监听在同一个端口
  server.listen(8080, () => {
    console.log(`worker ${process.pid} server is up...`)
  })
}

In the above code, we use the cluster.fork function to create the same number of 工作进程 according to the number of CPU cores of the current device, and these worker processes are all listening on 8080 above the port. Seeing this, you may ask if there will be any problems with all processes listening on the same port, but it is not, because Cluster the bottom layer of the module will do some work to make the final monitoring in 8080 the port is 主进程 , and the main process is 所有流量的入口 , which receives HTTP connections and hits them to different worker processes. Without further ado, let's run the node service:

From the above output, the cluster starts 10 workers (my computer has 10 cores) to process web requests. At this time, let's request again /hard_work this interface:

We found that this request is still stuck, and then let's see if the Cluster module has solved the problem of 其它请求也被阻塞 :

We can see that the previous 9个请求 all returned the results smoothly, but when we arrived 第10个请求 our interface was stuck, why is this? The reason is that we have a total of 10 worker processes. The default load balancing strategy used by the main process when it sends traffic to the child process is round-robin (in turn), so the 10th request (actually the 11th request) , because the request for the first hard_work is included) just returned to the first worker, and this worker has not finished the task of hard_work , so this task of easy_work is also its stuck. The load balancing algorithm of cluster can be modified by cluster.schedulingPolicy , interested readers can read the official document.

From the above results, the Cluster Module seems to be 解决了一部分 our problem, but some requests are still affected. So can the Cluster Module be used to solve this CPU密集型 task problem in actual development? My opinion is: it depends on the situation. If your CPU intensive interface 调用不频繁 and 运算时间不会太长 , you can use this Cluster Module to optimize. But if your interface is called frequently and each interface is time-consuming, you may need to look at the solution using Child Process or Worker Thread .

Advantages and disadvantages of Cluster Module

Finally, we summarize the advantages of the Cluster Module:

• 资源利用率高 : You can make full use of CPU的多核能力 to improve request processing efficiency.
• API设计简单 : allows you to implement 简单的负载均衡 and 一定程度的高可用 . It is worth noting here that I am talking about a certain degree of high availability. This is because the high availability of the Cluster Module is 单机版的 , that is, when the host machine hangs, your service also hangs, so it is more High availability must be done using distributed clusters.
• 进程之间高度独立 , to prevent a system error in a process from causing the entire service to be unavailable.

The advantages are over, let's talk about the disadvantages of the Cluster Module:

• 资源消耗大 : Each child process is 独立的Node运行环境 , which can also be understood as an independent Node program, so 占用的资源也是巨大的 .
• 进程通信开销大 : The communication between child processes is carried out through 跨进程通信(IPC) , if data sharing is frequent, it is a relatively large overhead.

• 没能完全解决CPU密集任务 : Still a bit 抓紧见肘 for CPU-intensive tasks.

  Child Process

  In the Cluster Module, we can start more sub-processes to balance the load of some CPU-intensive tasks to different processes, so as to avoid the other interfaces from being stuck. But you can also see that this method 治标不治本 , if the user frequently calls the CPU-intensive interface, there will still be a large number of requests will be stuck. Another way to optimize this scenario is the child_process module.

  Concept introduction

  Child Process lets us start 子进程 for some CPU-intensive tasks. Let's first look at the code of the main process master_process.js :

   // node/master_process.js
  
  const { fork } = require('child_process')
  const http = require('http')
  const url = require('url')
  
  const server = http.createServer((req, resp) => {
  const urlParsed = url.parse(req.url, true)
  
  if (urlParsed.pathname === '/hard_work') {
    // 对于hard_work请求我们启动一个子进程来处理
    const child = fork('./child_process')
    // 告诉子进程开始工作
    child.send('START')
    
    // 接收子进程返回的数据，并且返回给客户端
    child.on('message', () => {
      resp.write('hard work')
      resp.end()
    })
  } else if (urlParsed.pathname === '/easy_work') {
    // 简单工作都在主进程进行
    resp.write('easy work')
    resp.end()
  } else {
    resp.end()
  }
  })
  
  server.listen(8080, () => {
  console.log('server is up...')
  })

  In the above code, for the request of the /hard_work interface, we will use the fork function to open a 新的子进程 to process the data. Return the result to the client. It is worth noting here that I did not release the resources of the child process when the child process completed the task. In actual projects, we should not frequently create and destroy child processes because this consumption is also very large. It is better to use 进程池 . The following is the implementation logic of 子进程 (child_process.js):

   // node/child_process.js
  
  const hardWork = () => {
  // 100亿次毫无意义的计算
  for (let i = 0; i < 10000000000; i++) {}
  }
  
  process.on('message', (message) => {
  if (message === 'START') {
    // 开始干活
    hardWork()
    // 干完活就通知子进程
    process.send(message)
  }
  })

  The code of the child process is also very simple. After it starts, it will listen to the message from the parent process in the way of process.on . After receiving the start command, it will perform the calculation of CPU密集型 and get the result. Then return to the parent process.

Running the code above master_process.js , we can find that even if we call the /hard_work interface, we can still call the /easy_work interface and get a response immediately, there is no screenshot here. , you can make up your mind about the process.

除了fork函数， child_process exec spawn子进程，并且这些进程可以执行任何的shell commands are not limited to Node scripts. Interested readers can learn about them through the official documents later, so I won't introduce them too much here.

Advantages and disadvantages of Child Process

Finally, let us summarize the advantages of Child Process :

• 灵活 : Not only limited to Node process, we can execute any shell command in the child process. This is actually a big advantage. If our CPU-intensive operations are using 其它语言实现 (such as c language processing images), and we don't want to use Node or C++ Binding to re-implement it again, we can pass shell command calls programs in other languages, and communicates with them through 标准输入输出 to get the result.
• 细粒度的资源控制 : Unlike the Cluster Module, the Child Process solution can dynamically adjust the number of child processes according to the actual demand for CPU-intensive computing, so as to achieve fine-grained control of resources, so 它理论上 can solve the problem that Cluster Module cannot solve CPU密集型接口调用频繁 .

However, the disadvantages of Child Process are also obvious:

• 资源消耗巨大 ：上面说它可以对资源进行细粒度控制时，也理论上 CPU密集型接口频繁调用的问题 ，这是因为实际In the scenario, our resources are also 有限的 , and each Child Process is an independent operating system process, which consumes huge resources. Therefore, for frequently called interfaces, we need to adopt a solution with lower energy consumption, which is what I will say below Worker Thread .

• 进程通信麻烦 : If the child process started is also a Node application, it is better, because there is 内置的API to communicate with the parent process. If the child process is not a Node application, we can only 标准输入输出 or other ways to communicate between processes, this is a very troublesome thing.

  Worker Thread

  Both Cluster Module and Child Process are actually based on sub-processes, and they all have a huge disadvantage is 资源消耗大 . In order to solve this problem, Node has supported the CPU密集型操作 轻量级的线程解决方案 worker_threads module since the v10.5.0 version (v12.11.0 stable).

  Concept introduction

  Node's Worker Thread is the same as threads in other languages, that is 并发 to run your code. Note here is 并发 not 并行 . 并行 just means 一段时间内多件事情同时发生 , while 并发 is 某个时间点多件事情同时发生 . 并行例子就是React的Fiber架构 ，因为它是通过---54740a2b1975b06e566acd74cfbd244f 时分复用的方式来调度不同的任务来避免React渲染浏览器other behaviors, so essentially all its operations are still performed in 同一个操作系统线程 . But it is worth noting here: Although 并发 emphasizes the simultaneous execution of multiple tasks, in the case of a single-core CPU, 并发会退化为并行 . This is because the CPU can only do one thing at the same time. When you have multiple 线程需要执行的话 , you need to use 资源抢占 to execute certain tasks 时分复用 . But these are all things that the operating system needs to care about, and it has nothing to do with us.

The above mentioned that Node's Worker Thead is similar to the thread of other language threads, and then let's take a look at the differences between them. If you have used multi-threaded programming in other languages, you will find that Node's multi-threading is very different from them, because Node's multi-threading 数据共享起来 is really 太麻烦了 ! Node does not allow you to share data by 共享内存变量 e90a0a1f904db7f51dab998dfd59c6b1---, you can only use ArrayBuffer or SharedArrayBuffer to transfer and share data. Although this is very inconvenient, but it also makes us not need to think too much 多线程环境下数据安全等一系列问题 , it can be said that there are advantages and disadvantages.

Next, let's take a look at how to use Worker Thread to handle the above CPU-intensive tasks, first look at the code of the main thread (master_thread.js):

 // node/master_thread.js

const { Worker } = require('worker_threads')
const http = require('http')
const url = require('url')

const server = http.createServer((req, resp) => {
  const urlParsed = url.parse(req.url, true)

  if (urlParsed.pathname === '/hard_work') {
    // 对于每一个hard_work接口，我们都启动一个子线程来处理
    const worker = new Worker('./child_process')
    // 告诉子线程开始任务
    worker.postMessage('START')
    
    worker.on('message', () => {
      // 在收到子线程回复后返回结果给客户端
      resp.write('hard work')
      resp.end()
    })
  } else if (urlParsed.pathname === '/easy_work') {
    // 其它简单操作都在主线程执行
    resp.write('easy work')
    resp.end()
  } else {
    resp.end()
  }
})

server.listen(8080, () => {
  console.log('server is up...')
})

In the above code, every time our server receives a /hard_work request, it will start a Worker thread through new Worker --- to process the task, and the worker will process the task. Then we return the result to the client, this process is asynchronous. Then take a look at the code implementation of 子线程 (worker_thead.js):

 // node/worker_thread.js

const { parentPort } = require('worker_threads')

const hardWork = () => {
  // 100亿次毫无意义的计算
  for (let i = 0; i < 10000000000; i++) {}
}

parentPort.on('message', (message) => {
  if (message === 'START') {
    hardWork()
    parentPort.postMessage()
  }
})

In the above code, the worker thread starts to execute the CPU密集型 operation after receiving the command of the main thread, and finally informs the parent thread that the task has been completed by parentPort.postMessage . Thread communication is quite convenient.

Advantages and disadvantages of Worker Thread

Finally, let's summarize the advantages and disadvantages of Worker Thread. First of all I think its advantages are:

• 资源消耗小 : Unlike Cluster Module and Child Process, which are based on processes, Worker Thread is based on more lightweight threads, so its resource overhead is 相对较小的 . But 麻雀虽小五脏俱全 , each Worker Thread has its own independent v8引擎实例 and 事件循环 system. This means that even 主线程卡死 our Worker Thread can continue to work, and we can actually do a lot of interesting things based on this.
• 父子线程通信方便高效 : Unlike the previous two methods, Worker Thread does not need to communicate through IPC, and all data is shared and transmitted within the process.

But Worker Thread is not perfect:

• 线程隔离性低 : Since the child thread is not executed in one 独立的环境 , the hanging of a child thread will still affect other threads. In this case, you need to take some additional measures to protect the rest of the threads from being affected.

• 线程数据共享实现麻烦 : Compared with other back-end languages, Node's data sharing is more troublesome, but this actually avoids the need to consider many data security issues under multi-threading.

  Summarize

  In this article, I introduced to you why Node is suitable for I/O-intensive applications but difficult to handle CPU-intensive tasks, and provided you with three alternatives to deal with CPU-intensive tasks in actual development. Task. In fact, each scheme has advantages and disadvantages, we must choose according to the actual situation, 永远不要为了要用某个技术而一定要采取某个方案 .

Personal technology trends

Creation is not easy. If you learn something from this article, please give me a like or follow. Your support is the biggest motivation for me to continue to create!

At the same time, welcome the onions who pay attention to the attack on the public account to learn and grow together