Front-end architect's breaking skills, NodeJS implements WebSocket practice

The source of this article is the public number: Programmer succeeded

This article introduces the implementation of WebSocket in Node.js from the perspective of network protocol, technical background, security and production applications.

Outline preview

The content presented in this article includes the following aspects:

• network protocol evolution
• Socket.IO？
• ws module implementation
• Express integration
• WebSocket instance
• message broadcast
• Security and Authentication
• BFF application

network protocol evolution

The HTTP protocol is the most familiar network communication protocol on the front end. We usually open web pages and request interfaces, which are all HTTP requests.

The characteristics of an HTTP request are: request->response. The client initiates a request, the server responds after receiving the request, and a request is completed. That is to say, the HTTP request must be initiated by the client before the server can respond passively.

In addition, before initiating an HTTP request, a TCP connection needs to be established through a three-way handshake. The feature of HTTP/1.0 is that each communication has to go through a "three-step" process - TCP connection -> HTTP communication -> disconnection of TCP connection.

Each such request is independent, and the connection will be disconnected once the request is completed.

HTTP1.1 optimizes the request process. After the TCP connection is established, we can conduct multiple HTTP communications, and wait until there is no HTTP request to TCP for a period of time before disconnecting the .

But even so, the communication method is still initiated by the client and responded by the server. This fundamental logic will not change.

With the complexity of application interaction, we found that in some scenarios, it is necessary to obtain server-side messages in real time.

For example, instant chat, such as message push, the user does not actively initiate a request, but when the server has a new message, the client needs to know immediately and give feedback to the user.

HTTP does not support active push by the server, but these scenarios urgently need solutions, so polling (polling) appeared in the early days. Polling is that the client periodically initiates a request to the server to detect whether there is an update on the server, and if so, returns new data.

Although this polling method is simple and rude, it obviously has two disadvantages:

1. request consumes too much . Clients keep making requests, wasting traffic and server resources, and putting pressure on the server.
2. cannot guarantee timely . The client needs to balance timeliness and performance, and the request interval must not be too small, so there will be delays.

With the introduction of WebSocket in HTML5, the instant messaging scene finally ushered in a fundamental solution. WebSocket is a full-duplex communication protocol. After the client establishes a connection with the server, the two parties can send data to each other. In this case, the client does not need to obtain data in an inefficient way of polling, and the server pushes new messages directly. to the client.

The traditional HTTP connection method is as follows:

## 普通连接
http://localhost:80/test
## 安全连接
https://localhost:80/test

WebSocket is another protocol, the connection is as follows:

## 普通连接
ws://localhost:80/test
## 安全连接
wss://localhost:80/test

However, WebSocket is not completely separated from HTTP. To establish a WebSocket connection, the client must actively initiate an HTTP request to establish a connection. After the connection is successful, the client and the server can conduct two-way communication.

Socket.IO？

When it comes to implementing WebSocket with Node.js, everyone will definitely think of a library: Socket.IO

Yes, Socket.IO is currently the best choice for Node.js to develop WebSocket applications in a production environment. It is powerful, high performance, low latency, and can be integrated into the express framework in one step.

But maybe you don't know, Socket.IO is not a pure WebSocket framework. It encapsulates Websocket, polling mechanism and other real-time communication methods into a common interface to achieve more efficient two-way communication.

Strictly speaking, Websocket is only part of Socket.IO.

Maybe you will ask: Since Socket.IO has done so many optimizations on the basis of WebSocket and is very mature, why should you build a native WebSocket service?

First, Socket.IO cannot connect via the native ws protocol. For example, if you try to connect to the Socket.IO service through ws://localhost:8080/test-socket in the browser, it cannot be connected. Because the Socket.IO server must be connected through the Socket.IO client, the default WebSocket connection is not supported.

Second, Socket.IO has a very high degree of encapsulation, and using it may not help you understand the principles of WebSocket connection establishment.

Therefore, in this article, we use the basic ws module in Node.js to implement a native WebSocket service from scratch, and use the ws protocol to connect directly at the front end to experience the feeling of two-way communication!

ws module implementation

ws is the next simple, fast, and highly customized WebSocket implementation for Node.js, including both server and client.

The server built with ws can be directly connected by the browser through the native WebSocket constructor, which is very convenient. The ws client is a WebSocket constructor that simulates a browser and is used to connect to other WebSocket servers for communication.

Note: ws can only be used in Node.js environment, not available in browsers. For browsers, please use the native WebSocket constructor .

The following starts to access, the first step is to install ws:

$ npm install ws

After installation, we first build a ws server.

Server

Building a websocket server requires the WebSocketServer constructor.

const { WebSocketServer } = require('ws')
const wss = new WebSocketServer({
  port: 8080
})
wss.on('connection', (ws, req) => {
  console.log('客户端已连接：', req.socket.remoteAddress)
  ws.on('message', data => {
    console.log('收到客户端发送的消息：', data)
  })
  ws.send('我是服务端') // 向当前客户端发送消息
})

Write this code into ws-server.js and run:

$ node ws-server.js

Such a WebSocket server listening on port 8080 is already running.

client

The WebSocket server was built in the previous step, now we connect and listen for messages on the front end:

var ws = new WebSocket('ws://localhost:8080')

ws.onopen = function(mevt) {
  console.log('客户端已连接')
}
ws.onmessage = function(mevt) {
  console.log('客户端收到消息: ' + evt.data)
  ws.close()
}
ws.onclose = function(mevt) {
  console.log('连接关闭')
}

Write the code into wsc.html and open it with a browser, and see the print as follows:

It can be seen that after the browser is successfully connected, it receives a message actively pushed by the server, and then the browser can actively close the connection.

In the Node.js environment, let's see how the ws module initiates the connection:

const WebSocket = require('ws')
var ws = new WebSocket('ws://localhost:8080')

ws.on('open', () => {
  console.log('客户端已连接')
})
ws.on('message', data => {
  console.log('客户端收到消息: ' + data)
  ws.close()
})
ws.on('close', () => {
  console.log('连接关闭')
})

The code is exactly the same as the browser's logic, but the writing method is slightly different, pay attention to the difference.

One point that needs special explanation is that the browser side listens to the callback function of the message event. The parameter is an instance object of MessageEvent . The actual data sent by the server needs to be obtained through mevt.data .

In the ws client, this parameter is the actual data of the server, which can be obtained directly.

Express integration

The ws module is generally not used alone, and a better solution is to integrate it into an existing framework. In this section we integrate the ws module into the Express framework.

The advantage of integrating into the Express framework is that we do not need to monitor a single port, just use the port started by the framework, and we can also specify access to a certain route before initiating a WebSocket connection.

Fortunately, all this does not need to be implemented manually, the express-ws module has already done most of the integration work for us.

First install, then import in the entry file:

var expressWs = require('express-ws')(app)

Like Express's Router, express-ws also supports registering global and local routes.

First look at the global routing, connect through [host]/test-ws :

app.ws('/test-ws', (ws, req) => {
  ws.on('message', msg => {
    ws.send(msg)
  })
})

Local routes are sub-routes registered under a routing group . Configure a routing group named websocket and point to the websocket.js file, the code is as follows:

// websocket.js
var router = express.Router()

router.ws('/test-ws', (ws, req) => {
  ws.on('message', msg => {
    ws.send(msg)
  })
})

module.exports = router

This sub-route can be accessed by connecting to [host]/websocket/test-ws .

The role of the routing group is to define a websocket connection group, and different sub-routes under this group are connected to different requirements. For example, you can set single chat and group chat as two sub-routes to handle their respective connection communication logic.

The complete code is as follows:

var express = require('express')
var app = express()
var wsServer = require('express-ws')(app)
var webSocket = require('./websocket.js')

app.ws('/test-ws', (ws, req) => {
  ws.on('message', msg => {
    ws.send(msg)
  })
})

app.use('/websocket', webSocket)

app.listen(3000)

Small methods to obtain common information in actual development:

// 客户端的IP地址
req.socket.remoteAddress
// 连接参数
req.query

WebSocket instance

The WebSocket instance refers to the client connection object, and the first parameter of the server connection.

var ws = new WebSocket('ws://localhost:8080')
app.ws('/test-ws', (ws, req) => {}

ws in the code is the WebSocket instance, which means the established connection.

browser

The information contained in the browser's ws object is as follows:

{
  binaryType: 'blob'
  bufferedAmount: 0
  extensions: ''
  onclose: null
  onerror: null
  onmessage: null
  onopen: null
  protocol: ''
  readyState: 3
  url: 'ws://localhost:8080/'
}

First and foremost are the four listening properties, which are used to define functions:

• onopen : function after connection is established
• onmessage : The function that receives the push message from the server
• onclose : function for connection close
• onerror : Connection exception function

The most commonly used is the onmessage attribute, which is assigned as a function to listen for server messages:

ws.onmessage = mevt => {
  console.log('消息：', mevt.data)
}

There is also a key attribute is readyState , indicating the connection status, the value is a number. And each value can be represented by constant, the corresponding relationship and meaning are as follows:

• 0 : constant WebSocket.CONNECTING , indicating connecting
• 1 : constant WebSocket.OPEN , means connected
• 2 : constant WebSocket.CLOSING , indicating closing
• 3 : constant WebSocket.CLOSED , indicating closed

Of course, the most important is the send method for sending information and sending data to the server:

ws.send('要发送的信息')

Server

The ws object of the server represents a client currently initiating a connection, and the basic properties are roughly the same as those of the browser.

For example, the four listening attributes of the client above, the readyState attribute, and the send method are all the same. However, because the server is implemented in Node.js, there will be richer support.

For example, the following two types of listening events have the same effect:

// Node.js 环境
ws.onmessage = str => {
  console.log('消息：', str)
}
ws.on('message', str => {
  console.log('消息：', mevt.data)
})

For detailed attributes and introductions, please refer to official document

message broadcast

The WebSocket server does not have only one client connection. Message broadcasting means sending information to all connected clients, like a loudspeaker, which everyone can hear. The classic scenario is hotspot push.

So before broadcasting, a problem must be solved, how does obtain the currently connected (online) client ?

In fact, the ws module provides a quick access method:

var wss = new WebSocketServer({ port: 8080 })
// 获取所有已连接客户端
wss.clients

Be convenient. Let's see how to get express-ws :

var wsServer = expressWebSocket(app)
var wss = wsServer.getWss()
// 获取所有已连接客户端
wss.clients

After getting wss.clients , let's see what it looks like. After printing, it is found that its data structure is simpler than expected, which is a Set collection composed of WebSocket instances of all online clients.

Then, to get the current number of online clients:

wss.clients.size

Simple and rude implementation of broadcasting:

wss.clients.forEach(client => {
  if (client.readyState === 1) {
    client.send('广播数据')
  }
})

This is a very simple, basic implementation. Imagine if there are 10,000 online customers at the moment, then this loop will probably be stuck. Therefore, there are libraries like socket.io, which have done a lot of optimization and encapsulation of basic functions to improve concurrent performance.

The above broadcast belongs to the global broadcast, which is to send the message to everyone. However, there is another scenario, such as a group chat of 5 people, the broadcast at this time is only to send messages to the small group of 5 people, so this is also called local broadcast.

The implementation of local broadcasting is more complicated, and it generally combines specific business scenarios. This requires us to persist the client data when the client connects. For example, use Redis store the status and data of the online client, so that retrieval and classification are faster and more efficient.

Local broadcast is implemented, and that one-on-one private chat is easier. Just find the WebSocket instances corresponding to the two clients and send messages to each other.

Security and Authentication

The WebSocket server built earlier can be connected by any client by default, which is definitely not acceptable in a production environment. We need to ensure the security of the WebSocket server, mainly from two aspects:

1. Token connection authentication
2. wss support

The following is my implementation idea.

Token connection authentication

We generally do JWT authentication for the HTTP request interface, with a specified Header in the request header, and pass a token string in the past.

As we said above, the first step for WebSocket to establish a connection is for the client to initiate an HTTP connection request, then we verify this HTTP request. If the verification fails, the middle WebSocket connection will be created, isn't that ok?

Following this line of thought, let's transform the server-side code.

Because the verification needs to be done at the HTTP layer, the http module is used to create a server, and the port of the WebSocket service is closed.

var server = http.createServer()
var wss = new WebSocketServer({ noServer: true })

server.listen(8080)

When the client connects to the server through ws:// , the server will upgrade the protocol, that is, the http protocol will be upgraded to the websocket protocol, and the upgrade event will be triggered:

server.on('upgrade', (request, socket) => {
  // 用 request 获取参数做验证
  // 1. 验证不通过判断
  if ('验证失败') {
    socket.write('HTTP/1.1 401 Unauthorized\r\n\r\n')
    socket.destroy()
    return
  }
  // 2. 验证通过，继续建立连接
  wss.handleUpgrade(request, socket, _, ws => {
    wss.emit('connection', ws, request)
  })
})

// 3. 监听连接
wss.on('connection', (ws, request) => {
  console.log('客户端已连接')
  ws.send('服务端信息')
})

In this way, the server authentication is added, and the specific authentication method is determined according to the parameter transmission method of the client.

The WebSocket client connection does not support custom headers, so the JWT scheme cannot be used. There are two available schemes:

• Basic Auth
• Quary pass parameters

Basic Auth authentication is simply account + password authentication, and the account password is carried in the URL.

Suppose I have an account ruims and password 123456 , then the client connection is like this:

var ws = new WebSocket('ws://ruims:123456@localhost:8080')

Then the server will receive such a request header:

wss.on('connection', (ws, req) => {
  if(req.headers['authorization']) {
    let auth = req.headers['authorization']
    console.log(auth)
    // 打印的值：Basic cnVpbXM6MTIzNDU2
  }
}

Among them, cnVpbXM6MTIzNDU2 is the base64 code of ruims:123456 , and the server can obtain this code for authentication.

Quary is relatively simple to pass parameters, that is, ordinary URL parameters, you can take a shorter encrypted string in the past, and the server obtains the string and then performs authentication:

var ws = new WebSocket('ws://localhost:8080?token=cnVpbXM6MTIzNDU2')

The server gets the parameters:

wss.on('connection', (ws, req) => {
  console.log(req.query.token)
}

wss support

The WebSocket client uses the ws:// protocol to connect, what does wss mean?

In fact, it is very simple, exactly the same as the principle of https.

https represents the secure http protocol, the composition is HTTP + SSL

wss means the secure ws protocol, the composition is WS + SSL

So why must wss be used? In addition to security, there is another key reason: if your web application is https protocol, you must use WebSocket in your current application to be wss protocol, otherwise the browser will refuse the connection.

To configure wss, add a location directly to the https configuration, and configure it directly on nginx:

location /websocket {
  proxy_pass http://127.0.0.1:8080;
  proxy_redirect off;
  proxy_http_version 1.1;
  proxy_set_header Upgrade $http_upgrade;
  proxy_set_header Connection upgrade;
}

Then the client connection becomes like this:

var ws = new WebSocket('wss://[host]/websocket')

BFF application

You may have heard of BFF, the full name is Backend For Frontend , which means the back-end serving the front-end. In the actual application architecture, it belongs to a middle layer between the front-end and the back-end.

This middle layer is generally implemented by Node.js, so what does it do?

As we all know, the mainstream architecture of the back-end is microservices. In the case of microservices, APIs will be very finely divided. Commodity services are commodity services, and notification services are notification services. When you want to send a notification to the user when the product is on the shelf, you may need to adjust at least two interfaces.

This is actually unfriendly to the front-end, so the BFF middle layer appeared later, which is equivalent to an intermediate proxy station for back-end requests. The front-end can directly request the BFF interface, and then the BFF requests the back-end interface to transfer the required data. Combined, one return to the front end.

So how do we apply a lot of WebSocket knowledge above in the BFF layer?

There are at least 4 application scenarios that I can think of:

1. View the current online number, online user information
2. Log in to new device, log out of other devices
3. Detect network connection/disconnection
4. In-site news, small dot reminder

These functions were previously implemented on the backend and would be coupled with other business functions. Now with BFF, WebSocket can be fully implemented at this layer, allowing the backend to focus on core data logic.

It can be seen that, after mastering the practical application of WebSocket in Node.js, as a front-end, we can break the inner volume and continue to play value in another field. Isn't it beautiful?

Source code + Q&A

All the code in this article has been practiced by me. In order to facilitate the review and experimentation of my friends, I have built a GitHub repository to store the complete source code of this article and the complete source code of subsequent articles.

The warehouse address is here: Yang Chenggong's blog source code

You are welcome to check and test. If you have any doubts, please add me on WeChat ruidoc for consultation, and all your thoughts and ideas about the practice of WebSocket are welcome to communicate with me~