1. Introduction to Load Balancing
1.1. Challenges faced by large websites
Large-scale websites have to face the challenges of a huge number of users, high concurrency, and massive data. In order to improve the overall performance of the system, two methods of vertical expansion and horizontal expansion can be used.
vertical expansion : In the early stage of website development, you can increase the processing capacity of the server from the stand-alone point of view, such as CPU processing capacity, memory capacity, disk, etc., to achieve the improvement of server processing capacity. However, a stand-alone machine has a performance bottleneck. Once the bottleneck is touched, if you want to upgrade it, the cost and price will be extremely high. Obviously, this cannot meet all the challenges of large-scale distributed systems (websites) such as large traffic, high concurrency, and massive data.
horizontal expansion : share the traffic of large websites through clusters. The application servers (nodes) in the cluster are usually designed to be stateless, and users can request any node, and these nodes share the access pressure. There are two main points for horizontal expansion:
- Application cluster: Deploy the same application on multiple machines to form a processing cluster, receive requests distributed by load balancing equipment, process them, and return corresponding data.
- Load balancing: Distribute user access requests to nodes in the cluster through a certain algorithm.
1.2. What is load balancing
Load Balance (LB) is an indispensable key component of a high-concurrency and high-availability system. The goal is to try to evenly distribute network traffic to multiple servers to improve the overall response speed and availability of the system.
The main functions of load balancing are as follows:
high concurrency : Load balancing adjusts the load through algorithms, and tries to evenly distribute the workload of each node in the application cluster, thereby improving the concurrent processing capacity (throughput) of the application cluster.
Scalability : Add or reduce the number of servers, and then load balance for distribution control. This makes application clusters scalable.
Highly available : The load balancer can monitor candidate servers. When the server is unavailable, it will automatically skip and distribute the request to the available servers. This makes application clusters highly available.
security protection : Some load balancing software or hardware provides security functions, such as: black and white list processing, firewall, anti-DDos attack, etc.
Second, the classification of load balancing
There are many technologies that support load balancing, and we can classify them through different dimensions.
2.1 Carrier dimension classification
From the perspective of carriers that support load balancing, load balancing can be divided into two categories: hardware load balancing, software load balancing
2.1.1 Hardware load balancing
Hardware load balancing, generally an independent load balancing server running on a custom processor, is expensive and exclusive to local tyrants. The mainstream products of hardware load balancing are: F5 and A10.
hardware load balancing:
- Powerful: Support global load balancing and provide more comprehensive and complex load balancing algorithms.
- Powerful performance: Because hardware load balancing runs on a dedicated processor, it has a large throughput and can support a single machine with more than one million concurrency.
- High security: often equipped with firewall, anti-DDos attack and other security functions.
hardware load balancing:
- Expensive: The cost of buying and maintaining hardware load balancing is high.
- Poor scalability: When the number of visits increases suddenly, the capacity cannot be dynamically expanded if the limit is exceeded.
2.1.2 Software load balancing
Software load balancing is the most widely used, and it is used by both large and small companies.
Software load balancing implements load balancing at the software level, and can generally run on any standard physical device.
The mainstream products of software load balancing are: Nginx, HAProxy, LVS .
- LVS can be used as a four-tier load balancer. Its load balancing performance is better than Nginx.
- HAProxy can be used as HTTP and TCP load balancer.
- Nginx and HAProxy can be used as a four-layer or seven-layer load balancer.
Advantages of software load balancing:
- Good scalability: Adapt to dynamic changes, and can dynamically expand beyond the initial capacity by adding software load balancing instances.
- Low cost: Software load balancing can run on any standard physical equipment, reducing the cost of purchase and operation and maintenance.
Disadvantages of software load balancing:
- Slightly poor performance: Compared with hardware load balancing, the performance of software load balancing is slightly lower.
2.2 Network communication classification
From the perspective of communication, software load balancing can be divided into four-layer and seven-layer load balancing.
1) seven-layer load balancing : It can forward the request to a specific host according to the HTTP request header and URL information of the visiting user.
- DNS redirect
- HTTP redirect
- Reverse proxy
2) four-layer load balancing : request forwarding based on IP address and port.
- Modify IP address
- Modify MAC address
2.2.1 DNS load balancing
DNS load balancing is generally used in Internet companies, and complex business systems are not suitable for use. Large-scale websites generally use DNS load balancing as the first-level load balancing method, and then use other methods to do the second-level load balancing internally. DNS load balancing is a seven-layer load balancing.
DNS stands for Domain Name Resolution Service and is the seventh layer of OSI network protocol. DNS is designed as a distributed application with a tree structure, from top to bottom: root domain name server, first-level domain name server, second-level domain name server,..., local domain name server. Obviously, if all data is stored in the root domain name server, the load and overhead of DNS queries will be very large.
Therefore, DNS query is a reverse recursive process relative to the DNS hierarchical structure. The DNS client sequentially requests the local DNS server, the upper-level DNS server, the upper-level DNS server,..., the root DNS server (also called authoritative DNS server), once hit, return immediately. In order to reduce the number of queries, each level of DNS server will set up a DNS query cache.
The working principle of DNS load balancing is: Based on the DNS query cache, the IP addresses of different servers are returned according to the load conditions.
Advantages of DNS redirection:
Easy to use: load balancing work is handed over to the DNS server to process, saving the trouble of maintaining the load balancing server
Improve performance: It can support address-based domain name resolution and resolve to the server address closest to the user (similar to the principle of CDN), which can speed up access and improve performance;
Disadvantages of DNS redirection:
Poor usability: DNS resolution is a multi-level resolution. After adding/modifying DNS, the resolution time is longer; during the resolution process, users will fail to access the website;
Low scalability: DNS load balancing is controlled by the domain name provider, and it cannot be improved and extended more;
Poor maintainability: it can't reflect the current running status of the server; it supports few algorithms; it can't distinguish the difference of the server (can't judge the load based on the state of the system and the service).
2.2.2 HTTP load balancing
HTTP load balancing is based on HTTP redirection. HTTP load balancing is a seven-layer load balancing.
The principle of HTTP redirection is: calculate a real server address according to the user's HTTP request, write the server address into the HTTP redirection response, return it to the browser, and the browser will re-access.
Advantages of HTTP redirection: The solution is simple.
Disadvantages of HTTP redirection:
Poor performance: Each visit requires two requests to the server, which increases the latency of the visit.
Decrease search ranking: After using redirects, search engines will consider SEO cheating.
If the load balancer goes down, the site cannot be accessed.
Due to its obvious shortcomings, this load balancing strategy is rarely used in practice.
2.2.3 Reverse proxy load balancing
Reverse Proxy refers to the use of a proxy server to accept network requests, then forward the request to the server in the intranet, and return the results obtained from the server in the intranet to the client of the network request. Reverse proxy load balancing belongs to seven-layer load balancing.
The mainstream products of reverse proxy services: Nginx, Apache .
What is the difference between forward proxy and reverse proxy?
Forward proxy: occurs on the client side and is initiated by the user. Over-the-wall software is a typical forward proxy. The client actively accesses the proxy server, allowing the proxy server to obtain the required external network data, and then forward it back to the client.
Reverse proxy: occurs on the server side, and the user does not know the existence of the proxy.
How does the reverse proxy achieve load balancing? Take Nginx as an example, as shown below:
First, set up the load balancing rules on the proxy server. Then, when receiving a client request, the reverse proxy server intercepts the specified domain name or IP request, and distributes the request to the candidate server according to the load balancing algorithm. Secondly, if a candidate server goes down, the reverse proxy server will be fault-tolerant. For example, if the distribution request fails more than 3 times, the request will be distributed to other candidate servers.
Advantages of reverse proxy:
- Multiple load balancing algorithms: Support multiple load balancing algorithms to meet the needs of different scenarios.
- Server can be monitored: Based on the HTTP protocol, the status of the forwarding server can be monitored, such as: system load, response time, availability, number of connections, traffic, etc., so as to adjust the load balancing strategy based on these data.
Disadvantages of reverse proxy:
- Additional forwarding overhead: The forwarding operation of the reverse proxy itself has performance overhead, which may include operations such as creating a connection, waiting for a connection response, and analyzing the response result.
- Increasing system complexity: Reverse proxy is often used for horizontal expansion of distributed applications, but reverse proxy services have the following problems. In order to solve the following problems, additional complexity and operation and maintenance costs will be added to the system as a whole:
- If the reverse proxy service is down, the site cannot be accessed, so a high-availability solution is required. Common solutions are: active-standby mode (one active and one standby), dual active mode (mutually active and standby).
- The reverse proxy service itself also has performance bottlenecks. As the number of requests that need to be forwarded continues to rise, a scalable solution is needed.
2.2.4 IP load balancing
IP load balancing is to perform load balancing at the network layer by modifying the request destination address.
As shown in the figure above, the IP balance processing flow is roughly as follows:
The client requests 192.168.137.10, and the load balancing server receives the packet.
The load balancing server selects a service node 192.168.0.1 according to the algorithm, and then changes the message request address to the IP of the node.
The real service node receives the request message, processes it, and returns the response data to the load balancing server.
The load balancing server changes the source address of the response data to the address of the load balancing server, and returns it to the client.
IP load balancing completes data distribution in the kernel process, and has better slave processing performance than reverse proxy load balancing. However, since all requests and responses go through the load balancing server, the throughput of the cluster is restricted by the bandwidth of the load balancing server.
2.2.5 Data link layer load balancing
Data link layer load balancing refers to modifying the mac address at the data link layer of the communication protocol for load balancing.
The best link layer load balancing open source product on the Linux platform is LVS (Linux Virtual Server). LVS is a load balancing system based on the netfilter framework in the Linux kernel. Netfilter is a kernel-mode Linux firewall mechanism, which can set several checkpoints (hook functions) according to rules to perform related operations during the flow of data packets.
The workflow of LVS is roughly as follows:
When the user visits www.sina.com.cn , the user data passes through the layer-by-layer network, and finally enters the LVS server network card through the switch, and enters the kernel network layer.
After entering PREROUTING, after routing search, it is determined that the destination VIP of the visit is the local IP address, so the data packet enters the INPUT chain
IPVS works on the INPUT chain. It will vip+port . If it is, the registered IPVS HOOK function will be called to perform the IPVS related main process, forcibly modify the relevant data of the data packet, and change the data. The package is sent to the POSTROUTING chain.
After receiving the data packet on POSTROUTING, according to the target IP address (back-end server), through routing, the data packet is finally sent to the back-end server.
The open source LVS version has 3 working modes. Each mode has a completely different working principle. It is said that each mode has its own advantages and disadvantages, which are suitable for different application scenarios, but the ultimate essential function is to achieve balanced traffic scheduling and good The scalability. It mainly includes three modes: DR mode, NAT mode, and Tunnel mode.
Three, load balancing algorithm
The implementation of the load balancer can be divided into two parts:
Select a server from the list of candidate servers according to the load balancing algorithm;
Send the requested data to the server.
The load balancing algorithm is the core of the load balancing service core. There are many kinds of load balancing products, but the principles of various load balancing algorithms are common. There are many load balancing algorithms, which are suitable for different application scenarios. This article only introduces the characteristics and principles of the most common load balancing algorithms: polling, random, minimum active number, source address hash, consistent hash .
Note : Implementation of load balancing algorithm, it is recommended to read Dubbo official load balancing algorithm description , the source code explanation is very detailed, it is worth learning.
3.1 Random
3.1.1 Random algorithm
Random The algorithm randomly distributes requests to candidate servers.
The random algorithm is suitable for scenarios where the server hardware is the same. Those who have studied probability theory know that when the call volume is small, the load may be uneven. The larger the call volume, the more balanced the load.
[Example] Random algorithm implementation example
Load balancing interface
public interface LoadBalance<N extends Node> {
N select(List<N> nodes, String ip);
}
Load balancing abstract class
public abstract class BaseLoadBalance<N extends Node> implements LoadBalance<N> {
@Override
public N select(List<N> nodes, String ip) {
if (CollectionUtil.isEmpty(nodes)) {
return null;
}
// 如果 nodes 列表中仅有一个 node，直接返回即可，无需进行负载均衡
if (nodes.size() == 1) {
return nodes.get(0);
}
return doSelect(nodes, ip);
}
protected abstract N doSelect(List<N> nodes, String ip);
}
Server node class
public class Node implements Comparable<Node> {
protected String url;
protected Integer weight;
protected Integer active;
// ...
}
Random algorithm implementation
public class RandomLoadBalance<N extends Node> extends BaseLoadBalance<N> implements LoadBalance<N> {
private final Random random = new Random();
@Override
protected N doSelect(List<N> nodes, String ip) {
// 在列表中随机选取一个节点
int index = random.nextInt(nodes.size());
return nodes.get(index);
}
}
3.1.2 Weighted random algorithm
weighted random (Weighted Rando m ) algorithm is based on the random algorithm and adjusts the weight according to the probability to perform load distribution.
[Example] Example of weighted random algorithm implementation
public class WeightRandomLoadBalance<N extends Node> extends BaseLoadBalance<N> implements LoadBalance<N> {
private final Random random = ThreadLocalRandom.current();
@Override
protected N doSelect(List<N> nodes, String ip) {
int length = nodes.size();
AtomicInteger totalWeight = new AtomicInteger(0);
for (N node : nodes) {
Integer weight = node.getWeight();
totalWeight.getAndAdd(weight);
}
if (totalWeight.get() > 0) {
int offset = random.nextInt(totalWeight.get());
for (N node : nodes) {
// 让随机值 offset 减去权重值
offset -= node.getWeight();
if (offset < 0) {
// 返回相应的 Node
return node;
}
}
}
// 直接随机返回一个
return nodes.get(random.nextInt(length));
}
}
3.2 Polling
3.2.1 Polling Algorithm
Round Robin algorithm is to distribute requests to candidate servers in turn.
As shown in the figure below, the load balancer receives 6 requests from the client, the request of (1, 3, 5) will be sent to server 1, and the request of (2, 4, 6) will be sent to server 2.
This algorithm is suitable for scenarios: the processing capabilities of the servers are similar, and the workload of each transaction has little difference. If there is a large difference, the slower server may have a backlog of requests and ultimately cannot bear the excessive load.
[Example] Polling algorithm example
Implementation of polling load balancing algorithm
public class RoundRobinLoadBalance<N extends Node> extends BaseLoadBalance<N> implements LoadBalance<N> {
private final AtomicInteger position = new AtomicInteger(0);
@Override
protected N doSelect(List<N> nodes, String ip) {
int length = nodes.size();
// 如果位置值已经等于节点数，重置为 0
position.compareAndSet(length, 0);
N node = nodes.get(position.get());
position.getAndIncrement();
return node;
}
}
3.2.2 Weighted polling algorithm
weighted round robin (Weighted Round Robbin) algorithm on the basis of the round robin algorithm, increase the weight attribute to adjust the number of requests from the forwarding server. Nodes with high performance and fast processing speed should be set with higher weights, so that the request is distributed to the nodes with higher weights in priority.
As shown in the figure below, server A sets the weight to 5, server B sets the weight to 1, and the load balancer receives 6 requests from the client, then (1, 2, 3, 4, 5) requests will be sent to server A , (6) The request will be sent to server B.
[Example] Implementation example of weighted polling algorithm
The following implementation is based on Dubbo weighted polling algorithm with some simplifications.
public class WeightRoundRobinLoadBalance<N extends Node> extends BaseLoadBalance<N> implements LoadBalance<N> {
/**
* 60秒
*/
private static final int RECYCLE_PERIOD = 60000;
/**
* Node hashcode 到 WeightedRoundRobin 的映射关系
*/
private ConcurrentMap<Integer, WeightedRoundRobin> weightMap = new ConcurrentHashMap<>();
/**
* 原子更新锁
*/
private AtomicBoolean updateLock = new AtomicBoolean();
@Override
protected N doSelect(List<N> nodes, String ip) {
int totalWeight = 0;
long maxCurrent = Long.MIN_VALUE;
// 获取当前时间
long now = System.currentTimeMillis();
N selectedNode = null;
WeightedRoundRobin selectedWRR = null;
// 下面这个循环主要做了这样几件事情：
// 1. 遍历 Node 列表，检测当前 Node 是否有相应的 WeightedRoundRobin，没有则创建
// 2. 检测 Node 权重是否发生了变化，若变化了，则更新 WeightedRoundRobin 的 weight 字段
// 3. 让 current 字段加上自身权重，等价于 current += weight
// 4. 设置 lastUpdate 字段，即 lastUpdate = now
// 5. 寻找具有最大 current 的 Node，以及 Node 对应的 WeightedRoundRobin，
// 暂存起来，留作后用
// 6. 计算权重总和
for (N node : nodes) {
int hashCode = node.hashCode();
WeightedRoundRobin weightedRoundRobin = weightMap.get(hashCode);
int weight = node.getWeight();
if (weight < 0) {
weight = 0;
}
// 检测当前 Node 是否有对应的 WeightedRoundRobin，没有则创建
if (weightedRoundRobin == null) {
weightedRoundRobin = new WeightedRoundRobin();
// 设置 Node 权重
weightedRoundRobin.setWeight(weight);
// 存储 url 唯一标识 identifyString 到 weightedRoundRobin 的映射关系
weightMap.putIfAbsent(hashCode, weightedRoundRobin);
weightedRoundRobin = weightMap.get(hashCode);
}
// Node 权重不等于 WeightedRoundRobin 中保存的权重，说明权重变化了，此时进行更新
if (weight != weightedRoundRobin.getWeight()) {
weightedRoundRobin.setWeight(weight);
}
// 让 current 加上自身权重，等价于 current += weight
long current = weightedRoundRobin.increaseCurrent();
// 设置 lastUpdate，表示近期更新过
weightedRoundRobin.setLastUpdate(now);
// 找出最大的 current
if (current > maxCurrent) {
maxCurrent = current;
// 将具有最大 current 权重的 Node 赋值给 selectedNode
selectedNode = node;
// 将 Node 对应的 weightedRoundRobin 赋值给 selectedWRR，留作后用
selectedWRR = weightedRoundRobin;
}
// 计算权重总和
totalWeight += weight;
}
// 对 weightMap 进行检查，过滤掉长时间未被更新的节点。
// 该节点可能挂了，nodes 中不包含该节点，所以该节点的 lastUpdate 长时间无法被更新。
// 若未更新时长超过阈值后，就会被移除掉，默认阈值为60秒。
if (!updateLock.get() && nodes.size() != weightMap.size()) {
if (updateLock.compareAndSet(false, true)) {
try {
// 遍历修改，即移除过期记录
weightMap.entrySet().removeIf(item -> now - item.getValue().getLastUpdate() > RECYCLE_PERIOD);
} finally {
updateLock.set(false);
}
}
}
if (selectedNode != null) {
// 让 current 减去权重总和，等价于 current -= totalWeight
selectedWRR.decreaseCurrent(totalWeight);
// 返回具有最大 current 的 Node
return selectedNode;
}
// should not happen here
return nodes.get(0);
}
protected static class WeightedRoundRobin {
// 服务提供者权重
private int weight;
// 当前权重
private AtomicLong current = new AtomicLong(0);
// 最后一次更新时间
private long lastUpdate;
public long increaseCurrent() {
// current = current + weight；
return current.addAndGet(weight);
}
public long decreaseCurrent(int total) {
// current = current - total;
return current.addAndGet(-1 * total);
}
public int getWeight() {
return weight;
}
public void setWeight(int weight) {
this.weight = weight;
// 初始情况下，current = 0
current.set(0);
}
public AtomicLong getCurrent() {
return current;
}
public void setCurrent(AtomicLong current) {
this.current = current;
}
public long getLastUpdate() {
return lastUpdate;
}
public void setLastUpdate(long lastUpdate) {
this.lastUpdate = lastUpdate;
}
}
}
3.3 Minimum active number
minimum active number (Least Active) algorithm distributes the request to the candidate server with the least number of connections/requests (the server with the least number of requests currently processed).
- Features: According to the current number of request connections of the candidate server, it is dynamically allocated.
- Scenario: It is suitable for scenarios where the system load is more sensitive or the request connection time varies greatly.
Since the connection duration of each request is different, if a simple round-robin or random algorithm is used, there may be cases where the current number of connections to some servers is too large, while the connections to other servers are too small, which causes the load to be unreal. balanced. Although both polling and algorithms can adjust the load by weighting attributes, the weighting method is difficult to cope with dynamic changes.
For example, in the figure below, (1, 3, 5) requests will be sent to server 1, but (1, 3) will be disconnected soon. At this time, only (5) requests to connect to server 1; (2, 4, 6) ) The request is sent to server 2, only (2) is disconnected. When the system continues to run, server 2 will bear an excessive load.
The minimum active number algorithm records the current moment, the number of connections each candidate node is processing, and then selects the node with the smallest number of connections. This strategy can dynamically and real-timely reflect the current status of the server, reasonably distribute the responsibilities evenly, and is suitable for scenarios that are more sensitive to the current system load.
For example, in the figure below, server 1 currently has the smallest number of connections, then the newly arrived request 6 will be sent to server 1.
(Weighted Least Connection) on the basis of the minimum active number, assign a weight to each server according to the performance of the server, and then calculate the number of connections that each server can handle based on the weight.
The main points of the realization of the minimum active number algorithm: the smaller the number of active calls, the higher the processing capacity of the service node, and the more requests that can be processed per unit time, and the requests should be distributed to the service first. In a specific implementation, each service node corresponds to an active number active. In the initial situation, the active count of all service providers is 0. Each time a request is received, the number of actives is increased by 1, and after the request is completed, the number of actives is reduced by 1. After the service runs for a period of time, service providers with good performance can process requests faster, so the number of actives decreases faster. At this time, such service providers can get new service requests first, which is the minimum active number The basic idea of load balancing algorithm.
[Example] Implementation of the minimum active number algorithm
The following implementation is based on Dubbo's minimum active load balancing algorithm with some changes.
public class LeastActiveLoadBalance<N extends Node> extends BaseLoadBalance<N> implements LoadBalance<N> {
private final Random random = new Random();
@Override
protected N doSelect(List<N> nodes, String ip) {
int length = nodes.size();
// 最小的活跃数
int leastActive = -1;
// 具有相同“最小活跃数”的服务者提供者（以下用 Node 代称）数量
int leastCount = 0;
// leastIndexs 用于记录具有相同“最小活跃数”的 Node 在 nodes 列表中的下标信息
int[] leastIndexs = new int[length];
int totalWeight = 0;
// 第一个最小活跃数的 Node 权重值，用于与其他具有相同最小活跃数的 Node 的权重进行对比，
// 以检测是否“所有具有相同最小活跃数的 Node 的权重”均相等
int firstWeight = 0;
boolean sameWeight = true;
// 遍历 nodes 列表
for (int i = 0; i < length; i++) {
N node = nodes.get(i);
// 发现更小的活跃数，重新开始
if (leastActive == -1 || node.getActive() < leastActive) {
// 使用当前活跃数更新最小活跃数 leastActive
leastActive = node.getActive();
// 更新 leastCount 为 1
leastCount = 1;
// 记录当前下标值到 leastIndexs 中
leastIndexs[0] = i;
totalWeight = node.getWeight();
firstWeight = node.getWeight();
sameWeight = true;
// 当前 Node 的活跃数 node.getActive() 与最小活跃数 leastActive 相同
} else if (node.getActive() == leastActive) {
// 在 leastIndexs 中记录下当前 Node 在 nodes 集合中的下标
leastIndexs[leastCount++] = i;
// 累加权重
totalWeight += node.getWeight();
// 检测当前 Node 的权重与 firstWeight 是否相等，
// 不相等则将 sameWeight 置为 false
if (sameWeight && i > 0
&& node.getWeight() != firstWeight) {
sameWeight = false;
}
}
}
// 当只有一个 Node 具有最小活跃数，此时直接返回该 Node 即可
if (leastCount == 1) {
return nodes.get(leastIndexs[0]);
}
// 有多个 Node 具有相同的最小活跃数，但它们之间的权重不同
if (!sameWeight && totalWeight > 0) {
// 随机生成一个 [0, totalWeight) 之间的数字
int offsetWeight = random.nextInt(totalWeight);
// 循环让随机数减去具有最小活跃数的 Node 的权重值，
// 当 offset 小于等于0时，返回相应的 Node
for (int i = 0; i < leastCount; i++) {
int leastIndex = leastIndexs[i];
// 获取权重值，并让随机数减去权重值
offsetWeight -= nodes.get(leastIndex).getWeight();
if (offsetWeight <= 0) {
return nodes.get(leastIndex);
}
}
}
// 如果权重相同或权重为0时，随机返回一个 Node
return nodes.get(leastIndexs[random.nextInt(leastCount)]);
}
}
3.4 Source address hash
source address hash (IP Hash) algorithm According to the request source IP, a value is obtained through hash calculation, and the value is used to perform modulo operation in the candidate server list, and the result is the selected server.
It can be guaranteed that requests from clients of the same IP will be forwarded to the same server to achieve sticky sessions.
Features: Ensure that specific users always request the same server. If the server goes down, the session will be lost.
[Example] Source address hash algorithm implementation example
public class IpHashLoadBalance<N extends Node> extends BaseLoadBalance<N> implements LoadBalance<N> {
@Override
protected N doSelect(List<N> nodes, String ip) {
if (StrUtil.isBlank(ip)) {
ip = "127.0.0.1";
}
int length = nodes.size();
int index = hash(ip) % length;
return nodes.get(index);
}
public int hash(String text) {
return HashUtil.fnvHash(text);
}
}
3.5 Consistent Hash
The goal of the Consistent Hash algorithm is: the same request falls on the same server as much as possible.
Consistent hashing can solve the stability problem very well. All storage nodes can be arranged on a Hash ring that is connected end to end. After calculating the Hash, each key will find the adjacent storage node clockwise and store it. When a node joins or exits, it only affects the subsequent nodes that are adjacent to the node clockwise on the Hash ring.
1) The same request means: Generally, when using consistent hashing, you need to specify a key for hash calculation, which may be:
User ID
Requester IP
Request service name, a string of parameter lists
2) As far as possible, it means that the server may go online and offline, and the changes of a few servers should not affect the majority of requests.
When a candidate server goes down, the request originally sent to that server will be distributed to other candidate servers based on the virtual node, without causing drastic changes.
Advantages : Adding and deleting nodes only affects the neighboring nodes in the clockwise direction in the hash ring, and has no effect on other nodes.
Disadvantage subtracting nodes will cause some data in the hash ring to fail to hit. When a small number of nodes are used, node changes will affect the data mapping in the hash ring on a large scale, which is not suitable for a distributed scheme with a small number of data nodes. Ordinary consistent hash partitioning needs to double or subtract half of the nodes when adding or subtracting nodes to ensure the balance of data and load.
Note : Because of these shortcomings of consistent hash partitioning, some distributed systems use virtual slots to improve consistent hashing, such as the Dynamo system.
[Example] Example of consistent hashing algorithm
public class ConsistentHashLoadBalance<N extends Node> extends BaseLoadBalance<N> implements LoadBalance<N> {
private final ConcurrentMap<String, ConsistentHashSelector<?>> selectors = new ConcurrentHashMap<>();
@SuppressWarnings("unchecked")
@Override
protected N doSelect(List<N> nodes, String ip) {
// 分片数，这里设为节点数的 4 倍
Integer replicaNum = nodes.size() * 4;
// 获取 nodes 原始的 hashcode
int identityHashCode = System.identityHashCode(nodes);
// 如果 nodes 是一个新的 List 对象，意味着节点数量发生了变化
// 此时 selector.identityHashCode != identityHashCode 条件成立
ConsistentHashSelector<N> selector = (ConsistentHashSelector<N>) selectors.get(ip);
if (selector == null || selector.identityHashCode != identityHashCode) {
// 创建新的 ConsistentHashSelector
selectors.put(ip, new ConsistentHashSelector<>(nodes, identityHashCode, replicaNum));
selector = (ConsistentHashSelector<N>) selectors.get(ip);
}
// 调用 ConsistentHashSelector 的 select 方法选择 Node
return selector.select(ip);
}
/**
* 一致性哈希选择器
*/
private static final class ConsistentHashSelector<N extends Node> {
/**
* 存储虚拟节点
*/
private final TreeMap<Long, N> virtualNodes;
private final int identityHashCode;
/**
* 构造器
*
* @param nodes 节点列表
* @param identityHashCode hashcode
* @param replicaNum 分片数
*/
ConsistentHashSelector(List<N> nodes, int identityHashCode, Integer replicaNum) {
this.virtualNodes = new TreeMap<>();
this.identityHashCode = identityHashCode;
// 获取虚拟节点数，默认为 100
if (replicaNum == null) {
replicaNum = 100;
}
for (N node : nodes) {
for (int i = 0; i < replicaNum / 4; i++) {
// 对 url 进行 md5 运算，得到一个长度为16的字节数组
byte[] digest = md5(node.getUrl());
// 对 digest 部分字节进行 4 次 hash 运算，得到四个不同的 long 型正整数
for (int j = 0; j < 4; j++) {
// h = 0 时，取 digest 中下标为 0 ~ 3 的4个字节进行位运算
// h = 1 时，取 digest 中下标为 4 ~ 7 的4个字节进行位运算
// h = 2, h = 3 时过程同上
long m = hash(digest, j);
// 将 hash 到 node 的映射关系存储到 virtualNodes 中，
// virtualNodes 需要提供高效的查询操作，因此选用 TreeMap 作为存储结构
virtualNodes.put(m, node);
}
}
}
}
public N select(String key) {
// 对参数 key 进行 md5 运算
byte[] digest = md5(key);
// 取 digest 数组的前四个字节进行 hash 运算，再将 hash 值传给 selectForKey 方法，
// 寻找合适的 Node
return selectForKey(hash(digest, 0));
}
private N selectForKey(long hash) {
// 查找第一个大于或等于当前 hash 的节点
Map.Entry<Long, N> entry = virtualNodes.ceilingEntry(hash);
// 如果 hash 大于 Node 在哈希环上最大的位置，此时 entry = null，
// 需要将 TreeMap 的头节点赋值给 entry
if (entry == null) {
entry = virtualNodes.firstEntry();
}
// 返回 Node
return entry.getValue();
}
}
/**
* 计算 hash 值
*/
public static long hash(byte[] digest, int number) {
return (((long) (digest[3 + number * 4] & 0xFF) << 24)
| ((long) (digest[2 + number * 4] & 0xFF) << 16)
| ((long) (digest[1 + number * 4] & 0xFF) << 8)
| (digest[number * 4] & 0xFF))
& 0xFFFFFFFFL;
}
/**
* 计算 MD5 值
*/
public static byte[] md5(String value) {
MessageDigest md5;
try {
md5 = MessageDigest.getInstance("MD5");
} catch (NoSuchAlgorithmException e) {
throw new IllegalStateException(e.getMessage(), e);
}
md5.reset();
byte[] bytes = value.getBytes(StandardCharsets.UTF_8);
md5.update(bytes);
return md5.digest();
}
}
The above example is based on Dubbo's consistent hash load balancing algorithm with some simplifications.
Four, reference materials
1. Comparing Load Balancing Algorithms
2. "Technical Architecture of Large Websites: Core Principles and Case Studies"
3. large-scale website architecture series: load balancing detailed explanation (1)
6. Dubbo official load balancing algorithm description
7. load balancing algorithm and means
8. uses dns resolution to achieve load balancing of the website
Author: vivo Internet team-Zhang Peng
**粗体** _斜体_ [链接](http://example.com) `代码` - 列表 > 引用。你还可以使用@来通知其他用户。