Circuit Breaker function of Spring Cloud Gateway

Welcome to my GitHub

https://github.com/zq2599/blog_demos

Content: Classification and summary of all original articles and supporting source code, involving Java, Docker, Kubernetes, DevOPS, etc.;

Overview of this article

• Let's learn more about the circuit breaker (CircuitBreaker) function of Spring Cloud Gateway:
• Talk about theory first
• Combined with official and great god information to determine the technology stack
• Develop again, implement first and then verify
• Strike the iron while it's hot and look at its source code
• Finally, review the shortcomings (the next article addresses these shortcomings)

About CircuitBreaker

• The following figure is from the resilience4j official document, which introduces what a circuit breaker is:

1. In the CLOSED state, normal release is requested
2. When the request failure rate reaches the set threshold, it becomes the OPEN state, and all requests are not released at this time
3. After the OPEN state lasts for a set time, it enters the half-open state (HALE_OPEN), letting go of some requests
4. In the half-open state, if the failure rate is lower than the set threshold, enter the CLOSE state, that is, all release
5. In the half-open state, if the failure rate is higher than the set threshold, it will enter the OPEN state, that is, all will not be released.

Confirm concept

• There is a concept to confirm first, that is, the <font color="blue">Spring Cloud circuit breaker</font> and the <font color="blue">Spring Cloud Gateway circuit breaker function</font> are not the same concept, Spring Cloud The Gateway circuit breaker function also involves filters, that is, the use of circuit breakers under the rules of the filter:

• The focus of this article is how Spring Cloud Gateway configures and uses CircuitBreaker, so the details of Resilience4J will not be discussed. If you want to learn more about Resilience4J, the recommended information is Spring Cloud Circuit Breaker

About Spring Cloud Circuit Breaker

• First look at the Spring Cloud circuit breaker. As shown in the figure below, Hystrix and Sentinel are familiar concepts:

About the circuit breaker function of Spring Cloud Gateway

• Take a look at the official documentation of Spring Cloud Gateway, as shown in the figure below. A few key points will be introduced later:

• The above figure reveals several key information:
• Spring Cloud Gateway has a built-in circuit breaker filter,
• The specific method is to use the API of the Spring Cloud circuit breaker to encapsulate the routing logic of the gateway into the circuit breaker
• Libraries with multiple circuit breakers can be used in Spring Cloud Gateway (unfortunately, which ones are not listed)
• Resilience4J is out of the box for Spring Cloud
• Simply put, the circuit breaker function of Spring Cloud Gateway is implemented through a built-in filter, which uses the Spring Cloud circuit breaker;
• The official said that multiple circuit breaker libraries can be used in Spring Cloud Gateway, but it did not say which ones are specific, which is depressing. At this time, let's understand the point of view of a great man: Piotr Mińkowski, the author of the following book :

• Piotr Mińkowski's blog a detailed introduction to the circuit breaker function of Spring Cloud Gateway, as shown in the figure below. Several important information will be mentioned later:

• The above figure can get three key information:
• Starting from version 2.2.1, Spring Cloud Gateway integrates Resilience4J's circuit breaker implementation
• Netflix's Hystrix has entered the maintenance phase (can it be understood as being about to retire?)
• Netflix's Hystrix is still available, but has been deprecated, and future versions of Spring Cloud may not support
• Resilience4 is also used as an example to link to the official documents (as shown in the figure below). I don’t seem to have any other choice if I am timid, just Resilience4J:

• That's it for the theoretical analysis. Next, we will start the actual combat. The specific steps are as follows:
• Preparation work: The service provider adds a new web interface <font color="blue">/account/{id}</font>. Depending on the input parameters, the interface can return immediately or with a delay of 500 milliseconds.
• Add a new sub-project named <font color="blue">circuitbreaker-gateway</font>, which is a Spring Cloud Gateway application with circuit breaker function
• Write unit test code in <font color="blue">circuitbreaker-gateway</font> to verify whether the circuit breaker is normal
• Run the unit test code to observe whether the circuit breaker takes effect
• Add fallback to the circuit breaker and verify that it works
• Do a simple source code analysis, one is for students who want to understand the circuit breaker to clear the source code path, and the other is to check whether their previous knowledge of springboot is helpful when reading the source code

Source download

• The complete source code in this actual combat can be downloaded on GitHub, the address and link information are shown in the following table ( https://github.com/zq2599/blog_demos):

name	Link	Remark
Project homepage	https://github.com/zq2599/blog_demos	The project's homepage on GitHub
git warehouse address (https)	https://github.com/zq2599/blog_demos.git	The warehouse address of the source code of the project, https protocol
git warehouse address (ssh)	[e-mail]:zq2599/blog_demos.git	The warehouse address of the source code of the project, ssh protocol

• There are multiple folders in this git project. The source code of this article is under the <font color="blue">spring-cloud-tutorials</font> folder, as shown in the red box below:

• There are multiple sub-projects under the <font color="blue">spring-cloud-tutorials</font> folder. The code for this article is <font color="red">circuitbreaker-gateway</font>, as shown in the red box below Shown:

Ready to work

• We need to prepare a controllable web interface and control its success or failure through parameters so that the circuit breaker can be triggered
• In the actual combat of this article, the service provider is still <font color="blue">provider-hello</font>. In order to meet the needs of this actual combat, we add a web interface to the Hello.java file, the corresponding source code as follows:

    @RequestMapping(value = "/account/{id}", method = RequestMethod.GET)
    public String account(@PathVariable("id") int id) throws InterruptedException {
        if(1==id) {
            Thread.sleep(500);
        }

        return Constants.ACCOUNT_PREFIX + dateStr();
    }

• The above code is very simple: it is to receive the id parameter, if it is equal to 1, it will delay five hundred milliseconds, and it will return immediately if it is not equal to 1.
• If the circuit breaker is set to more than two hundred milliseconds and it fails, then by controlling the value of the id parameter, we can simulate the success or failure of the request. <font color="blue">This is the key to verifying the function of the circuit breaker</font >
• Preparation is complete, start to write code

Actual combat

• Add a subproject <font color="blue">circuitbreaker-gateway</font> under the parent project <font color="blue">spring-cloud-tutorials</font>
• Add the following dependencies

<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-starter-circuitbreaker-reactor-resilience4j</artifactId>
</dependency>

• The configuration file application.yml is as follows:

server:
  #服务端口
  port: 8081
spring:
  application:
    name: circuitbreaker-gateway
  cloud:
    gateway:
      routes:
        - id: path_route
          uri: http://127.0.0.1:8082
          predicates:
            - Path=/hello/**
          filters:
            - name: CircuitBreaker
              args:
                name: myCircuitBreaker

• Startup class:

package com.bolingcavalry.circuitbreakergateway;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;

@SpringBootApplication
public class CircuitbreakerApplication {
    public static void main(String[] args) {
        SpringApplication.run(CircuitbreakerApplication.class,args);
    }
}

• The configuration class is as follows, which is the parameter configuration related to the circuit breaker:

package com.bolingcavalry.circuitbreakergateway.config;

import io.github.resilience4j.circuitbreaker.CircuitBreakerConfig;
import io.github.resilience4j.timelimiter.TimeLimiterConfig;
import org.springframework.cloud.circuitbreaker.resilience4j.ReactiveResilience4JCircuitBreakerFactory;
import org.springframework.cloud.circuitbreaker.resilience4j.Resilience4JConfigBuilder;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import java.time.Duration;

@Configuration
public class CustomizeCircuitBreakerConfig {

    @Bean
    public ReactiveResilience4JCircuitBreakerFactory defaultCustomizer() {

        CircuitBreakerConfig circuitBreakerConfig = CircuitBreakerConfig.custom() //
                .slidingWindowType(CircuitBreakerConfig.SlidingWindowType.TIME_BASED) // 滑动窗口的类型为时间窗口
                .slidingWindowSize(10) // 时间窗口的大小为60秒
                .minimumNumberOfCalls(5) // 在单位时间窗口内最少需要5次调用才能开始进行统计计算
                .failureRateThreshold(50) // 在单位时间窗口内调用失败率达到50%后会启动断路器
                .enableAutomaticTransitionFromOpenToHalfOpen() // 允许断路器自动由打开状态转换为半开状态
                .permittedNumberOfCallsInHalfOpenState(5) // 在半开状态下允许进行正常调用的次数
                .waitDurationInOpenState(Duration.ofSeconds(5)) // 断路器打开状态转换为半开状态需要等待60秒
                .recordExceptions(Throwable.class) // 所有异常都当作失败来处理
                .build();

        ReactiveResilience4JCircuitBreakerFactory factory = new ReactiveResilience4JCircuitBreakerFactory();
        factory.configureDefault(id -> new Resilience4JConfigBuilder(id)
                .timeLimiterConfig(TimeLimiterConfig.custom().timeoutDuration(Duration.ofMillis(200)).build())
                .circuitBreakerConfig(circuitBreakerConfig).build());

        return factory;
    }
}

• The above code requires attention once: The <font color="blue">timeLimiterConfig</font> method sets a timeout period. If the service provider does not respond for more than 200 milliseconds, Spring Cloud Gateway will return failure to the caller.
• The development is complete, the next thing to consider is how to verify

Unit test class

• In order to verify the circuit breaker function of Spring Cloud Gateway, we can use Junit unit test to precisely control the request parameters and the number of requests. The test class is as follows. It can be seen that the test class will send a hundred requests in a row. In the first fifty times, the request parameters are always Switch between 0 and 1. When the parameter is equal to 1, the interface will have a delay of 500 milliseconds, which exceeds the 200 millisecond timeout limit of Spring Cloud Gateway. At this time, it will return to failure. When there are too many failures, the circuit breaker will be triggered. The disconnection:

package com.bolingcavalry.circuitbreakergateway;

import io.github.resilience4j.circuitbreaker.CircuitBreaker;
import org.junit.jupiter.api.RepeatedTest;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.autoconfigure.web.reactive.AutoConfigureWebTestClient;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.http.MediaType;
import org.springframework.test.context.junit.jupiter.SpringExtension;
import org.springframework.test.web.reactive.server.WebTestClient;

@SpringBootTest
@ExtendWith(SpringExtension.class)
@AutoConfigureWebTestClient
public class CircuitbreakerTest {

    // 测试的总次数
    private static int i=0;

    @Autowired
    private WebTestClient webClient;

    @Test
    @RepeatedTest(100)
    void testHelloPredicates() throws InterruptedException {
        // 低于50次时，gen在0和1之间切换，也就是一次正常一次超时，
        // 超过50次时，gen固定为0，此时每个请求都不会超时
        int gen = (i<50) ? (i % 2) : 0;

        // 次数加一
        i++;

        final String tag = "[" + i + "]";

        // 发起web请求
        webClient.get()
                .uri("/hello/account/" + gen)
                .accept(MediaType.APPLICATION_JSON)
                .exchange()
                .expectBody(String.class).consumeWith(result  -> System.out.println(tag + result.getRawStatusCode() + " - " + result.getResponseBody()));

        Thread.sleep(1000);
    }
}

verify

• Start nacos (the service provider depends on)
• Start the sub-project <font color="blue">provider-hello</font>
• Run the unit test class we just developed, and the intercepted part of the console input is as follows, which will be analyzed later:

[2]504 - {"timestamp":"2021-08-28T02:55:42.920+00:00","path":"/hello/account/1","status":504,"error":"Gateway Timeout","message":"","requestId":"594efed1"}
[3]200 - Account2021-08-28 10:55:43
[4]504 - {"timestamp":"2021-08-28T02:55:45.177+00:00","path":"/hello/account/1","status":504,"error":"Gateway Timeout","message":"","requestId":"427720b"}
[5]200 - Account2021-08-28 10:55:46
[6]503 - {"timestamp":"2021-08-28T02:55:47.227+00:00","path":"/hello/account/1","status":503,"error":"Service Unavailable","message":"","requestId":"6595d7f4"}
[7]503 - {"timestamp":"2021-08-28T02:55:48.250+00:00","path":"/hello/account/0","status":503,"error":"Service Unavailable","message":"","requestId":"169ae1c"}
[8]503 - {"timestamp":"2021-08-28T02:55:49.259+00:00","path":"/hello/account/1","status":503,"error":"Service Unavailable","message":"","requestId":"53b695a1"}
[9]503 - {"timestamp":"2021-08-28T02:55:50.269+00:00","path":"/hello/account/0","status":503,"error":"Service Unavailable","message":"","requestId":"4a072f52"}
[10]504 - {"timestamp":"2021-08-28T02:55:51.499+00:00","path":"/hello/account/1","status":504,"error":"Gateway Timeout","message":"","requestId":"4bdd96c4"}
[11]200 - Account2021-08-28 10:55:52
[12]504 - {"timestamp":"2021-08-28T02:55:53.745+00:00","path":"/hello/account/1","status":504,"error":"Gateway Timeout","message":"","requestId":"4e0e7eab"}
[13]200 - Account2021-08-28 10:55:54
[14]504 - {"timestamp":"2021-08-28T02:55:56.013+00:00","path":"/hello/account/1","status":504,"error":"Gateway Timeout","message":"","requestId":"27685405"}
[15]503 - {"timestamp":"2021-08-28T02:55:57.035+00:00","path":"/hello/account/0","status":503,"error":"Service Unavailable","message":"","requestId":"3e40c5db"}
[16]503 - {"timestamp":"2021-08-28T02:55:58.053+00:00","path":"/hello/account/1","status":503,"error":"Service Unavailable","message":"","requestId":"2bf2698b"}
[17]503 - {"timestamp":"2021-08-28T02:55:59.075+00:00","path":"/hello/account/0","status":503,"error":"Service Unavailable","message":"","requestId":"38cb1840"}
[18]503 - {"timestamp":"2021-08-28T02:56:00.091+00:00","path":"/hello/account/1","status":503,"error":"Service Unavailable","message":"","requestId":"21586fa"}
[19]200 - Account2021-08-28 10:56:01
[20]504 - {"timestamp":"2021-08-28T02:56:02.325+00:00","path":"/hello/account/1","status":504,"error":"Gateway Timeout","message":"","requestId":"4014d6d4"}
[21]200 - Account2021-08-28 10:56:03
[22]504 - {"timestamp":"2021-08-28T02:56:04.557+00:00","path":"/hello/account/1","status":504,"error":"Gateway Timeout","message":"","requestId":"173a3b9d"}
[23]200 - Account2021-08-28 10:56:05
[24]504 - {"timestamp":"2021-08-28T02:56:06.811+00:00","path":"/hello/account/1","status":504,"error":"Gateway Timeout","message":"","requestId":"aa8761f"}
[25]200 - Account2021-08-28 10:56:07
[26]504 - {"timestamp":"2021-08-28T02:56:09.057+00:00","path":"/hello/account/1","status":504,"error":"Gateway Timeout","message":"","requestId":"769bfefc"}
[27]200 - Account2021-08-28 10:56:10
[28]504 - {"timestamp":"2021-08-28T02:56:11.314+00:00","path":"/hello/account/1","status":504,"error":"Gateway Timeout","message":"","requestId":"2fbcb6c0"}
[29]503 - {"timestamp":"2021-08-28T02:56:12.332+00:00","path":"/hello/account/0","status":503,"error":"Service Unavailable","message":"","requestId":"58e4e70f"}
[30]503 - {"timestamp":"2021-08-28T02:56:13.342+00:00","path":"/hello/account/1","status":503,"error":"Service Unavailable","message":"","requestId":"367651c5"}

• Analyze the return code of the above output:

1. 504 is the error returned by the timeout, 200 is the normal return of the service provider
2. Both 504 and 200 return codes indicate that the request has reached the service provider, so the circuit breaker is closed at this time
3. After multiple 504 errors, the configured threshold is reached and the circuit breaker is triggered to open
4. The 503 that appears continuously is the return code after the circuit breaker is turned on. At this time, the request cannot reach the service provider.
5. After a continuous 503, 504 and 200 appear alternately again, which proves that it enters the half-open state at this time, and then 504 reaches the threshold again to trigger the circuit breaker to turn from half open to open. After fifty times, the circuit breaker is closed because the timeout request is not sent. state

fallback

• Through the above test, it can be seen that Spring Cloud Gateway informs the caller of the error information through the return code. This method is not friendly enough. We can customize the fallback and use it to construct the return information when an error is returned.
• Develop another web interface, yes, add a web interface to the <font color="blue">circuitbreaker-gateway</font> project:

package com.bolingcavalry.circuitbreakergateway.controller;

import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;
import java.text.SimpleDateFormat;
import java.util.Date;

@RestController
public class Fallback {

    private String dateStr(){
        return new SimpleDateFormat("yyyy-MM-dd hh:mm:ss").format(new Date());
    }

    /**
     * 返回字符串类型
     * @return
     */
    @GetMapping("/myfallback")
    public String helloStr() {
        return "myfallback, " + dateStr();
    }
}

• The application.yml configuration is as follows, it can be seen that the <font color="blue">fallbackUri</font> attribute is added to the filter:

server:
  #服务端口
  port: 8081
spring:
  application:
    name: circuitbreaker-gateway
  cloud:
    gateway:
      routes:
        - id: path_route
          uri: http://127.0.0.1:8082
          predicates:
            - Path=/hello/**
          filters:
            - name: CircuitBreaker
              args:
                name: myCircuitBreaker
                fallbackUri: forward:/myfallback

• Run the unit test again, it can be seen that the return code is all 200, and all the original errors are now returned by the newly added interface:

[2]200 - myfallback, 2021-08-28 11:15:02
[3]200 - Account2021-08-28 11:15:03
[4]200 - myfallback, 2021-08-28 11:15:04
[5]200 - Account2021-08-28 11:15:05
[6]200 - myfallback, 2021-08-28 11:15:06
[7]200 - myfallback, 2021-08-28 11:15:08
[8]200 - myfallback, 2021-08-28 11:15:09
[9]200 - myfallback, 2021-08-28 11:15:10
[10]200 - myfallback, 2021-08-28 11:15:11
[11]200 - Account2021-08-28 11:15:12
[12]200 - myfallback, 2021-08-28 11:15:13
[13]200 - Account2021-08-28 11:15:14
[14]200 - myfallback, 2021-08-28 11:15:15

• So far, we have completed the development and testing of the circuit breaker function of Spring Cloud Gateway. If you are smart and easy to learn, you are not satisfied with these few lines of configuration and code. If you want to understand the internals of the circuit breaker, please read on , Let’s talk about its source code;

Source code analysis

• The RouteDefinitionRouteLocator construction method (bean injection) has the following code to bind the name to the instance:

gatewayFilterFactories.forEach(factory -> this.gatewayFilterFactories.put(factory.name(), factory));

• Then use this map in the loadGatewayFilters method to find the bean put above;
• The final effect: The name specified in the routing configuration is equal to <font color="blue">CircuitBreaker</font>, which corresponds to a bean of type SpringCloudCircuitBreakerFilterFactory, because its name method returns "CircuitBreaker", as shown in the following figure:

• The question now: What is the bean of type SpringCloudCircuitBreakerFilterFactory? As shown in the red box in the following figure, SpringCloudCircuitBreakerResilience4JFilterFactory is the only subclass of SpringCloudCircuitBreakerFilterFactory:

• From the above figure, the filter of type CircuitBreaker should be SpringCloudCircuitBreakerResilience4JFilterFactory, but that is only inferred from the inheritance relationship, and there is still a key piece of evidence: in spring, is there a bean of type SpringCloudCircuitBreakerResilience4JFilterFactory?
• Finally found the configuration in GatewayResilience4JCircuitBreakerAutoConfiguration, which can prove that SpringCloudCircuitBreakerResilience4JFilterFactory will be instantiated and registered to spring:

@Bean
    @ConditionalOnBean(ReactiveResilience4JCircuitBreakerFactory.class)
    @ConditionalOnEnabledFilter
    public SpringCloudCircuitBreakerResilience4JFilterFactory springCloudCircuitBreakerResilience4JFilterFactory(
            ReactiveResilience4JCircuitBreakerFactory reactiveCircuitBreakerFactory,
            ObjectProvider<DispatcherHandler> dispatcherHandler) {
        return new SpringCloudCircuitBreakerResilience4JFilterFactory(reactiveCircuitBreakerFactory, dispatcherHandler);
    }

• In summary, when you configure the CircuitBreaker filter, the SpringCloudCircuitBreakerResilience4JFilterFactory class is actually serving you, and the key codes are concentrated in its parent class SpringCloudCircuitBreakerFilterFactory;
• So, if you want to learn more about the circuit breaker function of Spring Cloud Gateway, please read SpringCloudCircuitBreakerFilterFactory.apply method

A little regret

• Remember the content that was output from the analysis console just now? It is the section in the red box in the figure below. At that time, we used the return code to guess the state of the circuit breaker:

• I believe that when you read this plain text, you still have doubts about Xin Chen's analysis. According to the return code, the status of the circuit breaker is determined? For example, is 504 closed or half open? It is possible, so this kind of speculation can only prove that the circuit breaker is working, but it is impossible to determine the specific state at a certain moment.
• Therefore, we need a more accurate way to know the state of the circuit breaker at each moment, so that we have a deep understanding of the circuit breaker.
• In the next article, let’s go a step further on today’s results, print the circuit breaker status in the request, then... stay tuned, Xinchen original, never let you down;

You are not alone, Xinchen and original are with you all the way

1. Java series
2. Spring series
3. Docker series
4. kubernetes series
5. database + middleware series
6. DevOps series

Welcome to pay attention to the public account: programmer Xin Chen

Search "Programmer Xin Chen" on WeChat, I am Xin Chen, and I look forward to traveling the Java world with you...
https://github.com/zq2599/blog_demos