Why does audio and video technology need microservices
Microservice, English name: microservice, Baidu Baike defines it as: a variant of SOA architecture. Microservices (or microservices architecture) is a way of structuring an application as a set of low-coupling services.
Microservices have some distinctive features:
- single function
- service granularity is small
- Strong independence between services
- Weak inter-service dependency
- Service independent maintenance
- Service independent deployment
For each microservice, the function it provides should be single; its granularity is very small; it only provides related interfaces related to a certain business function. For example, in the order system, payment system, product system, etc. of the e-commerce system, each system service is only an independent function of the system, and does not involve functional logic that does not belong to it.
The dependency between microservices should be as weak as possible. This has the advantage of not causing the failure of other systems to operate normally due to the downtime of a single system service, thereby affecting the user experience. Also take the e-commerce system as an example: after the user adds the product to the shopping cart, submits the order, then pays and finds that the payment cannot be made. At this time, the order can be put into the pending payment state, thereby preventing the loss of the order and the unfriendly user experience . If the order system is strongly dependent on the payment system, the order system will always wait for the payment system to respond, which will cause the user's interface to always be in a loading state, which will result in the user being unable to perform any operations.
When a certain microservice function needs to be upgraded, or a certain function needs to fix a bug, only the current service needs to be compiled and deployed. There is no need to package the entire product business functions one by one, independent maintenance, Independent deployment.
described above actually highlight its distinctive characteristics: 1612630cb8071f high cohesion, low coupling , the problem is coming. What is high cohesion and what is low coupling? The so-called high cohesion: It means that each service is under the same network or domain, and compared to the outside, the whole is a closed and secure box. The external interface of the box is unchanged, and the interface between the modules inside the box is also unchanged, but the internal content of each module can be changed. Modules only expose minimal interfaces to the outside, avoiding strong dependencies. Adding or deleting a module should only affect related modules that have dependencies, and irrelevant ones should not be affected.
The so-called low coupling: From a small point of view, it is to reduce the coupling between each Java class, use multiple interfaces, and use the encapsulation, inheritance, and polymorphism of Java object-oriented programming ideas to hide implementation details. In terms of modules, it is necessary to reduce the relationship between each module, reduce the complexity of redundancy, repetition, and crossover, and make the module function division as simple as possible.
In audio and video application technology, we know that the main resources occupied are cpu and memory, and involve resource sharing, so we need to combine NFS to achieve cross-node resource sharing. Of course, the problem exposed by a single node is that once the client maintains a long-term connection with the server, and different clients send requests at the same time, the pressure on the single node is very high. It is very likely that the CPU and memory will be tight, which will cause the node to crash, which is not conducive to the high availability of the system and the robustness of the service. At this time, what needs to be solved is the problem of tight resources in audio and video communication. In the field of systems, multi-node methods can usually be used to achieve distributed and high concurrent requests. When the request comes, load balancing can be used. Through certain strategies, such as: according to the minimum number of requests, or assigning a weight value to each server, the longer the server response time, the smaller the weight of this server, and the lower the probability of being selected. In this way, the service request pressure is controlled, so that the client and the server can maintain long-term and effective communication.
How to use Springboot framework to build microservices
introduce
With the rapid development in recent years, microservices have become more and more popular. Among them, Spring Cloud has been updated and used by most companies. Representative is Alibaba. Around November 2018, Spring Cloud co-founder Spencer Gibb announced on the blog page of Spring's official website: Alibaba open sourced Spring Cloud Alibaba and released the first preview version. Subsequently, Spring Cloud official Twitter also released this news.
In Spring Boot 1.x, it mainly includes Eureka, Zuul, Config, Ribbon, Hystrix, etc. In Spring Boot 2.x, the gateway uses its own Gateway. Of course, in the Alibaba version, its components are even richer: use Alibaba's Nacos as the registration center and configuration center. Use the built-in component Sentinel as a current limiting and fuse artifact.
Build a registry
Today we mainly use Springboot combined with Alibaba's plug-ins to realize the micro-chat system micro-service design. First, let's create a registry Nacos.
We first download Nacos, Nacos address: https://github.com/alibaba/nacos/releases we download the binary file of the corresponding system, corresponding to our own system, execute the following command:
Linux/Unix/Mac:sh startup.sh -m standalone
Windows:cmd startup.cmd -m standalone
After the startup is complete, visit: http://127.0.0.1:8848/nacos/ , you can enter the Nacos service management page, as follows:
The default username and password are both nacos.
After logging in, open the service management, you can see the list of services registered to Nacos:
You can click Configuration Management to view the configuration:
If you have not configured any service configuration, you can create a new one:
The above describes how Nacos acts as a registration center and configuration center. It's very simple.
The first microservice
Next, for microservices, a service needs to be registered and discovered. We explain the service provider code:
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.damon</groupId>
<artifactId>provider-service</artifactId>
<version>0.0.1-SNAPSHOT</version>
<packaging>jar</packaging>
<name>provider-service</name>
<url>http://maven.apache.org</url>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.1.8.RELEASE</version>
<relativePath/>
</parent>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
<java.version>1.8</java.version>
<swagger.version>2.6.1</swagger.version>
<xstream.version>1.4.7</xstream.version>
<pageHelper.version>4.1.6</pageHelper.version>
<fastjson.version>1.2.51</fastjson.version>
<!-- <springcloud.version>2.1.8.RELEASE</springcloud.version> -->
<springcloud.version>Greenwich.SR3</springcloud.version>
<springcloud.kubernetes.version>1.1.1.RELEASE</springcloud.kubernetes.version>
<mysql.version>5.1.46</mysql.version>
<alibaba-cloud.version>2.1.1.RELEASE</alibaba-cloud.version>
<springcloud.alibaba.version>0.9.0.RELEASE</springcloud.alibaba.version>
</properties>
<dependencyManagement>
<dependencies>
<!-- <dependency>
<groupId>com.alibaba.cloud</groupId>
<artifactId>spring-cloud-alibaba-dependencies</artifactId>
<version>${alibaba-cloud.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency> -->
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-alibaba-dependencies</artifactId>
<version>${springcloud.alibaba.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-dependencies</artifactId>
<version>${springcloud.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
<exclusions>
<exclusion>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-tomcat</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-undertow</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-oauth2</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-alibaba-nacos-discovery</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-alibaba-nacos-config</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-netflix-ribbon</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-netflix-hystrix</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.alibaba</groupId>
<artifactId>fastjson</artifactId>
<version>${fastjson.version}</version>
</dependency>
<!-- swagger -->
<dependency>
<groupId>io.springfox</groupId>
<artifactId>springfox-swagger2</artifactId>
<version>${swagger.version}</version>
</dependency>
<dependency>
<groupId>io.springfox</groupId>
<artifactId>springfox-swagger-ui</artifactId>
<version>${swagger.version}</version>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-lang3</artifactId>
</dependency>
<dependency>
<groupId>commons-collections</groupId>
<artifactId>commons-collections</artifactId>
<version>3.2.2</version>
</dependency>
<!--分页插件-->
<dependency>
<groupId>com.github.pagehelper</groupId>
<artifactId>pagehelper</artifactId>
<version>${pageHelper.version}</version>
</dependency>
<!-- mybatis -->
<dependency>
<groupId>org.mybatis.spring.boot</groupId>
<artifactId>mybatis-spring-boot-starter</artifactId>
<version>1.1.1</version>
</dependency>
<dependency>
<groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId>
<version>${mysql.version}</version>
</dependency>
<!-- datasource pool-->
<dependency>
<groupId>com.alibaba</groupId>
<artifactId>druid</artifactId>
<version>1.1.3</version>
</dependency>
<!-- 对redis支持,引入的话项目缓存就支持redis了,所以必须加上redis的相关配置,否则操作相关缓存会报异常 -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-redis</artifactId>
</dependency>
<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
<version>19.0</version>
</dependency>
<dependency>
<groupId>io.jsonwebtoken</groupId>
<artifactId>jjwt</artifactId>
<version>0.9.0</version>
</dependency>
</dependencies>
<build>
<finalName>${project.artifactId}</finalName>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<configuration>
<jvmArguments>-Dfile.encoding=UTF-8</jvmArguments>
<fork>true</fork>
</configuration>
</plugin>
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.7.8</version>
<executions>
<execution>
<goals>
<goal>prepare-agent</goal>
<goal>report</goal>
</goals>
</execution>
</executions>
</plugin>
<!-- 自动生成代码 插件 begin -->
<!-- <plugin>
<groupId>org.mybatis.generator</groupId>
<artifactId>mybatis-generator-maven-plugin</artifactId>
<version>1.3.2</version>
<configuration>
<configurationFile>src/main/resources/generatorConfig.xml</configurationFile>
<verbose>true</verbose>
<overwrite>true</overwrite>
</configuration>
<dependencies>
<dependency>
<groupId>org.mybatis.generator</groupId>
<artifactId>mybatis-generator-core</artifactId>
<version>1.3.2</version>
</dependency>
</dependencies>
</plugin> -->
</plugins>
</build>
</project>
As always, introduce dependencies and configure the bootstrap file:
management:
endpoint:
restart:
enabled: true
health:
enabled: true
info:
enabled: true
spring:
application:
name: provider-service
cloud:
nacos:
discovery:
server-addr: 127.0.0.1:8848
config:
server-addr: 127.0.0.1:8848
refreshable-dataids: actuator.properties,log.properties
http:
encoding:
charset: UTF-8
enabled: true
force: true
mvc:
throw-exception-if-no-handler-found: true
main:
allow-bean-definition-overriding: true #当遇到同样名称时,是否允许覆盖注册
logging:
path: /data/${spring.application.name}/logs
cas-server-url: http://oauth-cas #http://localhost:2000#设置可以访问的地址
security:
oauth2: #与cas对应的配置
client:
client-id: provider-service
client-secret: provider-service-123
user-authorization-uri: ${cas-server-url}/oauth/authorize #是授权码认证方式需要的
access-token-uri: ${cas-server-url}/oauth/token #是密码模式需要用到的获取 token 的接口
resource:
loadBalanced: true
#jwt: #jwt存储token时开启
#key-uri: ${cas-server-url}/oauth/token_key
#key-value: test_jwt_sign_key
id: provider-service
#指定用户信息地址
user-info-uri: ${cas-server-url}/api/user #指定user info的URI,原生地址后缀为/auth/user
prefer-token-info: false
#token-info-uri:
authorization:
check-token-access: ${cas-server-url}/oauth/check_token #当此web服务端接收到来自UI客户端的请求后,需要拿着请求中的 token 到认证服务端做 token 验证,就是请求的这个接口
application 文件;
server:
port: 2001
undertow:
accesslog:
enabled: false
pattern: combined
servlet:
session:
timeout: PT120M
cookie:
name: PROVIDER-SERVICE-SESSIONID #防止Cookie冲突,冲突会导致登录验证不通过
client:
http:
request:
connectTimeout: 8000
readTimeout: 30000
mybatis:
mapperLocations: classpath:mapper/*.xml
typeAliasesPackage: com.damon.*.model
backend:
ribbon:
client:
enabled: true
ServerListRefreshInterval: 5000
ribbon:
ConnectTimeout: 3000
# 设置全局默认的ribbon的读超时
ReadTimeout: 1000
eager-load:
enabled: true
clients: oauth-cas,consumer-service
MaxAutoRetries: 1 #对第一次请求的服务的重试次数
MaxAutoRetriesNextServer: 1 #要重试的下一个服务的最大数量(不包括第一个服务)
#listOfServers: localhost:5556,localhost:5557
#ServerListRefreshInterval: 2000
OkToRetryOnAllOperations: true
NFLoadBalancerRuleClassName: com.netflix.loadbalancer.RoundRobinRule
hystrix.command.BackendCall.execution.isolation.thread.timeoutInMilliseconds: 5000
hystrix.threadpool.BackendCallThread.coreSize: 5
Next start the class:
package com.damon;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
import org.springframework.boot.autoconfigure.security.oauth2.client.EnableOAuth2Sso;
import org.springframework.cloud.client.discovery.EnableDiscoveryClient;
import org.springframework.context.annotation.ComponentScan;
import org.springframework.context.annotation.Configuration;
/**
* @author Damon
* @date 2020年1月13日 下午3:23:06
*
*/
@Configuration
@EnableAutoConfiguration
@ComponentScan(basePackages = {"com.damon"})
@EnableDiscoveryClient
@EnableOAuth2Sso
public class ProviderApp {
public static void main(String[] args) {
SpringApplication.run(ProviderApp.class, args);
}
}
Note: The annotations @EnableDiscoveryClient and @EnableOAuth2Sso are both required.
At this time, ResourceServerConfig and SecurityConfig also need to be configured.
If you need a database, you can add:
package com.damon.config;
import java.util.Properties;
import javax.sql.DataSource;
import org.apache.ibatis.plugin.Interceptor;
import org.apache.ibatis.session.SqlSessionFactory;
import org.mybatis.spring.SqlSessionFactoryBean;
import org.mybatis.spring.SqlSessionTemplate;
import org.mybatis.spring.annotation.MapperScan;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.core.env.Environment;
import org.springframework.core.io.support.PathMatchingResourcePatternResolver;
import org.springframework.jdbc.datasource.DataSourceTransactionManager;
import org.springframework.stereotype.Component;
import org.springframework.transaction.annotation.EnableTransactionManagement;
import com.alibaba.druid.pool.DruidDataSourceFactory;
import com.github.pagehelper.PageHelper;
/**
*
*
* created by Damon
* 2018年5月23日 下午7:39:37
*
*/
@Component
@Configuration
@EnableTransactionManagement
@MapperScan("com.damon.*.dao")
public class MybaitsConfig {
@Autowired
private EnvConfig envConfig;
@Autowired
private Environment env;
@Bean(name = "dataSource")
public DataSource getDataSource() throws Exception {
Properties props = new Properties();
props.put("driverClassName", envConfig.getJdbc_driverClassName());
props.put("url", envConfig.getJdbc_url());
props.put("username", envConfig.getJdbc_username());
props.put("password", envConfig.getJdbc_password());
return DruidDataSourceFactory.createDataSource(props);
}
@Bean
public SqlSessionFactory sqlSessionFactory(@Qualifier("dataSource") DataSource dataSource) throws Exception {
SqlSessionFactoryBean fb = new SqlSessionFactoryBean();
// 指定数据源(这个必须有,否则报错)
fb.setDataSource(dataSource);
// 下边两句仅仅用于*.xml文件,如果整个持久层操作不需要使用到xml文件的话(只用注解就可以搞定),则不加
fb.setTypeAliasesPackage(env.getProperty("mybatis.typeAliasesPackage"));// 指定基包
fb.setMapperLocations(new PathMatchingResourcePatternResolver().getResources(env.getProperty("mybatis.mapperLocations")));// 指定xml文件位置
// 分页插件
PageHelper pageHelper = new PageHelper();
Properties props = new Properties();
// 启用合理化时,如果pageNum<1会查询第一页,如果pageNum>pages会查询最后一页
//禁用合理化时,如果pageNum<1或pageNum>pages会返回空数据
props.setProperty("reasonable", "true");
//指定数据库
props.setProperty("dialect", "mysql");
//支持通过Mapper接口参数来传递分页参数
props.setProperty("supportMethodsArguments", "true");
//总是返回PageInfo类型,check检查返回类型是否为PageInfo,none返回Page
props.setProperty("returnPageInfo", "check");
props.setProperty("params", "count=countSql");
pageHelper.setProperties(props);
// 添加插件
fb.setPlugins(new Interceptor[] { pageHelper });
try {
return fb.getObject();
} catch (Exception e) {
throw e;
}
}
/**
* 配置事务管理器
* @param dataSource
* @return
* @throws Exception
*/
@Bean
public DataSourceTransactionManager transactionManager(DataSource dataSource) throws Exception {
return new DataSourceTransactionManager(dataSource);
}
@Bean
public SqlSessionTemplate sqlSessionTemplate(SqlSessionFactory sqlSessionFactory) {
return new SqlSessionTemplate(sqlSessionFactory);
}
}
Next, write a new controller class:
package com.damon.user.controller;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.security.access.prepost.PreAuthorize;
import org.springframework.security.core.Authentication;
import org.springframework.security.oauth2.provider.authentication.OAuth2AuthenticationDetails;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
import com.damon.commons.Response;
import com.damon.user.service.UserService;
/**
*
*
* @author Damon
* @date 2020年1月13日 下午3:31:07
*
*/
@RestController
@RequestMapping("/api/user")
public class UserController {
private static final Logger logger = LoggerFactory.getLogger(UserController.class);
@Autowired
private UserService userService;
@GetMapping("/getCurrentUser")
@PreAuthorize("hasAuthority('admin')")
public Object getCurrentUser(Authentication authentication) {
logger.info("test password mode");
return authentication;
}
@PreAuthorize("hasAuthority('admin')")
@GetMapping("/auth/admin")
public Object adminAuth() {
logger.info("test password mode");
return "Has admin auth!";
}
@GetMapping(value = "/get")
@PreAuthorize("hasAuthority('admin')")
//@PreAuthorize("hasRole('admin')")//无效
public Object get(Authentication authentication){
//Authentication authentication = SecurityContextHolder.getContext().getAuthentication();
authentication.getCredentials();
OAuth2AuthenticationDetails details = (OAuth2AuthenticationDetails)authentication.getDetails();
String token = details.getTokenValue();
return token;
}
@GetMapping("/getUserInfo")
@PreAuthorize("hasAuthority('admin')")
public Response<Object> getUserInfo(Authentication authentication) {
logger.info("test password mode");
Object principal = authentication.getPrincipal();
if(principal instanceof String) {
String username = (String) principal;
return userService.getUserByUsername(username);
}
return null;
}
}
Basically one code is completed. Next test:
Certification:
curl -i -X POST -d "username=admin&password=123456&grant_type=password&client_id=provider-service&client_secret=provider-service-123" http://localhost:5555/oauth-cas/oauth/token
After getting the token:
curl -i -H "Accept: application/json" -H "Authorization:bearer f4a42baa-a24a-4342-a00b-32cb135afce9" -X GET http://localhost:5555/provider-service/api/user/getCurrentUser
Port 5555 is used here. This is a gateway service. Well, since this is mentioned, let's look at the gateway next and introduce dependencies:
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.damon</groupId>
<artifactId>alibaba-gateway</artifactId>
<version>0.0.1-SNAPSHOT</version>
<packaging>jar</packaging>
<name>alibaba-gateway</name>
<url>http://maven.apache.org</url>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.1.8.RELEASE</version>
<relativePath/>
</parent>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
<java.version>1.8</java.version>
<swagger.version>2.6.1</swagger.version>
<xstream.version>1.4.7</xstream.version>
<pageHelper.version>4.1.6</pageHelper.version>
<fastjson.version>1.2.51</fastjson.version>
<!-- <springcloud.version>2.1.8.RELEASE</springcloud.version> -->
<springcloud.version>Greenwich.SR3</springcloud.version>
<springcloud.kubernetes.version>1.1.1.RELEASE</springcloud.kubernetes.version>
<mysql.version>5.1.46</mysql.version>
<alibaba-cloud.version>2.1.1.RELEASE</alibaba-cloud.version>
<springcloud.alibaba.version>0.9.0.RELEASE</springcloud.alibaba.version>
</properties>
<dependencyManagement>
<dependencies>
<!-- <dependency>
<groupId>com.alibaba.cloud</groupId>
<artifactId>spring-cloud-alibaba-dependencies</artifactId>
<version>${alibaba-cloud.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency> -->
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-alibaba-dependencies</artifactId>
<version>${springcloud.alibaba.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-dependencies</artifactId>
<version>${springcloud.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
<dependencies>
<!-- 不要依赖spring-boot-starter-web,会和spring-cloud-starter-gateway冲突,启动时异常 -->
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-gateway</artifactId>
</dependency>
<!--基于 reactive stream 的redis -->
<!-- <dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-redis-reactive</artifactId>
</dependency> -->
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-commons</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-alibaba-nacos-discovery</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-netflix-ribbon</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-netflix-hystrix</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.alibaba</groupId>
<artifactId>fastjson</artifactId>
<version>${fastjson.version}</version>
</dependency>
<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
<version>19.0</version>
</dependency>
</dependencies>
<build>
<finalName>${project.artifactId}</finalName>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<configuration>
<jvmArguments>-Dfile.encoding=UTF-8</jvmArguments>
<fork>true</fork>
</configuration>
</plugin>
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.7.8</version>
<executions>
<execution>
<goals>
<goal>prepare-agent</goal>
<goal>report</goal>
</goals>
</execution>
</executions>
</plugin>
<!-- 自动生成代码 插件 begin -->
<!-- <plugin>
<groupId>org.mybatis.generator</groupId>
<artifactId>mybatis-generator-maven-plugin</artifactId>
<version>1.3.2</version>
<configuration>
<configurationFile>src/main/resources/generatorConfig.xml</configurationFile>
<verbose>true</verbose>
<overwrite>true</overwrite>
</configuration>
<dependencies>
<dependency>
<groupId>org.mybatis.generator</groupId>
<artifactId>mybatis-generator-core</artifactId>
<version>1.3.2</version>
</dependency>
</dependencies>
</plugin> -->
</plugins>
</build>
</project>
Also use Nacos to discover services.
The registration configuration here is:
spring:
cloud:
gateway:
discovery:
locator:
enabled: true #并且我们并没有给每一个服务单独配置路由 而是使用了服务发现自动注册路由的方式
lowerCaseServiceId: true
nacos:
discovery:
server-addr: 127.0.0.1:8848
config:
server-addr: 127.0.0.1:8848
refreshable-dataids: actuator.properties,log.properties
The previous one is kubernetes.
Well, after the gateway is configured, you can start the Nacos dashboard and you can see the service, indicating that the registration service is successful. Then you can use it to call other services. Specific curl command:
curl -i -H "Accept: application/json" -H "Authorization:bearer f4a42baa-a24a-4342-a00b-32cb135afce9" -X GET http://localhost:5555/consumer-service/api/order/getUserInfo
Ok, now the authentication center, service provider, service consumer, service registration and discovery, configuration center and other functions have been completed.
Why choose Netty as the technical framework for instant messaging
Introduction
Netty is a high-performance, asynchronous event-driven NIO framework, which provides support for TCP, UDP and file transfer. As the most popular NIO framework, Netty has been widely used in the Internet field, big data distributed computing field, game industry, communication industry, etc.
Features
- high concurrency
- transmission
- packaged
Advantages of Netty Communication
Netty is a high-performance, highly scalable asynchronous event-driven network application framework, which greatly simplifies network programming such as TCP and UDP client and server development. Its four important contents are:
- Memory management: Enhanced ByteBuf buffer
- Reactor threading model: a high-performance multi-threaded programming
- Enhanced channel concept
- ChannelPipeline responsibility chain design pattern: event handling mechanism
Netty implements the Reactor thread model. The Reactor model has four core concepts: Resources (requests/tasks), Synchronous Event Demultiplexer, Dispatcher, and Request Handler. Mainly through two EventLoopGroup (thread group, the bottom layer is the thread pool of the JDK) to separately handle the connection and data reading, thereby improving the utilization of threads.
Channel in Netty is an abstract concept, which can be understood as an enhancement and expansion of JDK NIO Channel. Many attributes and methods have been added.
The ChannelPipeline responsibility chain saves all processor information of the channel. When a new channel is created, a dedicated pipeline is automatically created, and the corresponding inbound event (usually refers to the inbound data generated by the I/O thread, see ChannelInboundHandler for details) and outbound event (often refers to the actual execution of the I/O thread) For output operations, refer to ChannelOutboundHandler) when calling the processor on the pipeline. When an event is inbound, the execution order is from first to last of the pipeline. When an outbound event occurs, the execution order is from last to first in the pipeline. The order of the processors in the pipeline is determined when they are added.
JDK's ByteBuffer has problems such as inability to dynamically expand its capacity and complicated API usage. Netty's own ByteBuf solves the problems. ByteBuf has implemented four enhancements: convenient API operation, dynamic expansion, multiple implementations of ByteBuf, and efficient zero-copy mechanism.
Realize a simple Netty client and server communication
Actual server
The advantages and characteristics of Netty's practice in the audio and video basins were introduced earlier. Next, let's write a server first. First create a Java project:
After creating the project, we need to introduce basic dependencies:
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.damon</groupId>
<artifactId>netty-client-service</artifactId>
<version>0.0.1-SNAPSHOT</version>
<packaging>jar</packaging>
<name>netty-client-service</name>
<url>http://maven.apache.org</url>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.1.1.RELEASE</version>
<relativePath/>
</parent>
<properties>
<java.version>1.8</java.version>
<spring-boot.version>2.1.1.RELEASE</spring-boot.version>
<springcloud.kubernetes.version>1.0.1.RELEASE</springcloud.kubernetes.version>
<springcloud.version>2.1.1.RELEASE</springcloud.version>
<swagger.version>2.6.1</swagger.version>
<fastjson.version>1.2.51</fastjson.version>
<pageHelper.version>4.1.6</pageHelper.version>
<protostuff.version>1.0.10</protostuff.version>
<objenesis.version>2.4</objenesis.version>
</properties>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-dependencies</artifactId>
<type>pom</type>
<scope>import</scope>
<version>${spring-boot.version}</version>
</dependency>
</dependencies>
</dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
<exclusions>
<exclusion>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-tomcat</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-undertow</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<!-- <dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-kubernetes-core</artifactId>
<version>${springcloud.kubernetes.version}</version>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-kubernetes-discovery</artifactId>
<version>${springcloud.kubernetes.version}</version>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-kubernetes-ribbon</artifactId>
<version>${springcloud.kubernetes.version}</version>
</dependency> -->
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-commons</artifactId>
<version>${springcloud.version}</version>
</dependency>
<dependency>
<groupId>com.alibaba</groupId>
<artifactId>fastjson</artifactId>
<version>${fastjson.version}</version>
</dependency>
<dependency>
<groupId>org.jsoup</groupId>
<artifactId>jsoup</artifactId>
<version>1.11.3</version>
</dependency>
<!-- swagger -->
<dependency>
<groupId>io.springfox</groupId>
<artifactId>springfox-swagger2</artifactId>
<version>${swagger.version}</version>
</dependency>
<dependency>
<groupId>io.springfox</groupId>
<artifactId>springfox-swagger-ui</artifactId>
<version>${swagger.version}</version>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-lang3</artifactId>
</dependency>
<dependency>
<groupId>commons-collections</groupId>
<artifactId>commons-collections</artifactId>
<version>3.2.2</version>
</dependency>
<!-- mybatis -->
<!-- <dependency>
<groupId>org.mybatis.spring.boot</groupId>
<artifactId>mybatis-spring-boot-starter</artifactId>
<version>1.1.1</version>
</dependency> -->
<dependency>
<groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId>
</dependency>
<!-- <dependency>
<groupId>org.bytedeco</groupId>
<artifactId>javacv-platform</artifactId>
<version>1.4.1</version>
</dependency> -->
<!-- <dependency>
<groupId>org.bytedeco.javacpp-presets</groupId>
<artifactId>opencv-platform</artifactId>
<version>3.4.1-1.4.1</version>
</dependency> -->
<dependency>
<groupId>io.netty</groupId>
<artifactId>netty-all</artifactId>
<version>4.1.64.Final</version>
</dependency>
<!-- protobuf -->
<dependency>
<groupId>com.google.protobuf</groupId>
<artifactId>protobuf-java</artifactId>
<version>3.5.0</version>
</dependency>
<dependency>
<groupId>com.googlecode.protobuf-java-format</groupId>
<artifactId>protobuf-java-format</artifactId>
<version>1.2</version>
</dependency>
<dependency>
<groupId>com.dyuproject.protostuff</groupId>
<artifactId>protostuff-core</artifactId>
<version>${protostuff.version}</version>
</dependency>
<dependency>
<groupId>com.dyuproject.protostuff</groupId>
<artifactId>protostuff-runtime</artifactId>
<version>${protostuff.version}</version>
</dependency>
<dependency>
<groupId>org.objenesis</groupId>
<artifactId>objenesis</artifactId>
<version>${objenesis.version}</version>
</dependency>
</dependencies>
</project>
Service startup class:
@EnableScheduling
@SpringBootApplication(scanBasePackages = { "com.damon" })
public class StorageServer {
public static void main(String[] args) {
SpringApplication.run(StorageServer.class, args);
}
}
When starting the netty service first, we only need to add the Netty configuration:
spring.application.name=netty-server
server.port=2002
netty.host=127.0.0.1
netty.port=9999
logging.path=/data/${spring.application.name}/logs
spring.profiles.active=dev
spring.http.encoding.charset=UTF-8
spring.http.encoding.enabled=true
spring.http.encoding.force=true
spring.mvc.throw-exception-if-no-handler-found=true
server.undertow.accesslog.enabled=false
server.undertow.accesslog.pattern=combined
client.http.request.readTimeout=30000
client.http.request.connectTimeout=8000
After adding the configuration, we can start the service to see, at this time there is a log:
After adding the netty service configuration, here you need to inject a Server Handle, which is used when the client actively connects to the server. At this time, the processing class will be triggered to execute some messages:
@Override
public void channelActive(ChannelHandlerContext ctx) throws Exception {
SocketChannel channel = (SocketChannel) ctx.channel();
logger.info("链接报告开始");
logger.info("链接报告信息:有一客户端链接到本服务端");
logger.info("链接报告IP:{}", channel.localAddress().getHostString());
logger.info("链接报告Port:{}", channel.localAddress().getPort());
logger.info("链接报告完毕");
ChannelHandler.channelGroup.add(ctx.channel());
// 通知客户端链接建立成功
String str = "通知客户端链接建立成功" + " " + new Date() + " " + channel.localAddress().getHostString() + "\r\n";
ByteBuf buf = Unpooled.buffer(str.getBytes().length);
buf.writeBytes(str.getBytes("GBK"));
ctx.writeAndFlush(buf);
}
This means that if a client connects to the server at this time, some information will be printed. Here is the result of printing after I joined the client in advance:
When the client actively disconnects from the server, this channel is inactive. In other words, the client and server have closed the communication channel and cannot transmit data:
@Override
public void channelInactive(ChannelHandlerContext ctx) throws Exception {
logger.info("客户端断开链接{}", ctx.channel().localAddress().toString());
ChannelHandler.channelGroup.remove(ctx.channel());
}
Of course the function to get data is here:
@Override
public void channelRead(ChannelHandlerContext ctx, Object msg) throws UnsupportedEncodingException {
if(msg instanceof ByteBuf) {
ByteBuf buf = (ByteBuf) msg;
byte[] msgByte = new byte[buf.readableBytes()];
buf.readBytes(msgByte);
System.out.println(new String(msgByte, Charset.forName("GBK")));
//通知客户端链消息发送成功
String str = "服务端收到:" + new Date() + " " + new String(msgByte, Charset.forName("GBK")) + "\r\n";
ByteBuf buf2 = Unpooled.buffer(str.getBytes().length);
buf2.writeBytes(str.getBytes("GBK"));
ctx.writeAndFlush(buf2);
}
}
If an exception occurs, catch the exception. When an exception occurs, you can do some corresponding treatments, such as printing logs and closing the link :
@Override
public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) throws Exception {
ctx.close();
logger.info("异常信息:\r\n" + cause.getMessage());
}
In addition, on the server side, it is generally necessary to define some information protocol information, such as: connection information, whether it is spontaneous or mass-sent, which communication channel is, and communication information, etc.:
public class ServerMsgProtocol {
private int type; //链接信息;1自发信息、2群发消息
private String channelId; //通信管道ID,实际使用中会映射成用户名
private String userHeadImg; //用户头像[模拟分配]
private String msgInfo; //通信消息
public int getType() {
return type;
}
public void setType(int type) {
this.type = type;
}
public String getChannelId() {
return channelId;
}
public void setChannelId(String channelId) {
this.channelId = channelId;
}
public String getUserHeadImg() {
return userHeadImg;
}
public void setUserHeadImg(String userHeadImg) {
this.userHeadImg = userHeadImg;
}
public String getMsgInfo() {
return msgInfo;
}
public void setMsgInfo(String msgInfo) {
this.msgInfo = msgInfo;
}
}
The above is a simple server, it is quite clear to sort out.
Actual client
Next, let's see how the client connects to the server and communicates with it? If the client wants to communicate with the server, it must first connect with the server. Here, add a configuration server NIO thread group:
private EventLoopGroup workerGroup = new NioEventLoopGroup();
private Channel channel;
The logic of connecting to the server is:
public ChannelFuture connect(String inetHost, int inetPort) {
ChannelFuture channelFuture = null;
try {
Bootstrap b = new Bootstrap();
b.group(workerGroup);
b.channel(NioSocketChannel.class);
b.option(ChannelOption.AUTO_READ, true);
b.handler(new MyChannelInitializer());
channelFuture = b.connect(inetHost, inetPort).syncUninterruptibly();
this.channel = channelFuture.channel();
channel.closeFuture();
} catch (Exception e) {
e.printStackTrace();
} finally {
if (null != channelFuture && channelFuture.isSuccess()) {
System.out.println("demo-netty client start done.");
} else {
System.out.println("demo-netty client start error.");
}
}
return channelFuture;
}
Next, let's look at how to destroy the connection:
public void destroy() {
if (null == channel) return;
channel.close();
workerGroup.shutdownGracefully();
}
Finally, let's connect to the server:
new NettyClient().connect("127.0.0.1", 9999);
Since the netty ip and port of our server are set to: local, port 9999, configure directly here.
Similarly, if the client needs to receive data information, it also needs to define how to receive it in the pipeline:
public class MyChannelInitializer extends ChannelInitializer<SocketChannel> {
@Override
protected void initChannel(SocketChannel channel) throws Exception {
// 在管道中添加我们自己的接收数据实现方法
channel.pipeline().addLast(new MyClientHandler());
}
}
When the client actively connects to the server's link, the channel is active. That is, the client and the server have established a communication channel and can transmit data:
@Override
public void channelActive(ChannelHandlerContext ctx) throws Exception {
SocketChannel channel = (SocketChannel) ctx.channel();
System.out.println("链接报告开始");
System.out.println("链接报告信息:本客户端链接到服务端。channelId:" + channel.id());
System.out.println("链接报告IP:" + channel.localAddress().getHostString());
System.out.println("链接报告Port:" + channel.localAddress().getPort());
System.out.println("链接报告完毕");
}
When the client actively disconnects the server, the channel is inactive. In other words, the client and server have closed the communication channel and cannot transmit data:
@Override
public void channelInactive(ChannelHandlerContext ctx) throws Exception {
System.out.println("断开链接" + ctx.channel().localAddress().toString());
super.channelInactive(ctx);
}
When you encounter an exception, catch the exception. When an exception occurs, you can do some corresponding processing, such as printing the log and closing the link:
@Override
public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) throws Exception {
ctx.close();
System.out.println("异常信息:\r\n" + cause.getMessage());
}
After the client connects to the server, processes the information sent by the server, and handles exceptions, we will start the client at this time, and the client control surface will print the following information:
If the client actively disconnects, at this time, the server will prompt:
远程主机强迫关闭了一个现有的连接。
2021-05-13 19:33:35.691 INFO 148736 --- [ntLoopGroup-3-2] com.leinao.handler.ServerHandler : 客户端断开链接/127.0.0.1:9999
At this point, a simple Netty client-server communication is complete.
The actual combat chat system under the microservice Springboot
In the previous section, we introduced a simple example of Netty client and server communication. Next, we will start the actual chat system.
websocket server startup class
Based on the features of Netty mentioned earlier, the chat room here needs front and back ends. So, first of all, for the backend, we need to create a Websocket Server, here we need a pair of thread group EventLoopGroup, after the definition, we need to define a Server:
public static void main(String[] args) throws Exception {
EventLoopGroup mainGroup = new NioEventLoopGroup();
EventLoopGroup subGroup = new NioEventLoopGroup();
try {
ServerBootstrap server = new ServerBootstrap();
server.group(mainGroup, subGroup)
.channel(NioServerSocketChannel.class)
.childHandler(new WSServerInitialzer());
ChannelFuture future = server.bind(8088).sync();
future.channel().closeFuture().sync();
} finally {
mainGroup.shutdownGracefully();
subGroup.shutdownGracefully();
}
}
Add the thread group to the Server, next, you need to set up a channel: NioServerSocketChannel, and an initializer: WSServerInitialzer.
The second step is to bind the port version of the Server:
ChannelFuture future = server.bind(8088).sync()
Finally, the future needs to be monitored. And after the monitoring is over, the thread resources need to be closed:
mainGroup.shutdownGracefully();
subGroup.shutdownGracefully();
websocket sub-processor initialzer
The WebSocket Server mentioned above, then for the socket, there is an initialization processor, here we define one:
public class WSServerInitialzer extends ChannelInitializer<SocketChannel> {
@Override
protected void initChannel(SocketChannel ch) throws Exception {
ChannelPipeline pipeline = ch.pipeline();
pipeline.addLast(new HttpServerCodec());
pipeline.addLast(new ChunkedWriteHandler());
pipeline.addLast(new HttpObjectAggregator(1024*64));
pipeline.addLast(new WebSocketServerProtocolHandler("/ws"));
pipeline.addLast(new ChatHandler());
}
}
Since websocket is based on the http protocol, it needs to have the http codec HttpServerCodec. At the same time, on some http, there are some data stream processing, and the data stream is large or small, then you can add a large data stream processing: ChunkedWriteHandler.
Usually, httpMessage will be aggregated into FullHttpRequest or FullHttpResponse, and almost all programming in netty will use this hanler.
In addition, the protocol handled by the websocket server is used to specify the route for client connection access: "/ws". This handler will help you deal with some heavy and complicated things, for example, it will help you handle the handshake action: handshaking(close, ping, pong) ping + pong = heartbeat. For websocket, all are transmitted in frames, and different data types correspond to different frames.
Finally, we have customized a handler for processing messages: ChatHandler.
chatHandler for message processing
In Netty, there is an object TextWebSocketFrame that is used to process text specifically for websocket, and frame is the carrier of the message.
public class ChatHandler extends SimpleChannelInboundHandler<TextWebSocketFrame> {
private static ChannelGroup clients = new DefaultChannelGroup(GlobalEventExecutor.INSTANCE);
@Override
protected void channelRead0(ChannelHandlerContext ctx, TextWebSocketFrame msg)
throws Exception {
String content = msg.text();
System.out.println("接受到的数据:" + content);
clients.writeAndFlush(new TextWebSocketFrame("服务器时间在 " + LocalDateTime.now() + " 接受到消息, 消息为:" + content));
}
@Override
public void handlerAdded(ChannelHandlerContext ctx) throws Exception {
clients.add(ctx.channel());
System.out.println("客户端连接,channle对应的长id为:" + ctx.channel().id().asLongText());
System.out.println("客户端连接,channle对应的短id为:" + ctx.channel().id().asShortText());
}
@Override
public void handlerRemoved(ChannelHandlerContext ctx) throws Exception {
System.out.println("客户端断开,channle对应的长id为:" + ctx.channel().id().asLongText());
System.out.println("客户端断开,channle对应的短id为:" + ctx.channel().id().asShortText());
}
}
At the beginning, the message is in the carrier TextWebSocketFrame. At this time, you can get the content directly and print it out. And can send the message to the client corresponding to the request. Of course, the message can also be forwarded to all clients, which involves the channel in Netty. At this time, users in the channel need to be managed so that the message can be forwarded to all channel users. That is, the handlerAdded function above. When the client connects to the server, it opens the connection, gets the client's channle, and puts it in the ChannelGroup for management. At the same time, after the client disconnects from the server and closes the connection, the handlerRemoved function will be triggered, and the ChannelGroup will automatically remove the corresponding client channel.
Next, you need to refresh the data to all clients after obtaining:
for (Channel channel : clients) {
channel.writeAndFlush(new TextWebSocketFrame("[服务器在]" + LocalDateTime.now() + "接受到消息, 消息为:" + content));
}
Note: Here you need to use the carrier to flush the information, because the writeAndFlush function needs to pass the object carrier, not the direct string. In fact, as ChannelGroup clients, it provides the writeAndFlush function, which can directly output to all clients:
clients.writeAndFlush(new TextWebSocketFrame("服务器时间在 " + LocalDateTime.now() + " 接受到消息, 消息为:" + content));
Introduction to js-based websocket related api
First of all, a connection between the client and the server is required. This connection bridge is a socket in js:
var socket = new WebSocket("ws://192.168.174.145:8088/ws");
Let's take a look at its life cycle. In the backend, channel has its life cycle, while in the front-end socket:
- onopen(), when the client establishes a connection with the server, the onopen event will be triggered
- onmessage(), when the client receives a message, it will trigger the onmessage event
- onerror(), when an exception occurs, the front end will trigger an onerror event
- onclose(), after the connection between the client and the server is closed, the onclose event will be triggered
Next look at two proactive methods:
- Socket.send(), after the front end actively obtains the content, the message is sent through send
- Socket.close(), when the user triggers a button, the connection between the client and the server will be disconnected
The above is the api corresponding to the front-end websocket js.
Implement front-end websocket
RTE开发者社区
RTE 开发者社区是聚焦实时互动领域的中立开发者社区。不止于纯粹的技术交流,我们相信开发者具备更加丰盈的个体价值。行业发展变革、开发者职涯发展、技术创业创新资源,我们将陪跑开发者,共享、共建、共成长。
**粗体** _斜体_ [链接](http://example.com) `代码` - 列表 > 引用
。你还可以使用@
来通知其他用户。