Kratos distributed transaction practice

background

With the rapid development of business and the increasing complexity of business, microservices, as one of the best solutions, decouple services, reduce complexity and increase maintainability, but also bring some new problems.

When we need to ensure data consistency across services, the original database transaction is not enough, and it is impossible to put multiple operations across databases and services in one transaction. There are many such application scenarios, we can list many:

• In the case of inter-bank transfer, the data is not in the same database, but it is necessary to ensure that the balance deduction and balance increase either succeed or fail at the same time.
• After an article is published, update statistics such as the total number of articles. where publishing articles and updating statistics are usually in different microservices
• Order system after microservice
• Travel needs to book several tickets at the same time in the third-party system

Faced with these scenarios where local transactions cannot be resolved, we need a distributed transaction solution to ensure the consistency of data updates across services and databases.

As a very popular distributed transaction framework, dtm has already supported access to a variety of microservice frameworks. Let's focus on how go-kratos can access dtm to solve the problem of distributed transactions

run an example

Let's look at a runnable example, and then see how to develop a complete distributed transaction by ourselves

The following uses etcd as the registration service center, you can run an example of kratos as follows:

• configure dtm

  MicroService:
   Driver: 'dtm-driver-kratos' # name of the driver to handle register/discover
   Target: 'discovery://127.0.0.1:2379/dtmservice' # register dtm server to this url
   EndPoint: 'grpc://localhost:36790'

• start etcd

  # 前提：已安装etcd
  etcd

• start dtm

  # 请先配置好dtm的数据库
  go run app/main.go -c conf.yml # conf.yml 为你对应的 dtm 配置文件

• Running a kratos service

  git clone https://github.com/dtm-labs/dtmdriver-clients && cd dtmdriver-clients
  cd kratos/trans
  make build && ./bin/trans -conf configs/config.yaml

• Initiate a kratos transaction using dtm

  # 在 dtmdriver-clients 的目录下
  cd kratos/app && go run main.go

When you see in trans's log

INFO msg=config loaded: config.yaml format: yaml
INFO msg=[gRPC] server listening on: [::]:9000
2022/03/30 09:35:36 transfer out 30 cents from 1
2022/03/30 09:35:36 transfer in 30 cents to 2

That means the transaction is completed normally

development access

Refer to the code of dtm-labs/dtmdriver-clients

// 下面这些导入 kratos 的 dtm 驱动
import (
    _ "github.com/dtm-labs/driver-kratos"
)

// dtm 已经经过前面的配置，注册到下面这个地址，因此在 dtmgrpc 中使用该地址
var dtmServer = "discovery://localhost:2379/dtmservice"

// 业务地址，下面的 busi 换成实际在 server 初始化设置的名字
var busiServer = "discovery://localhost:2379/busi"

// 发起一个msg事务，保证TransOut和TransIn都会完成
gid := dtmgrpc.MustGenGid(dtmServer)
m := dtmgrpc.NewMsgGrpc(dtmServer, gid).
  Add(busiServer+"/api.trans.v1.Trans/TransOut", &busi.BusiReq{Amount: 30, UserId: 1}).
  Add(busiServer+"/api.trans.v1.Trans/TransIn", &busi.BusiReq{Amount: 30, UserId: 2})
m.WaitResult = true
err := m.Submit()
logger.FatalIfError(err)

Deep understanding of dynamic invocation

When kratos uses dtm's distributed transactions, many calls are initiated from the dtm server, such as TCC's Confirm/Cancel, and all SAGA/MSG calls.

The dtm does not need to know the strong typing of the relevant business APIs that make up the distributed transaction, it calls these APIs dynamically.

The invocation of grpc can be analogous to HTTP POST, where:

• "/api.trans.v1.Trans/TransIn" is equivalent to Path in the URL. Please note that this Path must be found from the Invoke function implementation of TransIn
• &busi.BusiReq{Amount: 30, UserId: 1} is equivalent to Body in Post
• v1.Response is equivalent to the response of the HTTP request

Through the following part of the code, dtm gets the complete information and can initiate a complete call

Add(busiServer+"/api.trans.v1.Trans/TransIn", &busi.BusiReq{Amount: 30, UserId: 1})

Access by other means

There are other ways of kratos microservices that are not etcd, and their access methods are listed below

direct connection

For direct connection, you only need to set Target to an empty string based on the etcd configuration of dtm above.

In the case of direct connection, there is no need to register the dtm with the registration center

summary

Welcome to use dtm , and support us with star to build a golang microservice ecosystem together