The seven most classic solutions for distributed transactions

叶东富
中文

With the rapid development of the business and the increasing complexity of the business, almost every company's system will move from a single unit to a distributed system, especially to a microservice architecture. Then it will inevitably encounter the problem of distributed transactions.

This article first introduces the relevant basic theories, then summarizes the most classic transaction schemes, and finally gives solutions to the out-of-order execution of sub-transactions (idempotence, null compensation, suspension problems), and shares with everyone.

Basic theory

Before explaining the specific plan, let's first understand the basic theoretical knowledge involved in distributed firms.

Let's take the transfer as an example. A needs to transfer 100 yuan to B, then the balance of A needs to be -100 yuan, and the balance of B +100 yuan. The entire transfer must be guaranteed, A-100 and B+100 succeed at the same time, or fail at the same time . Take a look at how to solve this problem in various scenarios.

Affairs

The function of operating multiple statements as a whole is called a database transaction. A database transaction can ensure that all operations within the scope of the transaction can all succeed or all fail.

Transaction has four attributes: atomicity, consistency, isolation, and durability. These four attributes are commonly referred to as ACID characteristics.

  • Atomicity (atomicity): All operations in a transaction are either completed or not completed at all, and will not end in a certain link in the middle. If an error occurs during the execution of the transaction, it will be restored to the state before the transaction started, as if the transaction had never been executed.
  • Consistency (consistency): Before the start of the transaction and after the end of the transaction, the integrity of the database has not been destroyed. Integrity, including foreign key constraints, application-defined constraints, etc. will not be destroyed.
  • Isolation: The ability of the database to allow multiple concurrent transactions to read, write and modify its data at the same time. Isolation can prevent data inconsistencies caused by cross execution when multiple transactions are executed concurrently.
  • Durability: After the transaction is completed, the modification of the data is permanent, and it will not be lost even if the system fails.

If our business system is not complicated and can modify the data in a database and a service to complete the transfer, then we can use database transactions to ensure the correct completion of the transfer business.

Distributed transaction

The cross-bank transfer service of a bank is a typical distributed transaction scenario. Assuming that A needs to transfer money across banks to B, then the data of two banks is involved. The ACID of the transfer cannot be guaranteed by the local transaction of a database, and it can only be solved by distributed transactions.

Distributed transaction means that the initiator of the transaction, the resource and resource manager, and the transaction coordinator are located on different nodes of the distributed system. In the above transfer business, the user A-100 operation and the user B+100 operation are not located on the same node. Essentially, distributed transactions are to ensure the correct execution of data operations in distributed scenarios.

Distributed transactions in a distributed environment, in order to meet the needs of availability, performance and degraded services, and reduce the requirements of consistency and isolation, on the one hand, follow the BASE theory (BASE related theory, involving a lot of content, interested students, you can (Refer to BASE theory):

  • Basic Availability (Basic Availability)
  • Soft state
  • Eventual consistency (Eventual consistency)

Similarly, distributed transactions also partially follow the ACID specification:

  • Atomicity: strictly follow
  • Consistency: The consistency after the transaction is completed is strictly followed; the consistency in the transaction can be appropriately relaxed
  • Isolation: Parallel transactions cannot be affected; the visibility of the intermediate results of the transaction allows security relaxation
  • Persistence: Strictly follow

Distributed transaction solution

Due to the distributed transaction scheme, the complete ACID guarantee cannot be achieved, and there is no perfect scheme that can solve all business problems. Therefore, in actual applications, the most suitable distributed transaction solution will be selected according to the different characteristics of the business.

Two-phase commit/XA

XA is a specification for distributed transactions proposed by the X/Open organization. The XA specification mainly defines the interface between the (global) transaction manager (TM) and the (local) resource manager (RM). Local databases such as mysql play the role of RM in XA

XA is divided into two stages:

The first stage (prepare): That is, all participants RM prepare to execute the transaction and lock the required resources. When the participant is ready, report to TM that it is ready.
The second stage (commit/rollback): When the transaction manager (TM) confirms that all participants (RM) are ready, send a commit command to all participants.
At present, mainstream databases basically support XA transactions, including mysql, oracle, sqlserver, and postgre

XA transactions are composed of one or more resource managers (RM), a transaction manager (TM), and an application program (ApplicationProgram).

The three roles of RM, TM, and AP here are classic role divisions, which will run through subsequent transaction modes such as Saga and Tcc.

Taking the above transfer as an example, the sequence diagram of a successfully completed XA transaction is as follows:
image.png

If any participant fails to prepare, TM will notify all participants who completed the preparation to roll back.

The characteristics of XA transactions are:

  • Simple and easy to understand, easy to develop
  • The resource has been locked for a long time, and the concurrency is low

If readers want to further study XA, go language, PHP, Python, Java, C#, Node, etc. can refer to DTM

SAGA

Saga is a scheme mentioned in this database paper sagas. The core idea is to split a long transaction into multiple local short transactions, which are coordinated by the Saga transaction coordinator. If it ends normally, it will be completed normally. If a step fails, the compensation operation will be called once in the reverse order.

Taking the above transfer as an example, the sequence diagram of a successfully completed SAGA transaction is as follows:

image.png

Once Saga reaches the Cancel stage, Cancel is not allowed to fail in business logic. If no success is returned due to network or other temporary failures, then TM will continue to retry until Cancel returns success.

Features of Saga transaction:

  • High concurrency, no need to lock resources for a long time like XA transactions
  • Need to define normal operation and compensation operation, the amount of development is larger than XA
  • The consistency is weak. For transfers, it may happen that user A has deducted the money and the transfer fails at the end

There are many SAGA content in the paper, including two recovery strategies, including concurrent execution of branch transactions. Our discussion here only includes the simplest SAGA

SAGA is suitable for many scenarios, suitable for long transactions, and suitable for business scenarios that are not sensitive to intermediate results

If readers want to study SAGA further, please refer to DTM , which includes examples of SAGA's success and failure rollback, as well as the handling of various network exceptions.

TCC

The concept of TCC (Try-Confirm-Cancel) was first proposed by Pat Helland in a paper entitled "Life beyond Distributed Transactions: an Apostate's Opinion" published in 2007.

TCC is divided into 3 stages

  • Try phase: try to execute, complete all business checks (consistency), reserve necessary business resources (quasi-isolation)
  • Confirm phase: Confirm that the actual execution of the business is performed without any business checks. Only the business resources reserved in the Try phase are used. The Confirm operation requires an idempotent design. After the Confirm fails, a retry is required.
  • Cancel phase: Cancel the execution and release the business resources reserved in the Try phase. The exception handling scheme in the Cancel phase is basically the same as the exception handling scheme in the Confirm phase, and the idempotent design is required.

Taking the above transfer as an example, the amount is usually frozen in Try but not deducted. The amount is deducted in Confirm and the amount is unfrozen in Cancel. The sequence diagram of a successfully completed TCC transaction is as follows:

image.png

The Confirm/Cancel phase of TCC is not allowed to return to failure in business logic. If the success cannot be returned due to network or other temporary failures, TM will continue to retry until Confirm/Cancel returns success.

TCC features are as follows:

  • High concurrency, no long-term resource lock.
  • The amount of development is large, and the Try/Confirm/Cancel interface needs to be provided.
  • The consistency is good, and there will be no transfer failure after SAGA has deducted the money
  • TCC is suitable for order-type businesses and businesses that have constraints on the intermediate state

If readers want to further study TCC, please refer to DTM

Local message table

The local message table was originally an article published by eBay architect Dan Pritchett to ACM in 2008. The core of the design is to asynchronously ensure the execution of tasks that require distributed processing through messages.

The general process is as follows:

image.png
Writing local messages and business operations are placed in one transaction, ensuring the atomicity of business and sending messages. Either they all succeed or all fail.

Fault tolerance mechanism:

  • When the balance deduction transaction fails, the transaction is rolled back directly without further steps
  • The round sequence production message fails, and the balance increase transaction fails will be retried

Features of local message table:

  • Long transaction only needs to be split into multiple tasks, easy to use
  • Producers need to create additional message tables
  • Every local message table needs to be polled
  • If the consumer’s logic cannot succeed through retrying, then more mechanisms are needed to roll back the operation

Suitable for businesses that can be executed asynchronously, and subsequent operations do not need to be rolled back

Transaction message

In the above-mentioned local message table solution, the producer needs to create an additional message table, and also needs to poll the local message table, and the business burden is heavy. Alibaba's open source RocketMQ version after 4.3 officially supports transaction messages, which essentially put the local message table on RocketMQ to solve the atomic problem of message sending on the production side and local transaction execution.

Transaction message sending and submission:

  • Send message (half message)
  • The server stores the message and responds to the writing result of the message
  • Perform local transactions based on the sending results (if the write fails, the half message is not visible to the business at this time, and the local logic is not executed)
  • Execute Commit or Rollback according to the local transaction status (Commit operation publishes the message, and the message is visible to consumers)

The flow chart of normal sending is as follows:

image.png

Compensation process:

For transaction messages without Commit/Rollback (messages in the pending state), initiate a "check back" from the server
Producer receives the check-back message and returns the status of the local transaction corresponding to the message, which is Commit or Rollback
The transaction message scheme is very similar to the local message table mechanism, the main difference is that the original related local table operation is replaced with a reverse lookup interface

The characteristics of transaction messages are as follows:

  • The long transaction only needs to be divided into multiple tasks, and a counter-check interface is provided, which is easy to use
  • If the consumer’s logic cannot succeed through retrying, then more mechanisms are needed to roll back the operation

Suitable for businesses that can be executed asynchronously, and subsequent operations do not need to be rolled back

If readers want to further study the transaction message, please refer to DTM , or Rocketmq

Best effort notice

The initiating party makes its best efforts to notify the recipient of the business processing result through a certain mechanism. Specifically:

There is a certain message repeat notification mechanism. Because the receiving notification party may not receive the notification, there must be a mechanism to repeatedly notify the message at this time.
Message proofreading mechanism. If the recipient is not notified by its best efforts, or the recipient wants to consume the message again after consuming the message, the recipient can actively query the informer for the message information to meet the demand.
The local message table and transaction messages described above are all reliable messages. How are they different from the best-effort notification described here?

Reliable message consistency. The initiating notification party needs to ensure that the message is sent out and the message is sent to the receiving notification party. The reliability of the message is guaranteed by the initiating notification party.

Best effort notification, the initiating notification party makes its best efforts to notify the service processing result as the receiving notification party, but the message may not be received. At this time, the receiving notification party needs to actively call the initiating notification party's interface to query the service processing results and the reliability of the notification The key is to receive the notification party.

In terms of solutions, the best-effort notification needs:

  • Provide an interface, so that the notification receiver can query the business processing results through the interface
  • Message queue ACK mechanism, the message queue will gradually increase the notification interval according to the interval of 1min, 5min, 10min, 30min, 1h, 2h, 5h, 10h until it reaches the upper limit of the time window required for notification. Do not notify afterwards

The best-effort notification is applicable to business notification types. For example, the result of a WeChat transaction is to notify each merchant through the best-effort notification method, which has both callback notifications and transaction query interfaces.

AT transaction mode

This is a transaction mode in the Ali open source project seata , which is also called FMT in Ant Financial. The advantage is that the transaction mode is similar to the XA mode. The business does not need to write various compensation operations. The rollback is automatically completed by the framework. The disadvantage is also similar to XA. There is a long-term lock and does not meet high concurrency scenarios. From a performance point of view, AT mode will be higher than XA, but it also brings new problems such as dirty rollback. Interested students can refer to seata-AT

Exception handling

In all aspects of distributed transactions, network and business failures may occur. These problems require the business side of distributed transactions to achieve the three characteristics of air defense rollback, idempotence, and anti-hanging.

abnormal situation

The following describes these abnormal situations with TCC transactions:

empty rollback:

In the case that the Try method of the TCC resource is not called, the second-stage Cancel method is called. The Cancel method needs to recognize that this is an empty rollback, and then directly returns success.

The reason is that when a branch transaction is down or the network is abnormal, the branch transaction call is recorded as a failure. At this time, the Try phase is not executed. When the failure is restored, the distributed transaction is rolled back and the two-phase Cancel is called. Method to form an empty rollback.

idempotent :

Since any request may have network abnormalities and repeated requests, all distributed transaction branches need to be idempotent

suspension:

Suspension means that for a distributed transaction, the second-stage Cancel interface is executed before the Try interface.

The reason is that when the RPC calls the branch transaction try, the branch transaction is registered first, and then the RPC call is executed. If the network called by the RPC is congested at this time, after the RPC times out, TM will notify the RM to roll back the distributed transaction, possibly rollback After completion, the RPC request of Try arrives at the participant for actual execution.

Let’s look at a sequence diagram of network abnormalities to better understand the above-mentioned problems.
image.png

  • When business processing request 4, Cancel is executed before Try, and empty rollback needs to be processed
  • When business processing request 6, Cancel is executed repeatedly and needs to be idempotent
  • When the business is processing request 8, Try is executed after Cancel, and the suspension needs to be processed

In the face of the above-mentioned complex network abnormalities, the solutions proposed by each company currently see that the business side uses a unique key to query whether the associated operation has been completed, and if it has been completed, it will directly return to success. The related judgment logic is more complicated, error-prone, and heavy business burden.

Sub-transaction barrier

In the project https://github.com/yedf/dtm , a seed transaction barrier technology appeared. Using this technology, this effect can be achieved. See the schematic diagram:
image.png

All these requests, after reaching the sub-transaction barrier: abnormal requests will be filtered; normal requests will pass the barrier. After the developer uses the sub-transaction barrier, the various exceptions mentioned above are all properly handled, and the business developer only needs to pay attention to the actual business logic, and the burden is greatly reduced.
Sub-transaction barrier provides the method ThroughBarrierCall, the prototype of the method is:

func ThroughBarrierCall(db *sql.DB, transInfo *TransInfo, busiCall BusiFunc)

Business developers write their own related logic in busiCall and call this function. ThroughBarrierCall guarantees that busiCall will not be called in scenarios such as empty rollback, suspension, etc.; when the business is called repeatedly, there is idempotent control, and it is guaranteed to be submitted only once.

Sub-transaction barrier will manage TCC, SAGA, transaction messages, etc., and can also be extended to other areas

Principle of Subtransaction Barrier

The principle of sub-transaction barrier technology is to establish a branch transaction status table sub_trans_barrier in the local database, the unique key is global transaction id-sub-transaction id-sub-transaction branch name (try|confirm|cancel)

  • Open transaction
  • If it is a Try branch, then insert ignore is inserted into gid-branchid-try, if it is successfully inserted, then the logic in the barrier is called
  • If it is a Confirm branch, then insert ignore inserts gid-branchid-confirm, if it is successfully inserted, call the logic in the barrier
  • If it is a Cancel branch, insert ignore into gid-branchid-try, and then insert gid-branchid-cancel, if the try is not inserted and cancel is inserted successfully, the logic in the barrier is called
  • The logic inside the barrier returns success, commits the transaction, and returns success
  • The logic within the barrier returns an error, rolls back the transaction, and returns an error

Under this mechanism, problems related to network abnormalities are solved

  • Null compensation control--If Try is not executed and Cancel is executed directly, then Cancel will be inserted into gid-branchid-try successfully, and the logic inside the barrier will not be followed, ensuring empty compensation control
  • Idempotent control-the unique key cannot be inserted repeatedly in any branch, which ensures that it will not be executed repeatedly
  • Anti-suspension control-Try to execute after Cancel, then the inserted gid-branchid-try is not successful, it will not be executed, ensuring anti-suspension control

For SAGA, transaction messages, etc., a similar mechanism is also used.

Sub-transaction barrier summary

The sub-transaction barrier technology was https://github.com/yedf/dtm . Its significance lies in the design of simple and easy-to-implement algorithms and provides a simple and easy-to-use interface. In the first creation, its significance lies in the simplicity of design. The implemented algorithm provides a simple and easy-to-use interface. With the help of these two items, developers are completely liberated from the handling of network exceptions.

This technology currently requires yedf/dtm transaction manager, and the SDK has been provided to developers of Go and Python languages. SDKs in other languages are being planned. For other distributed transaction frameworks, as long as appropriate distributed transaction information is provided, the technology can be quickly implemented in accordance with the above principles.

Summarize

This article introduces some basic theories of distributed transactions, and explains the commonly used distributed transaction schemes. In the second half of the article, the reasons, classifications and elegant solutions of transaction exceptions are also given.

yedf/dtm supports TCC, XA, SAGA, transaction messages, best-effort notifications (implemented using transaction messages), provides HTTP and gRPC protocol support, and is very easy to access.

yedf/dtm has supported the clients of Python, Java, PHP, C#, Node and other languages, please refer to: each language.

Welcome everyone to visit the https://github.com/yedf/dtm project, give star support!

阅读 9.8k

480 声望
2.5k 粉丝
0 条评论
你知道吗?

480 声望
2.5k 粉丝
宣传栏