[Business learning] Brief description of ID generator

introduction

Hello everyone, long time no see, I finally picked up blogging after a year.
In our daily work, we often need to use the global unique ID as the primary key of the database, or to generate the order id, to generate the product ID, and so on. This article mainly introduces our common ID generator method: using database generation and snowflake algorithm.

Use database to generate ID

Self-incrementing ID achieves the goal

Use the characteristics of MYSQL to increase the primary key to construct the ID generator. First generate an ID generator table, insert a row into this table every time we need to generate ID, get the primary key id generated by this row of records, and get a globally unique id, which basically meets the needs.

｜ id	ctime
1	2021-09-05 12:00:00
2	2021-09-05 12:01:00

Try to advance

Then some friends will say that this method will definitely affect the performance, every time a request needs to connect to the DB to get the id, it can't be hurt. It doesn't matter, let's optimize it. We use segmented requests, that is, each request for a segment of ID (for example, 20 IDs) is cached locally, so that we only need to wait for the local ID to be used up before requesting, which increases performance. At this point, some friends have said that the performance of inserting data in batches each time is still not good, and the performance is worse when multiple business parties use it at the same time. What should I do? Don't worry, look down.
We can design the table into a structure similar to the following table.

biz_tag (line of business)	max_id	step (number of IDs applied for each time)	update_time
app	1000	1000	2021-09-05 12:00:00
pc	2000	1000	2021-09-05 12:00:00

From the above table, we can see that the approximate process every time we request an ID is as follows: get the current max_id -> apply for the id between the line of business identification (max_id, max_id + step) -> the id is used up -> Re-initiate the process. In this way, the pressure on the DB will be much less, but there will also be some problems in the production environment: for example:

- 系统性能依赖DB的更新，如果更新DB出现尖刺，服务性能将收到影响
- 强依赖DB的可用性，DB一旦出现宕机将整个不可用

For the above two problems, we have the following solutions:

1. Pre-processing, when the local queue uses 70%, apply for a new ID (the ratio is set according to business needs)
2. Cache a queue locally as a "spare tire". When the service is unavailable and the normal queue is used up, you can use the "spare tire" to work, and constantly apply for new queues.

If there is a sudden increase in traffic in our service, we can also dynamically adjust the number of IDs applied for each time, and set the algorithm of the adaptive service to adjust this step. The specific algorithm will not be repeated here.

Of course, there are many ways in the industry to use databases to generate DB, and many optimizations have been made in how to ensure availability, because this way ultimately requires a strong dependence on DB

Snowflake algorithm

It assigns a unique ID to each machine, and then realizes a globally unique ID through timestamp + ID + self-increment. The advantage of this approach is that the ID generation algorithm is completely a stateless machine, without network calls, and is efficient and reliable. This method is also a common method in our business. Let's take a look at how the 52-bit and 64-bit IDs we use are generated.

64bit

42b timestamp + 8b counter + 8b countspace + 6b serverid

• timestamp: millisecond timestamp
• counter: self-increasing counter in milliseconds, value [0, 255]
• Countspace: Identifies the counterspace, there may be different counterspaces under the same business side, this 6bit is different
• serverid: server id, unique in the world

52bit

32bit timestamp + 16bit counter + 4bit server_id

• timestamp: second-level timestamp
• counter: second increment counter
• serverid: server id, unique in the world

The generation of distributed ID can be achieved through the above two rules. Of course, there are many different rules in the industry, but they are all the same. You can handle them according to your own needs.

Follow us

Readers who are interested in this series of articles are welcome to subscribe to our official account, and pay attention to the blogger not to get lost next time~