Redis Redisson distributed lock application and source code

1 Introduction

I wrote an article "Realization of Redis Distributed Locks" , which mainly introduced the original realization of Redis distributed locks. The core is based on setnx to lock, and the use of lua ensure the atomicity of transactions. But after all, it is relatively primitive, and different code implementations need to be implemented according to different application scenarios, and it is easy to be inconsiderate. Redisson framework was mentioned in the article. It just happened that I used it more recently in my work, so I will focus on it this time.

Redisson is a Java development framework based on Redis. The bottom layer is based on the Netty framework. It provides users with a series of commonly used tools with distributed characteristics. Redisson has very rich functions. For details, please refer to github Chinese wiki , but this article only introduces the Redisson distributed lock function.

2. Ordinary reentrant lock

2.1. Examples of use

It is very easy to lock the SpringBoot project through Redisson, and there is no need to write a lot of code as in the previous article, and the framework shields many details. The following example:

Config config = new Config();
config.useSingleServer().setAddress("redis://ip:port").setPassword("password").setDatabase(0);
RedissonClient redissonClient = Redisson.create(config);

RLock lock = redissonClient.getLock("LOCK_KEY");
long waitTime=500L;
long leaseTime=15000L;
boolean isLock;
try {
    isLock = lock.tryLock(waitTime, leaseTime, TimeUnit.MILLISECONDS);
    if (isLock) {
        // do something ...
    }
} catch (InterruptedException e) {
    Thread.currentThread().interrupt();
} finally {
    lock.unlock();
}

Note that there is no limit to Config in the code. In the example, it is a Redis single node connection, but it can actually be sentinel mode, cluster mode, master-slave mode, etc.

2.2. Source code explanation

In the previous example, the RLock interface is used for locking. Here is the source code:
org.redisson.api.RLock.java

public interface RLock extends Lock, RLockAsync {
    String getName();

    void lockInterruptibly(long var1, TimeUnit var3) throws InterruptedException;

    boolean tryLock(long var1, long var3, TimeUnit var5) throws InterruptedException;

    void lock(long var1, TimeUnit var3);

    boolean forceUnlock();

    boolean isLocked();

    boolean isHeldByThread(long var1);

    boolean isHeldByCurrentThread();

    int getHoldCount();

    long remainTimeToLive();
}

For reentrant locks, the implementation method corresponding to the interface is in the org.redisson.RedissonLock class, and the source code is not posted. It can be seen that when it falls into Redis, the actual "locking" and "unlocking" process is also a Lua script.

2.2.1. Locking

1、lua

if (redis.call('exists', KEYS[1]) == 0) then
    redis.call('hset', KEYS[1], ARGV[2], 1);
    redis.call('pexpire', KEYS[1], ARGV[1]);
    return nil;
end
if (redis.call('hexists', KEYS[1], ARGV[2]) == 1) then
    redis.call('hincrby', KEYS[1], ARGV[2], 1);
    redis.call('pexpire', KEYS[1], ARGV[1]);
    return nil;
end
return redis.call('pttl', KEYS[1]);

Parameter explanation:

• KEYS[1] : The name of the locked resource, and the framework is named NAME.
• ARGV[1] : Expiration time.
• ARGV[2] : The current program identifier (UUID + current threadId), the source code is named LOCK_NAME.

The logic of locking is to store a Hash type value in redis. Once the resource is locked, the value is set to 1 for the first time, and only the current program can be locked repeatedly, that is, the Value is increased by 1.

2, lock and block

The java method of locking is:

    public boolean tryLock(long waitTime, long leaseTime, TimeUnit unit) throws InterruptedException {
        long time = unit.toMillis(waitTime);
        long current = System.currentTimeMillis();
        long threadId = Thread.currentThread().getId();
        Long ttl = this.tryAcquire(leaseTime, unit, threadId);
        if (ttl == null) {
            return true;
        } else {
            time -= System.currentTimeMillis() - current;
            if (time <= 0L) {
                this.acquireFailed(threadId);
                return false;
            } else {
                current = System.currentTimeMillis();
                RFuture<RedissonLockEntry> subscribeFuture = this.subscribe(threadId);
                if (!subscribeFuture.await(time, TimeUnit.MILLISECONDS)) {
                    if (!subscribeFuture.cancel(false)) {
                        subscribeFuture.onComplete((res, e) -> {
                            if (e == null) {
                                this.unsubscribe(subscribeFuture, threadId);
                            }

                        });
                    }

                    this.acquireFailed(threadId);
                    return false;
                } else {
                    boolean var16;
                    try {
                        time -= System.currentTimeMillis() - current;
                        if (time <= 0L) {
                            this.acquireFailed(threadId);
                            boolean var20 = false;
                            return var20;
                        }

                        do {
                            long currentTime = System.currentTimeMillis();
                            ttl = this.tryAcquire(leaseTime, unit, threadId);
                            if (ttl == null) {
                                var16 = true;
                                return var16;
                            }

                            time -= System.currentTimeMillis() - currentTime;
                            if (time <= 0L) {
                                this.acquireFailed(threadId);
                                var16 = false;
                                return var16;
                            }

                            currentTime = System.currentTimeMillis();
                            if (ttl >= 0L && ttl < time) {
                                this.getEntry(threadId).getLatch().tryAcquire(ttl, TimeUnit.MILLISECONDS);
                            } else {
                                this.getEntry(threadId).getLatch().tryAcquire(time, TimeUnit.MILLISECONDS);
                            }

                            time -= System.currentTimeMillis() - currentTime;
                        } while(time > 0L);

                        this.acquireFailed(threadId);
                        var16 = false;
                    } finally {
                        this.unsubscribe(subscribeFuture, threadId);
                    }

                    return var16;
                }
            }
        }
    }

Combining lua and java source code, we can know that the locking thread will be blocked only when the following conditions are met at the same time:

• EXISTS NAME = 1, that is, the lock resource exists;
• HEXISTS LOCK_NAME = 0, that is, the current thread is not locked;
• waitTime> 0L, that is, there is no blocking waiting time.

3. Comparison of various locking methods

• declares whether the method has InterruptedException : InterruptedException is thrown when the method is declared, which means that while the current method is waiting, it supports other threads to interrupt the execution of the current thread method by calling the interrupt method.
• or without leaseTime : Setting leaseTime means setting the expiration time of the lock. If there is no leaseTime or leaseTime=-1L, the lock will be automatically renewed through watchDog.
• Whether there is waitTime : Set the maximum blocking time when the lock is blocked.
• lock and tryLock : often lock is blocked when the lock cannot be obtained, and tryLock will return immediately when the lock cannot be obtained, but if the waitTime parameter is included, it will be calculated separately.

4. Watchdog mechanism

If the node that gets the distributed lock goes down and the lock happens to be in the locked state, the locked state will appear. In order to avoid this, the lock will set an expiration time. leaseTime that was passed in when locking. In some application scenarios, if we have not completed the business within the specified time, we need to "renew" the lock at this time. If the whole process is fully controllable, you can manually renew the lock in the program. But if you want to automatically renew, you can use Redisson's Wath Dog (watchdog) mechanism.

Redisson provides a watchdog that monitors the lock. Its function is to continuously extend the validity period of the lock before the Redisson instance is closed. That is to say, if a thread that has obtained the lock has not completed the logic, then the watchdog will The help thread continues to extend the lock timeout time, and the lock will not be released due to the timeout. By default, the watchdog renewal time is 30s, and it can also be specified by modifying Config.lockWatchdogTimeout.

Here is locked source code, note that when you call locking method, if you want to use the watchdog, then pass leaseTime value -1L . If you set a valid value for leaseTime, the watchdog will not take effect, and the lock will not automatically renew, but will be automatically unlocked after the time you specify.

    private <T> RFuture<Long> tryAcquireAsync(long leaseTime, TimeUnit unit, long threadId) {
        if (leaseTime != -1L) {
            return this.tryLockInnerAsync(leaseTime, unit, threadId, RedisCommands.EVAL_LONG);
        } else {
            RFuture<Long> ttlRemainingFuture = this.tryLockInnerAsync(this.commandExecutor.getConnectionManager().getCfg().getLockWatchdogTimeout(), TimeUnit.MILLISECONDS, threadId, RedisCommands.EVAL_LONG);
            ttlRemainingFuture.onComplete((ttlRemaining, e) -> {
                if (e == null) {
                    if (ttlRemaining == null) {
                        this.scheduleExpirationRenewal(threadId);
                    }
                }
            });
            return ttlRemainingFuture;
        }
    }

2.2.2. Unlock

1、lua（unlock）

if (redis.call('hexists', KEYS[1], ARGV[3]) == 0) then
    return nil;
end
local counter = redis.call('hincrby', KEYS[1], ARGV[3], -1);
if (counter > 0) then
    redis.call('pexpire', KEYS[1], ARGV[2]);
    return 0;
else
    redis.call('del', KEYS[1]);
    redis.call('publish', KEYS[2], ARGV[1]);
    return 1;
end
return nil;

Parameter explanation:

• KEYS[1] : The name of the resource being locked.
• KEYS[2] : Broadcast the channel name when unlocking.
• ARGV[1] : Broadcast channel message when unlocking (value is 0L).
• ARGV[2] : Expiration time.
• ARGV[3] : The current program ID (UUID + current threadId).

The logic of unlocking is to first determine whether the name of the locked resource exists, and if it exists, subtract 1 from the Value. When the Value is 0, delete the Key and broadcast a message to the designated channel.

The design of the broadcast channel is very bright. When multiple threads compete for locks at the same time, the threads that have not grabbed the lock do not need invalid polling, and only need to subscribe to one channel. When the lock is released, a message is broadcast in the channel to notify those threads waiting to acquire the lock that the lock is now available, and those threads will compete for the lock again to avoid wasting performance resources.

2、lua（forkUnlock）

if (redis.call('del', KEYS[1]) == 1) then
    redis.call('publish', KEYS[2], ARGV[1]);
    return 1
else
    return 0
end

2.2.3. Interpretation of RLock interface

org.redisson.api.RLock interface inherits the java.util.concurrent.locks.Lock interface. I put them together and made method annotations based on my own understanding:

public interface RLock {
    /**
     * 获取锁的KEY （即:RedissonLock.getLock(String name);中的参数 name）
     */
    String getName();

    /**
     * 阻塞获取锁，如果获取不到锁，会一直阻塞等待
     * 加锁成功后，设置锁过期时间为leaseTime
     * 阻塞时，如果被其他线程执行interrupt方法，会终止阻塞并抛出InterruptedException返回
     *
     * @param leaseTime 锁租约（过期）时间
     * @param unit leaseTime对应的时间单位
     * @throws InterruptedException 线程中断异常
     */
    void lockInterruptibly(long leaseTime, TimeUnit unit) throws InterruptedException;

    /**
     * 尝试获取锁，如果获取不到锁，会最大阻塞等待waitTime，超时后会正常返回
     * 加锁成功后，设置锁过期时间为leaseTime
     * 阻塞时，如果被其他线程执行interrupt方法，会终止阻塞并抛出InterruptedException返回
     *
     * @param waitTime 阻塞最大等待时间
     * @param leaseTime 锁租约（过期）时间
     * @param unit waitTime、leaseTime共同对应的时间单位
     * @throws InterruptedException 线程中断异常
     */
    boolean tryLock(long waitTime, long leaseTime, TimeUnit unit) throws InterruptedException;

    /**
     * 阻塞获取锁，如果获取不到锁，会一直阻塞等待
     * 加锁成功后，设置锁过期时间为leaseTime
     * 阻塞时，不受其他线程的interrupt方法干扰，获取锁之前不会终止线程
     *
     * @param leaseTime 锁租约（过期）时间
     * @param unit leaseTime对应的时间单位
     */
    void lock(long leaseTime, TimeUnit unit);

    /**
     * 强制解锁，立即返回
     * 对于同一个锁NAME，会解锁所有线程资源加的锁；如果可重入锁多次加锁，也会一次性解除所有锁。
     * Redis：1、DEL NAME && 2、PUBLISH channelName msg （向通道广播解锁消息）
     */
    boolean forceUnlock();

    /**
     * 判断资源NAME是否有被锁（但可能被多个线程、加多次锁）
     * Redis：EXISTS NAME
     *
     * @return
     */
    boolean isLocked();

    /**
     * 判断指定线程是否获得锁（通过 HEXISTS 命令）
     * Redis：HEXISTS NAME LOCKNAME（id+threadId）
     *
     * @param threadId 线程ID
     */
    boolean isHeldByThread(long threadId);

    /**
     * 判断当前线程是否获得锁
     * Redis：HEXISTS NAME LOCKNAME（id+currentThreadId）
     *
     */
    boolean isHeldByCurrentThread();

    /**
     * 获取当前线程，获得锁的数量（通过 HGET 命令）
     * Redis：HGET NAME LOCKNAME（id+currentThreadId）
     *
     */
    int getHoldCount();

    /**
     * 获取锁资源剩余生存时间
     * Redis: PTTL NAME
     *
     * @return 完全redis pttl命令结果：返回单位为毫秒；当key不存在时，返回 -2；当key存在但没有设置剩余生存时间时，返回 -1。
     */
    long remainTimeToLive();

    // java.util.concurrent.locks.Lock.java

    /**
     * 阻塞获取锁，如果获取不到锁，会一直阻塞等待
     * 等同于 leaseTime=-1L，会触发 Watch Dog 机制，当业务未执行完成时，会自动给锁续期，默认每次续期30s
     * 阻塞时，不受其他线程的interrupt方法干扰，获取锁之前不会终止线程
     */
    void lock();

    /**
     * 阻塞获取锁，如果获取不到锁，会一直阻塞等待
     * 等同于 leaseTime=-1L，会触发 Watch Dog 机制，当业务未执行完成时，会自动给锁续期，默认每次续期30s
     * 阻塞时，如果被其他线程执行interrupt方法，会终止阻塞并抛出InterruptedException返回
     *
     * @throws InterruptedException 线程中断异常
     */
    void lockInterruptibly() throws InterruptedException;

    /**
     * 尝试获取锁，如果获取不到锁，会立即返回
     * 等同于 leaseTime=-1L，会触发 Watch Dog 机制，当业务未执行完成时，会自动给锁续期，默认每次续期30s
     * 阻塞时，不受其他线程的interrupt方法干扰，获取锁之前不会终止线程
     */
    boolean tryLock();

    /**
     * 尝试获取锁，如果获取不到锁，会最大阻塞等待waitTime，超时后会正常返回
     * 等同于 leaseTime=-1L，会触发 Watch Dog 机制，当业务未执行完成时，会自动给锁续期，默认每次续期30s
     * 阻塞时，如果被其他线程执行interrupt方法，会终止阻塞并抛出InterruptedException返回
     *
     * @param waitTime 阻塞最大等待时间
     * @param unit waitTime对应当时间单位
     * @throws InterruptedException
     */
    boolean tryLock(long waitTime, TimeUnit unit) throws InterruptedException;

    /**
     * 解锁
     * 对于某个锁KEY，只会解除当前线程加的锁；如果可重入锁多次加锁，只会解除单次锁
     */
    void unlock();

    /**
     * Redisson 不支持该方法的实现
     * @return throw new UnsupportedOperationException();
     */
    Condition newCondition();
}

3. RedLock

3.1. Concept description

Also in the previous article, RedLock was also mentioned, which is a literal translation of "红锁" in Chinese. In fact, that article has already been introduced, so I will introduce it here. There are loopholes when using redis to implement distributed locks. The specific scenarios:

Client A applies for a lock on the Redis master node. But the master crashed before synchronizing the stored key to the slave, and then the slave was promoted to master. And client B applies for a lock on the resource that client A already holds. Then what? Then what? There is a problem, both client A and B can apply for the same lock.

RedLock is an algorithm officially proposed by Redis. The specific process includes:

1. Get the current time.
2. N nodes acquire the lock in turn, and set the response timeout time to prevent a single node from acquiring the lock for too long.
3. Lock validity time = lock expiration time-the time it takes to acquire the lock, if the number of nodes successfully acquired in step 2 is greater than
   N/2+1, and the lock effective time is greater than 0, then the lock is successfully obtained.
4. If obtaining the lock fails, release the lock to all nodes.

To put it simply, during the lock expiration time, if more than half of the nodes successfully acquire the lock, it means that the lock is successfully acquired. This is a bit like ZooKeeper's election mechanism. Here to talk about the implementation method in Redisson.

3.2. Examples of use

Redisson's code for using RedLock is extremely simple. It just combines several locks into a "big lock", and then uses the "big lock" to lock/unlock normally.

Config config1 = new Config();
config1.useSingleServer().setAddress("redis://ip1:port1")
        .setPassword("password1").setDatabase(0);
RedissonClient redissonClient1 = Redisson.create(config1);

Config config2 = new Config();
config2.useSingleServer().setAddress("redis://ip2:port2")
        .setPassword("password2").setDatabase(0);
RedissonClient redissonClient2 = Redisson.create(config2);

Config config3 = new Config();
config3.useSingleServer().setAddress("redis://ip3:port3")
        .setPassword("password3").setDatabase(0);
RedissonClient redissonClient3 = Redisson.create(config3);

String lockKey = "REDLOCK_KEY";
RLock lock1 = redissonClient1.getLock(lockKey);
RLock lock2 = redissonClient2.getLock(lockKey);
RLock lock3 = redissonClient3.getLock(lockKey);

RedissonRedLock redLock = new RedissonRedLock(lock1, lock2, lock3);
boolean isLock;
long waitTime=500L;
long leaseTime=15000L;
try {
    isLock = redLock.tryLock(waitTime, leaseTime, TimeUnit.MILLISECONDS);
    if (isLock) {
        // do something ...
    }
} catch (Exception e) {
    ... ...
} finally {
    redLock.unlock();
}

Note that there is no limit to Config in the code. In the example, it is a Redis single node connection, but it can actually be sentinel mode, cluster mode, master-slave mode, etc.

3.3. Source code explanation

Before talking about Redisson's RedLock (red lock), MultiLock talk about 0617913919895b (interlocking). The reason is to look RedissonRedLock source code of 0617913919895d, which completely inherits all the functions of RedissonMultiLock

RedissonRedLock.java

public class RedissonRedLock extends RedissonMultiLock {
    public RedissonRedLock(RLock... locks) {
        super(locks);
    }

    protected int failedLocksLimit() {
        return this.locks.size() - this.minLocksAmount(this.locks);
    }

    protected int minLocksAmount(List<RLock> locks) {
        return locks.size() / 2 + 1;
    }

    protected long calcLockWaitTime(long remainTime) {
        return Math.max(remainTime / (long)this.locks.size(), 1L);
    }

    public void unlock() {
        this.unlockInner(this.locks);
    }
}

RedissonMultiLock.java core code

    // 加锁
    public boolean tryLock(long waitTime, long leaseTime, TimeUnit unit) throws InterruptedException {
        long newLeaseTime = -1L;
        if (leaseTime != -1L) {
            if (waitTime == -1L) {
                newLeaseTime = unit.toMillis(leaseTime);
            } else {
                newLeaseTime = unit.toMillis(waitTime) * 2L;
            }
        }

        long time = System.currentTimeMillis();
        long remainTime = -1L;
        if (waitTime != -1L) {
            remainTime = unit.toMillis(waitTime);
        }

        long lockWaitTime = this.calcLockWaitTime(remainTime);
        int failedLocksLimit = this.failedLocksLimit();
        List<RLock> acquiredLocks = new ArrayList(this.locks.size());
        ListIterator iterator = this.locks.listIterator();

        while(iterator.hasNext()) {
            RLock lock = (RLock)iterator.next();

            boolean lockAcquired;
            try {
                if (waitTime == -1L && leaseTime == -1L) {
                    lockAcquired = lock.tryLock();
                } else {
                    long awaitTime = Math.min(lockWaitTime, remainTime);
                    lockAcquired = lock.tryLock(awaitTime, newLeaseTime, TimeUnit.MILLISECONDS);
                }
            } catch (RedisResponseTimeoutException var21) {
                this.unlockInner(Arrays.asList(lock));
                lockAcquired = false;
            } catch (Exception var22) {
                lockAcquired = false;
            }

            if (lockAcquired) {
                acquiredLocks.add(lock);
            } else {
                if (this.locks.size() - acquiredLocks.size() == this.failedLocksLimit()) {
                    break;
                }

                if (failedLocksLimit == 0) {
                    this.unlockInner(acquiredLocks);
                    if (waitTime == -1L) {
                        return false;
                    }

                    failedLocksLimit = this.failedLocksLimit();
                    acquiredLocks.clear();

                    while(iterator.hasPrevious()) {
                        iterator.previous();
                    }
                } else {
                    --failedLocksLimit;
                }
            }

            if (remainTime != -1L) {
                remainTime -= System.currentTimeMillis() - time;
                time = System.currentTimeMillis();
                if (remainTime <= 0L) {
                    this.unlockInner(acquiredLocks);
                    return false;
                }
            }
        }

        if (leaseTime != -1L) {
            List<RFuture<Boolean>> futures = new ArrayList(acquiredLocks.size());
            Iterator var24 = acquiredLocks.iterator();

            while(var24.hasNext()) {
                RLock rLock = (RLock)var24.next();
                RFuture<Boolean> future = ((RedissonLock)rLock).expireAsync(unit.toMillis(leaseTime), TimeUnit.MILLISECONDS);
                futures.add(future);
            }

            var24 = futures.iterator();

            while(var24.hasNext()) {
                RFuture<Boolean> rFuture = (RFuture)var24.next();
                rFuture.syncUninterruptibly();
            }
        }

        return true;
    }
    
    // 解锁
    public void unlock() {
        List<RFuture<Void>> futures = new ArrayList(this.locks.size());
        Iterator var2 = this.locks.iterator();

        while(var2.hasNext()) {
            RLock lock = (RLock)var2.next();
            futures.add(lock.unlockAsync());
        }

        var2 = futures.iterator();

        while(var2.hasNext()) {
            RFuture<Void> future = (RFuture)var2.next();
            future.syncUninterruptibly();
        }

    }

RedissonRedLock.java rewrite the RedissonMultiLock.java in several methods:

• failedLocksLimit: 06179139198a9c is returned in 0 , and 06179139198a9e is returned in locks.size() / 2 - 1 .
• calcLockWaitTime: 06179139198ad9 is returned in remainTime , and 06179139198adb is returned in Math.max(remainTime / (long)this.locks.size(), 1L) .

It is easy to see from the source code that the RedLock algorithm in Redisson is completely implemented based on MultiLock. Redisson supports this "combined lock" concept, putting multiple RLock locks into an ArrayList, and then starting to traverse and lock. It's just that the requirements of MultiLock are more demanding. When all RLocks in the List are locked, there can be no lock failures, that is, failedLocksLimit=0. RedLock requires relaxation, as long as more than half of the lock is successful, that is, failedLocksLimit = locks.size() / 2-1. But when unlocking, it is required to unlock all the locks in the entire ArrayList.