[Advanced Go - Concurrent Programming] RWMutex

RWMutex in the standard library is a reader/writer mutex. RWMutex can only be held by any number of readers at a time, or only by a single writer. RWMutex also has very few methods, there are 5 in total:

• Lock/Unlock: The method called when writing. If the lock is already held by the reader or writer, the Lock method will block until the lock can be acquired; Unlock is the paired method of releasing the lock.
• RLock/RUnlock: The method called during read operation. If the lock is already held by the writer, the RLock method will block until the lock can be acquired, otherwise it will return directly; while RUnlock is the method for the reader to release the lock.
• RLocker: The function of this method is to return an object of the Locker interface for read operations. Its Lock method will call RWMutex's RLock method, and its Unlock method will call RWMutex's RUnlock method.

If you encounter a scenario where reader and writer goroutines can be clearly distinguished, and there are a large number of concurrent reads, a small number of concurrent writes, and strong performance requirements, you can consider replacing Mutex with a read-write lock RWMutex.

Implementation

Read-write lock design scheme

Based on the priority of read and write operations, the design and implementation of read-write locks can be divided into three categories:

• Read-preferring: Read-first design can provide high concurrency, but may lead to write starvation in the case of intense competition.
• Write-preferring: The write-first design means that if there is already a writer waiting for a request lock, it will prevent the new reader requesting the lock from acquiring the lock, so the writer is given priority.
• Do not specify priority: This design is relatively simple, and does not distinguish between reader and writer priorities. In some scenarios, this design without specifying priority is more effective.

The RWMutex design in the Go standard library is a Write-preferring scheme, a blocking call to Lock will preclude new reader requests to the lock.

data structure

RWMutex contains a Mutex, and four auxiliary fields writerSem, readerSem, readerCount and readerWait:

type RWMutex struct {
  w           Mutex   // 互斥锁解决多个writer的竞争
  writerSem   uint32  // writer信号量
  readerSem   uint32  // reader信号量
  readerCount int32   // reader的数量
  readerWait  int32   // writer等待完成的reader的数量
}

const rwmutexMaxReaders = 1 << 30

• Field w: Mutex used for writer competition;
• Field readerCount: record the current number of readers (and whether there is a writer competing for locks);
• readerWait: record the number of readers that need to wait for read completion when the writer requests a lock;
• writerSem and readerSem: Both are semaphores designed to block.
• Constant rwmutexMaxReaders: defines the maximum number of readers.

RLock/RUnlock

Let's take a look at the reduced RLock and RUnlock methods:

func (rw *RWMutex) RLock() {
    if atomic.AddInt32(&rw.readerCount, 1) < 0 {
        // rw.readerCount 是负值的时候，意味着此时有 writer 等待请求锁，因为 writer 优先级高，所以把后来的 reader 阻塞休眠
        runtime_SemacquireMutex(&rw.readerSem, false, 0)
    }
}

func (rw *RWMutex) RUnlock() {
    if r := atomic.AddInt32(&rw.readerCount, -1); r < 0 {
        rw.rUnlockSlow(r) // 有等待的 writer
    }
}

func (rw *RWMutex) rUnlockSlow(r int32) {
    if atomic.AddInt32(&rw.readerWait, -1) == 0 {
        // 最后一个 reader 了，writer 终于有机会获得锁了
        runtime_Semrelease(&rw.writerSem, false, 1)
    }
}

Line 2 increments the reader count by 1. You may be more confused, how can readerCount be negative? In fact, this is because the field readerCount has a double meaning:

• When there is no writer competition or lock is held, the readerCount is the same as the count of readers we normally understand;
• However, if there is a writer competing for a lock or holding a lock, then readerCount not only assumes the reader count function, but also identifies whether there is currently a writer competing or holding a lock. In this case, the processing of the reader requesting the lock Go to line 4 and block waiting for the lock to be released.

When calling RUnlock, we need to decrement the reader's count by 1 (line 9), because the number of readers has decreased by one. However, the return value of AddInt32 on line 9 has another meaning. If it is a negative value, it means that there is currently a writer competing for the lock. In this case, the rUnlockSlow method will also be called to check whether the readers have released the read locks. writer.

When rUnlockSlow decrements the count of readers holding locks by 1, it checks whether all existing readers have released locks. If all locks are released, it will wake up the writer and let the writer hold the lock.

Lock

RWMutex is a multi-writer multi-reader read-write lock, so there may be multiple writers and readers at the same time. Then, in order to avoid competition between writers, RWMutex will use a Mutex to ensure mutual exclusion of writers.

Once a writer acquires the internal mutex, it reverses the readerCount field, changing it from the original positive integer readerCount(>=0) to a negative number (readerCount-rwmutexMaxReaders), leaving the field with two meanings (both saving The number of readers, and it means that there is currently a writer).

Let's take a look at the code below. Line 5 also records the number of currently active readers. The so-called active readers refer to those readers that hold the read lock and have not been released.

func (rw *RWMutex) Lock() {
    // 首先解决其他 writer 竞争问题
    rw.w.Lock()
    // 反转 readerCount，告诉 reader 有 writer 竞争锁
    r := atomic.AddInt32(&rw.readerCount, -rwmutexMaxReaders) + rwmutexMaxReaders
    // 如果当前有 reader 持有锁，那么需要等待
    if r != 0 && atomic.AddInt32(&rw.readerWait, r) != 0 {
        runtime_SemacquireMutex(&rw.writerSem, false, 0)
    }
}

If readerCount is not 0, it means that there is currently a reader holding a read lock. RWMutex needs to assign the current readerCount to the readerWait field and save it (line 7). At the same time, the writer enters the blocking wait state (line 8).

Whenever a reader releases the read lock (when the RUnlock method is called), the readerWait field is decremented by 1, and the writer will not be woken up until all active readers have released the read lock.

Unlock

When a writer releases the lock, it reverses the readerCount field again. To be sure, because the current lock is held by the writer, the readerCount field is reversed, and minus the constant rwmutexMaxReaders, it becomes a negative number. So, the inversion method here is to add the constant value rwmutexMaxReaders to it.

Since the writer is going to release the lock, it needs to wake up the new readers, so they don't need to block them anymore, just let them continue to execute.

Before RWMutex's Unlock returns, the internal mutex needs to be released. After the release is complete, other writers can continue to compete for the lock.

func (rw *RWMutex) Unlock() {
    // 告诉 reader 没有活跃的 writer 了
    r := atomic.AddInt32(&rw.readerCount, rwmutexMaxReaders)
    
    // 唤醒阻塞的 reader 们
    for i := 0; i < int(r); i++ {
        runtime_Semrelease(&rw.readerSem, false, 0)
    }
    // 释放内部的互斥锁
    rw.w.Unlock()
}