Pay attention to this detail when using Go map and avoid relying on it!

Hello everyone, I am fried fish.

Recently, a classmate asked me about this Nikkei topic. When I wanted to transfer his article, I found that my official account hadn't been posted, so today I will nag a little bit so that everyone can avoid this "pit".

Some friends did not pay attention to the Go map output and traversal order. thought it was stable and orderly, and would directly rely on this result set sequence in the business program. As a result, they planted a big somersault , which ate online bugs.

Some friends know that it is disorderly, but don't know why, and some understand it incorrectly?

Today through this article, we will unveil for range map , and see what its internal implementation is and what is the order?

Begin the road to sucking fish.

Preface

Examples are as follows:

func main() {
    m := make(map[int32]string)
    m[0] = "EDDYCJY1"
    m[1] = "EDDYCJY2"
    m[2] = "EDDYCJY3"
    m[3] = "EDDYCJY4"
    m[4] = "EDDYCJY5"

    for k, v := range m {
        log.Printf("k: %v, v: %v", k, v)
    }
}

Suppose you run this code, what is the output? Is it output in order or out of order?

k: 3, v: EDDYCJY4
k: 4, v: EDDYCJY5
k: 0, v: EDDYCJY1
k: 1, v: EDDYCJY2
k: 2, v: EDDYCJY3

In terms of output results, it is output in a non-fixed order, that is, it is different every time. But why is this?

First of all, suggests that you first think about the reason . Secondly, I heard some sayings during the interview. Some people say that because it is a hash, there is no (disorder) order, etc. I was kind of? ? ?

This is also the reason for the appearance of this article. I hope everyone can discuss it together and clarify this problem:)

Take a look at the compilation

    ...
    0x009b 00155 (main.go:11)    LEAQ    type.map[int32]string(SB), AX
    0x00a2 00162 (main.go:11)    PCDATA    $2, $0
    0x00a2 00162 (main.go:11)    MOVQ    AX, (SP)
    0x00a6 00166 (main.go:11)    PCDATA    $2, $2
    0x00a6 00166 (main.go:11)    LEAQ    ""..autotmp_3+24(SP), AX
    0x00ab 00171 (main.go:11)    PCDATA    $2, $0
    0x00ab 00171 (main.go:11)    MOVQ    AX, 8(SP)
    0x00b0 00176 (main.go:11)    PCDATA    $2, $2
    0x00b0 00176 (main.go:11)    LEAQ    ""..autotmp_2+72(SP), AX
    0x00b5 00181 (main.go:11)    PCDATA    $2, $0
    0x00b5 00181 (main.go:11)    MOVQ    AX, 16(SP)
    0x00ba 00186 (main.go:11)    CALL    runtime.mapiterinit(SB)
    0x00bf 00191 (main.go:11)    JMP    207
    0x00c1 00193 (main.go:11)    PCDATA    $2, $2
    0x00c1 00193 (main.go:11)    LEAQ    ""..autotmp_2+72(SP), AX
    0x00c6 00198 (main.go:11)    PCDATA    $2, $0
    0x00c6 00198 (main.go:11)    MOVQ    AX, (SP)
    0x00ca 00202 (main.go:11)    CALL    runtime.mapiternext(SB)
    0x00cf 00207 (main.go:11)    CMPQ    ""..autotmp_2+72(SP), $0
    0x00d5 00213 (main.go:11)    JNE    193
    ...

Let's take a rough look at the overall process. The focus is on two runtime methods that deal with Go map loop iteration, as follows:

• runtime.mapiterinit
• runtime.mapiternext

But you may be thinking, for range are these two functions appearing and what's going on when you use 0613dcfb2256be for loop iteration?

Take a look after conversion

var hiter map_iteration_struct
for mapiterinit(type, range, &hiter); hiter.key != nil; mapiternext(&hiter) {
    index_temp = *hiter.key
    value_temp = *hiter.val
    index = index_temp
    value = value_temp
    original body
}

In fact, the compiler has different implementations for the loop iteration of slice and map. It is not that for , and some additional actions have been taken to deal with it. The above code is for range map after the compiler is expanded

Look at the source code

runtime.mapiterinit

func mapiterinit(t *maptype, h *hmap, it *hiter) {
    ...
    it.t = t
    it.h = h
    it.B = h.B
    it.buckets = h.buckets
    if t.bucket.kind&kindNoPointers != 0 {
        h.createOverflow()
        it.overflow = h.extra.overflow
        it.oldoverflow = h.extra.oldoverflow
    }

    r := uintptr(fastrand())
    if h.B > 31-bucketCntBits {
        r += uintptr(fastrand()) << 31
    }
    it.startBucket = r & bucketMask(h.B)
    it.offset = uint8(r >> h.B & (bucketCnt - 1))
    it.bucket = it.startBucket
    ...

    mapiternext(it)
}

By mapiterinit reading method, its main purpose is that traversal iteration Map initialization operation . There are three formal parameters, which are used to read the type information of the current hash table, the storage information of the current hash table, and the data of the current traversal iteration

why

We pay attention to fastrand in the source code. Is this method name familiar? Yes, it is a method of generating random numbers. Look at the context again:

...
// decide where to start
r := uintptr(fastrand())
if h.B > 31-bucketCntBits {
    r += uintptr(fastrand()) << 31
}
it.startBucket = r & bucketMask(h.B)
it.offset = uint8(r >> h.B & (bucketCnt - 1))

// iterator state
it.bucket = it.startBucket

In this code, it generates a random number. Used to decide where to start the loop iteration. To be more specific, select a bucket position as the starting point for traversal iteration based on random numbers

So every time you restart for range map , the results you see are different. That's because its starting position is not fixed at all!

runtime.mapiternext

func mapiternext(it *hiter) {
    ...
    for ; i < bucketCnt; i++ {
        ...
        k := add(unsafe.Pointer(b), dataOffset+uintptr(offi)*uintptr(t.keysize))
        v := add(unsafe.Pointer(b), dataOffset+bucketCnt*uintptr(t.keysize)+uintptr(offi)*uintptr(t.valuesize))
        ...
        if (b.tophash[offi] != evacuatedX && b.tophash[offi] != evacuatedY) ||
            !(t.reflexivekey || alg.equal(k, k)) {
            ...
            it.key = k
            it.value = v
        } else {
            rk, rv := mapaccessK(t, h, k)
            if rk == nil {
                continue // key has been deleted
            }
            it.key = rk
            it.value = rv
        }
        it.bucket = bucket
        if it.bptr != b {
            it.bptr = b
        }
        it.i = i + 1
        it.checkBucket = checkBucket
        return
    }
    b = b.overflow(t)
    i = 0
    goto next
}

In the previous section, we have selected the location of the starting bucket. The next step is by mapiternext for specific operation loops through . The method mainly involves the following:

• Start traversing from the selected bucket, looking for the next element in the bucket for processing
• If the bucket has been traversed, traverse the overflow bucket overflow buckets

Through the reading of this method, we can know the traversal rule for buckets and some processing for expansion (this is not the focus of this article. So there is no specific expansion)

Summarize

At the beginning of this article, let's first put forward the core discussion point: "Why is the Go map traversal output not in a fixed order?".

After this analysis, the reason is also very simple and clear. That is, when for range map started to process the loop logic, it did random seeding...

You want to ask why do you want to do this?

Of course it is the official intention, because in the early days of Go (1.0), although it was a stable iteration, in terms of results, there is actually no guarantee that each Go version iterative traversal rule is the same. This will lead to portability issues.

Therefore, change it. Please don't rely on...

If you have any questions, welcome feedback and communication in the comment area. The is mutual achievement of . Your likes is fried fish . Thank you for your support.

The article is continuously updated, you can read it on WeChat search [ 000 ] I have prepared the first-line big factory interview algorithm problem solution and information.

This article GitHub github.com/eddycjy/blog has been included, welcome Star to remind you.

refer to

• Go maps in action