Go daily library bitset

Introduction

We all know that computers are based on binary, and bit operations are the basic operations of computers. The advantages of bit operations are obvious, the CPU instructions are natively supported and fast. Replacing set data structures with bit sets based on bit operations can have unexpected results in limited scenarios. bitset The library implements bit set and related operations, so you can use it right away.

Install

The code in this article uses Go Modules.

Create a directory and initialize:

 $ mkdir -p bitset && cd bitset
$ go mod init github.com/darjun/go-daily-lib/bitset

Install bitset library:

 $ go get -u github.com/bits-and-blooms/bitset

use

The basic operations on bit sets are:

• Check bit (Test): Check if an index is 1. Analogy to check if an element is in a collection
• Set bit (Set): Set an index to 1. Analogy for adding elements to a collection
• Clear bit (Clear): Clear an index and set it to 0. Analogy for removing an element from a collection
• Flip bit: if an index is 1, it is set to 0, otherwise it is set to 1
• Union: Two sets of bits perform a union operation. union of analogous sets
• Intersection: Two sets of bits perform an intersection operation. The intersection of analogous sets

Bitsets are generally used in the context of fractional non-negative integers. Take the simple sign-in in the game as an example. Many games have sign-in activities, ranging from 7 days to 30 days. This is a good way to use bitsets. The value of each bit indicates whether there is a check-in on the day corresponding to its index position.

 type Player struct {
  sign *bitset.BitSet
}

func NewPlayer(sign uint) *Player {
  return &Player{
    sign: bitset.From([]uint64{uint64(sign)}),
  }
}

func (this *Player) Sign(day uint) {
  this.sign.Set(day)
}

func (this *Player) IsSigned(day uint) bool {
  return this.sign.Test(day)
}

func main() {
  player := NewPlayer(1) // 第一天签到
  for day := uint(2); day <= 7; day++ {
    if rand.Intn(100)&1 == 0 {
      player.Sign(day - 1)
    }
  }

  for day := uint(1); day <= 7; day++ {
    if player.IsSigned(day - 1) {
      fmt.Printf("day:%d signed\n", day)
    }
  }
}

bitset provides several ways to create BitSet objects.

First bitset.BitSet zero value is available, if you don't know how many elements there are at first, you can create it this way:

 var b bitset.BitSet

BitSet automatically resizes when set. If the length is known in advance, this value can be passed in when creating a BitSet, which can effectively avoid the overhead of automatic adjustment:

 b := bitset.New(100)

The bitset structure supports chained calls, and most methods return their own pointers, so they can be written like this:

 b.Set(10).Set(11).Clear(12).Flip(13);

Note that the index of the bitset is 0-based.

I remember reading a question on the Internet before:

A farmer came to the river with a wolf, a sheep and a cabbage. He needed to take them to the other side by boat. However, the boat can only accommodate the farmer himself and one other thing (either a wolf, or a sheep, or a cabbage). If the farmer is not present, the wolf will eat the sheep, and the sheep will eat the cabbage. Please solve this problem for the farmer.

This is actually a state search problem, which can be solved by backtracking. The farmer, wolf, sheep, and cabbage have two states, that is, on the left bank of the river (assuming that the farmer is on the left bank at the beginning) or the right bank of the river. There is actually a state of the ship here. Since the state of the ship must be consistent with the state of the farmer, there is no need for additional consideration. These states are easily represented by sets of bits:

 const (
  FARMER = iota
  WOLF
  SHEEP
  CABBAGE
)

Write a function to determine if a state is legal. There are two states that are not legal:

• The wolf is on the same side as the sheep and not on the same side as the farmer. The wolf will eat the sheep
• Sheep and cabbage are on the same side, and not on the same side as the farmer. The sheep will eat the cabbage

 func IsStateValid(state *bitset.BitSet) bool {
  if state.Test(WOLF) == state.Test(SHEEP) &&
    state.Test(WOLF) != state.Test(FARMER) {
    // 狼和羊在同一边，并且不和农夫在同一边
    // 狼会吃掉羊，非法
    return false
  }

  if state.Test(SHEEP) == state.Test(CABBAGE) &&
    state.Test(SHEEP) != state.Test(FARMER) {
    // 羊和白菜在同一边，并且不和农夫在同一边
    // 羊会吃掉白菜，非法
    return false
  }

  return true
}

Next write the search function:

 func search(b *bitset.BitSet, visited map[string]struct{}) bool {
  if !IsStateValid(b) {
    return false
  }

  if _, exist := visited[b.String()]; exist {
    // 状态已遍历
    return false
  }

  if b.Count() == 4 {
    return true
  }

  visited[b.String()] = struct{}{}
  for index := uint(FARMER); index <= CABBAGE; index++ {
    if b.Test(index) != b.Test(FARMER) {
      // 与农夫不在一边，不能带上船
      continue
    }

    // 带到对岸去
    b.Flip(index)
    if index != FARMER {
      // 如果 index 为 FARMER，表示不带任何东西
      b.Flip(FARMER)
    }

    if search(b, visited) {
      return true
    }

    // 状态恢复
    b.Flip(index)
    if index != FARMER {
      b.Flip(FARMER)
    }
  }

  return false
}

Main function:

 func main() {
  b := bitset.New(4)

  visited := make(map[string]struct{})
  fmt.Println(search(b, visited))
}

Initially, all states are 0, and all states are 1 after reaching the other side, so b.Count() == 4 means that they have reached the other side. Since the search is blind, it may loop indefinitely: this time the farmer brings the sheep to the opposite bank, and the next time he brings it back from the opposite bank. So we need to do state deduplication. bitset.String() Returns the string representation of the current bit set, we use this to judge whether the state is repeated.

The for loop tries to bring various items in turn, or nothing. Driver search process.

If you want to get the correct action sequence of the farmer, you can add a parameter to search to record the operation of each step:

 func search(b *bitset.BitSet, visited map[string]struct{}, path *[]*bitset.BitSet) bool {
  // 记录路径
  *path = append(*path, b.Clone())
  if b.Count() == 4 {
    return true
  }

  // ...
  *path = (*path)[:len(*path)-1]

  return false
}

func main() {
  b := bitset.New(4)

  visited := make(map[string]struct{})
  var path []*bitset.BitSet
  if search(b, visited, &path) {
    PrintPath(path)
  }
}

If a solution is found, print it:

 var names = []string{"农夫", "狼", "羊", "白菜"}

func PrintState(b *bitset.BitSet) {
  fmt.Println("=======================")
  fmt.Println("河左岸：")
  for index := uint(FARMER); index <= CABBAGE; index++ {
    if !b.Test(index) {
      fmt.Println(names[index])
    }
  }

  fmt.Println("河右岸：")
  for index := uint(FARMER); index <= CABBAGE; index++ {
    if b.Test(index) {
      fmt.Println(names[index])
    }
  }
  fmt.Println("=======================")
}

func PrintMove(cur, next *bitset.BitSet) {
  for index := uint(WOLF); index <= CABBAGE; index++ {
    if cur.Test(index) != next.Test(index) {
      if !cur.Test(FARMER) {
        fmt.Printf("农夫将【%s】从河左岸带到河右岸\n", names[index])
      } else {
        fmt.Printf("农夫将【%s】从河右岸带到河左岸\n", names[index])

      }
      return
    }
  }

  if !cur.Test(FARMER) {
    fmt.Println("农夫独自从河左岸到河右岸")
  } else {
    fmt.Println("农夫独自从河右岸到河左岸")
  }
}

func PrintPath(path []*bitset.BitSet) {
  cur := path[0]
  PrintState(cur)

  for i := 1; i < len(path); i++ {
    next := path[i]
    PrintMove(cur, next)
    PrintState(next)
    cur = next
  }
}

run:

 =======================
河左岸：
农夫
狼
羊
白菜

河右岸：
=======================
农夫将【羊】从河左岸带到河右岸
=======================
河左岸：
狼
白菜

河右岸：
农夫
羊
=======================
农夫独自从河右岸到河左岸
=======================
河左岸：
农夫
狼
白菜

河右岸：
羊
=======================
农夫将【狼】从河左岸带到河右岸
=======================
河左岸：
白菜

河右岸：
农夫
狼
羊
=======================
农夫将【羊】从河右岸带到河左岸
=======================
河左岸：
农夫
羊
白菜

河右岸：
狼
=======================
农夫将【白菜】从河左岸带到河右岸
=======================
河左岸：
羊

河右岸：
农夫
狼
白菜
=======================
农夫独自从河右岸到河左岸
=======================
河左岸：
农夫
羊

河右岸：
狼
白菜
=======================
农夫将【羊】从河左岸带到河右岸
=======================
河左岸：

河右岸：
农夫
狼
羊
白菜
=======================

That is, the farmer's operation process is: bring the sheep to the right bank -> return alone -> bring the cabbage to the right bank -> then bring the sheep back to the left bank -> bring the wolf to the right bank -> return alone -> finally bring the sheep to the right bank - > Done.

Why use it?

It seems that you can use the bit operation directly, why use the bitset library?

Because of generality, the two examples listed above are small integer values. If the integer value exceeds 64 bits, we must store it by slicing. At this time, the handwriting operation is very inconvenient and prone to errors.

The advantages of the library are reflected in:

• generic enough
• Continuous optimization
• mass use

As long as the externally provided interface remains the same, it can always optimize the internal implementation. Although we can do it, it is time-consuming and labor-intensive. Moreover, some optimizations involve more complex algorithms, which are difficult and error-prone to implement by themselves.

A very typical example is to find the number of 1s in the binary representation of a uint64 (popcnt, or population count). There are many ways to achieve this.

The most direct, we count one by one:

 func popcnt1(n uint64) uint64 {
  var count uint64

  for n > 0 {
    if n&1 == 1 {
      count++
    }

    n >>= 1
  }

  return count
}

Considering space for time, we can pre-calculate the number of 1s in the binary representation of the 256 numbers 0-255, and then calculate it every 8 bits, possibly reducing the number of calculations to 1/8 of the previous number. This is also how it is done in the standard library:

 const pop8tab = "" +
  "\x00\x01\x01\x02\x01\x02\x02\x03\x01\x02\x02\x03\x02\x03\x03\x04" +
  "\x01\x02\x02\x03\x02\x03\x03\x04\x02\x03\x03\x04\x03\x04\x04\x05" +
  "\x01\x02\x02\x03\x02\x03\x03\x04\x02\x03\x03\x04\x03\x04\x04\x05" +
  "\x02\x03\x03\x04\x03\x04\x04\x05\x03\x04\x04\x05\x04\x05\x05\x06" +
  "\x01\x02\x02\x03\x02\x03\x03\x04\x02\x03\x03\x04\x03\x04\x04\x05" +
  "\x02\x03\x03\x04\x03\x04\x04\x05\x03\x04\x04\x05\x04\x05\x05\x06" +
  "\x02\x03\x03\x04\x03\x04\x04\x05\x03\x04\x04\x05\x04\x05\x05\x06" +
  "\x03\x04\x04\x05\x04\x05\x05\x06\x04\x05\x05\x06\x05\x06\x06\x07" +
  "\x01\x02\x02\x03\x02\x03\x03\x04\x02\x03\x03\x04\x03\x04\x04\x05" +
  "\x02\x03\x03\x04\x03\x04\x04\x05\x03\x04\x04\x05\x04\x05\x05\x06" +
  "\x02\x03\x03\x04\x03\x04\x04\x05\x03\x04\x04\x05\x04\x05\x05\x06" +
  "\x03\x04\x04\x05\x04\x05\x05\x06\x04\x05\x05\x06\x05\x06\x06\x07" +
  "\x02\x03\x03\x04\x03\x04\x04\x05\x03\x04\x04\x05\x04\x05\x05\x06" +
  "\x03\x04\x04\x05\x04\x05\x05\x06\x04\x05\x05\x06\x05\x06\x06\x07" +
  "\x03\x04\x04\x05\x04\x05\x05\x06\x04\x05\x05\x06\x05\x06\x06\x07" +
  "\x04\x05\x05\x06\x05\x06\x06\x07\x05\x06\x06\x07\x06\x07\x07\x08"

func popcnt2(n uint64) uint64 {
  var count uint64

  for n > 0 {
    count += uint64(pop8tab[n&0xff])
    n >>= 8
  }

  return count
}

And finally the algorithm in the bitset library:

 func popcnt3(x uint64) (n uint64) {
  x -= (x >> 1) & 0x5555555555555555
  x = (x>>2)&0x3333333333333333 + x&0x3333333333333333
  x += x >> 4
  x &= 0x0f0f0f0f0f0f0f0f
  x *= 0x0101010101010101
  return x >> 56
}

Performance testing of the above three implementations:

 goos: windows
goarch: amd64
pkg: github.com/darjun/go-daily-lib/bitset/popcnt
cpu: Intel(R) Core(TM) i7-7700 CPU @ 3.60GHz
BenchmarkPopcnt1-8         52405             24409 ns/op
BenchmarkPopcnt2-8        207452              5443 ns/op
BenchmarkPopcnt3-8       1777320               602 ns/op
PASS
ok      github.com/darjun/go-daily-lib/bitset/popcnt    4.697s

popcnt3 has a 40x performance improvement over popcnt1. In learning, we can try to build wheels by ourselves to deepen our understanding of technology. But in engineering, it is usually more inclined to use stable and efficient libraries.

Summarize

This article introduces the use of bitsets with the help of the bitset library. And the application of bitset solves the problem of farmers crossing the river. Finally we discussed why use a library?

If you find a fun and easy-to-use Go language library, you are welcome to submit an issue on the Go daily library GitHub😄

a little gossip

I find human inertia to be terrifying. Although I haven't written an article in the past six months, it was initially due to work reasons, and later it was simply due to inertia and laziness. And always "pretending to be busy" to avoid things that take time and energy. In this kind of competition between wanting to move and not wanting to move, time just flies by.

We are always complaining about no time, no time. But think about it carefully and calculate carefully, we actually spend a lot of time on short videos and playing games.

Last week, I saw an article "Life is Not Short" in Teacher Ruan Yifeng's weekly magazine, and I was deeply moved after reading it. People always have to persist, and life has meaning.

refer to

1. bitset GitHub: github.com/bits-and-blooms/bitset
2. Go daily library GitHub: https://github.com/darjun/go-daily-lib

I

My blog: https://darjun.github.io

Welcome to pay attention to my WeChat public account [GoUpUp], learn together and make progress together~