Programming Caprice: Algorithms Based on Historical State

Recently I thought of a problem about current limiting. For example, an api is limited to a maximum of 100 requests per second, then we need to implement such a limiting mechanism locally to ensure that the number of requests is less than 100 times in any one-second time period.

We set the time period as T (here is 1 second), the number of requests is R (here is 100 times), and then take a positive integer n (n > 1), which means that the time period is divided into n equally. Definition t = T / n is the duration of each copy. Definition r(t(i)) = R - sum(r‘(t(i-n))...r’(t(i-1))) , which means the maximum number of requests in any time slice, which is equal to R minus the sum of the number of requests in the past n time slices. Sorry I haven't started learning Latex yet and can't write formulas accurately. In short, this method can meet our needs, that is, in any time period of T, the number of requests is less than R times.

Here is a simple implementation:

 defmodule Limit do
  use GenServer

  @n 30
  @_R 100
  # ms
  @_T 1000

  def req() do
    GenServer.call(__MODULE__, :req)
  end

  def start() do
    if GenServer.whereis(__MODULE__) do
      GenServer.stop(__MODULE__)
    end

    GenServer.start_link(__MODULE__, :ok, name: __MODULE__)
  end

  def init(_) do
    :timer.send_interval(div(@_T, @n), :update)

    {:ok,
     %{
       queue: :queue.from_list(List.duplicate(0, @n)),
       r_max: @_R,
       r: 0
     }}
  end

  def handle_call(:req, _from, state = %{r: r, r_max: r_max}) do
    r = r + 1

    if r <= r_max do
      {:reply, :ok, %{state | r: r}}
    else
      {:reply, {:error, :limit}, state}
    end
  end

  def handle_info(:update, state = %{r: r, queue: q}) do
    ## debug
    # :queue.to_list(q) |> IO.inspect()

    {_, q} = :queue.out(q)
    q = :queue.in(r, q)
    s = :queue.fold(fn x, acc -> x + acc end, 0, q)
    r_max = @_R - s
    {:noreply, %{state | r: 0, r_max: r_max, queue: q}}
  end
end

Test case:

 defmodule LimitTest do
  use ExUnit.Case

  test "100 request at same time should all return ok" do
    Limit.start()

    r =
      for _ <- 1..100 do
        Limit.req()
      end
      |> Enum.all?(fn x -> x == :ok end)

    assert r
  end

  test "101 request at same time should return 100 ok and 1 error at last req" do
    Limit.start()

    r =
      for _ <- 1..100 do
        Limit.req()
      end
      |> Enum.all?(fn x -> x == :ok end)

    assert r

    assert Limit.req() == {:error, :limit}
  end

  test "request capacity should re-fill after 1 second (500 ms more to avoid race)" do
    Limit.start()

    for _ <- 1..100 do
      Limit.req()
    end

    :timer.sleep(1500)

    assert Limit.req() == :ok
  end
end

This historical state-based algorithm is used in many fields. For example, in the Bitcoin network, the current proof-of-work difficulty is adjusted according to the block generation speed in the past period, so that the block generation time is kept at about 10 minutes.