Python Functional Programming Series 012: Lazy List Generator and Iterator

Because this series is still based on some Python , we won't go into too much detail here to introduce the basic knowledge. But to go back to our previous topic, we will use iterators and generators to implement the previous exponential function.

Of course, we still need to return to the question of what a lazy list is. In fact, returning to the original concept of lazy evaluation, the concept of a lazy list is actually a list of "values calculated when needed". When we call iter , there is no particular advantage over common objects. We can imaginary, in fact iter transformed [1, 2, 3, 4] results actually are as follows:

def yield_list():
    yield 1
    yield 2
    yield 3
    yield 4

The only advantage, as we have mentioned before, is that f and g are repeatedly applied, we calculate g(f(x)) f to each value in the list and then apply g . There is a great advantage here, that is, unnecessary calculations can be avoided when terminating early. For example, following a for which case, we are to find a list of ls application f after the function If the result is equal to a returns index otherwise None :

def find_index_apply_f(f, ls, a):
    for i, x in enumerate(ls):
        if f(x) == a:
            return i
        else:
            continue
    return None

>>> find_index_apply_f(lambda x: x + 1, [1, 2, 3, 4, 5], 3)
1

Now, jumping out here in advance can reduce a lot of calculations, but it is difficult to use a normal list. We map , but if lazy evaluation, we can stop when needed. This is something that must be implemented to replace loops with list operations.

The biggest application of the second lazy list is an infinite list. For example, in the following generator, we can generate an infinite list x Later we will talk about this abstraction that we have used in various occasions.

def yield_x_forever(x):
    while True:
        yield x

Implement some common (lazy) list operations

Most of the functions for manipulating iterators/generators can be found in itertoools . But, here we still have to achieve some very functional function, easy to operate beyond:

1. head

head is very simple, that is, take the first element of the (lazy) list:

head = next

2. take

take is the first N values of the list. This can be implemented as a triggered calculation (converted into a non-lazy object, generally a value or list) or a version that does not trigger the calculation. Below we implement the function that triggers the calculation.

def take(n, it):
    """将前n个元素固定转为列表
    """
    return [x for x in islice(it, n)]

take_curry = lambda n: lambda it: take(n, it)

3. drop

drop the contrary, 06177845987c0f deletes the first N values.

def drop(n, it):
    """剔除前n个元素
    """
    return islice(it, n, None)

4. tail

tail is head , which can be realized drop

from functools import partial

tail = partial(drop, 1)

5. iterate

iterate is the key function to be used, which is to realize an infinite list through an iterative function and initial value:

def iterate(f, x):
    yield x
    yield from iterate(f, f(x))

For example, to implement an infinite list of all positive and even numbers:

positive_even_number = iterate(lambda x: x + 2, 2)

Of course, more simply written using itertools inside repeat and accumulate :

def iterate(f, x):
    return accumulate(repeat(x), lambda fx, _: f(fx))

Simple practice

Example 1: Find the index

Let's go back to the previous example of finding the index, we can implement the lazy list version.

The first idea is to iterate directly from x , multiply by x each time, and then take out the first n values and get the last one:

power = lambda x, n: take(n, iterate(lambda xx: xx * x, x))[-1]

The other is to first x an infinite length of 06177845987dc6, take out the first n , and multiply it to reduce :

power = lambda x, n: reduce(
    lambda x, y: x * y, 
    take(n, iterate(lambda _: x, x))
)

Of course, we can also use the generator generate an infinite list:

def yield_power(x, init=x):
    yield init
    yield from yield_power(x, init * x)

Example 2: Find

Let's go back to the example explained above. We need to find the a that is equal to the value of index f an infinite list. If it is not lazy, this must be jumped out in advance and it is impossible to achieve it.

def find_a_in_lazylist(f, lls, a):
    return head(filter(lambda x: f(x[1]) == a, enumerat(lls)))[0]

Summarize

This chapter reviews the Python , and shows how to apply these concepts to some data manipulation applications. Of course, we must deeply feel that functional programming is very close to data. It focuses on data rather than project structure, which is very different from object programming. Most object-based programming tutorials tend to outline the concepts of layering and structure, really because this is where object-based programming is good.

In the teaching project I implemented fppy (click here to go to github ) , I implemented a LazyList class with the built-in python module, which can be used to complete all the above examples in chain writing:

power1 = lambda x, n: LazyList.from_iter(x)(lambda xx: x * x).take(n).last
power2 = lambda x, n: LazyList.from_iter(x)(lambda _: x).take(n).reduce(lambda xx, yy: xx * yy)

find_a_in_lazylist = lambda f, lls, a: LazyList(lls)\
    .zip_with(LazyList.from_iter(0)(lambda x: x + 1))\
    .filter(lambda x: f(x[1]) == a)\
    .split_head()[0]