Python Functional Programming Series 012: Lazy List Generator and Iterator

Because this series is still based on some Python , we won't go into too much detail here to introduce the basic knowledge. But to go back to our previous topic, we will use iterators and generators to implement the previous exponential function.

Of course, we still need to return to the question of what a lazy list is. In fact, returning to the original concept of lazy evaluation, the concept of a lazy list is actually a list of "values calculated when needed". When we call iter , there is no particular advantage over common objects. We can imaginary, in fact iter transformed [1, 2, 3, 4] results actually are as follows:

def yield_list():
    yield 1
    yield 2
    yield 3
    yield 4

The only advantage, as we have mentioned before, is that f and g are repeatedly applied, we calculate g(f(x)) f to each value in the list and then apply g . There is a great advantage here, that is, unnecessary calculations can be avoided when terminating early. For example, following a for which case, we are to find a list of ls application f after the function If the result is equal to a returns index otherwise None :

def find_index_apply_f(f, ls, a):
    for i, x in enumerate(ls):
        if f(x) == a:
            return i
    return None

>>> find_index_apply_f(lambda x: x + 1, [1, 2, 3, 4, 5], 3)

Now, jumping out here in advance can reduce a lot of calculations, but it is difficult to use a normal list. We map , but if lazy evaluation, we can stop when needed. This is something that must be implemented to replace loops with list operations.

The biggest application of the second lazy list is an infinite list. For example, in the following generator, we can generate an infinite list x Later we will talk about this abstraction that we have used in various occasions.

def yield_x_forever(x):
    while True:
        yield x

Implement some common (lazy) list operations

Most of the functions for manipulating iterators/generators can be found in itertoools . But, here we still have to achieve some very functional function, easy to operate beyond:

1. head

head is very simple, that is, take the first element of the (lazy) list:

head = next

2. take

take is the first N values of the list. This can be implemented as a triggered calculation (converted into a non-lazy object, generally a value or list) or a version that does not trigger the calculation. Below we implement the function that triggers the calculation.

def take(n, it):
    return [x for x in islice(it, n)]

take_curry = lambda n: lambda it: take(n, it)

3. drop

drop the contrary, 06177845987c0f deletes the first N values.

def drop(n, it):
    return islice(it, n, None)

4. tail

tail is head , which can be realized drop

from functools import partial

tail = partial(drop, 1)

5. iterate

iterate is the key function to be used, which is to realize an infinite list through an iterative function and initial value:

def iterate(f, x):
    yield x
    yield from iterate(f, f(x))

For example, to implement an infinite list of all positive and even numbers:

positive_even_number = iterate(lambda x: x + 2, 2)

Of course, more simply written using itertools inside repeat and accumulate :

def iterate(f, x):
    return accumulate(repeat(x), lambda fx, _: f(fx))

Simple practice

Example 1: Find the index

Let's go back to the previous example of finding the index, we can implement the lazy list version.

The first idea is to iterate directly from x , multiply by x each time, and then take out the first n values and get the last one:

power = lambda x, n: take(n, iterate(lambda xx: xx * x, x))[-1]

The other is to first x an infinite length of 06177845987dc6, take out the first n , and multiply it to reduce :

power = lambda x, n: reduce(
    lambda x, y: x * y, 
    take(n, iterate(lambda _: x, x))

Of course, we can also use the generator generate an infinite list:

def yield_power(x, init=x):
    yield init
    yield from yield_power(x, init * x)

Example 2: Find

Let's go back to the example explained above. We need to find the a that is equal to the value of index f an infinite list. If it is not lazy, this must be jumped out in advance and it is impossible to achieve it.

def find_a_in_lazylist(f, lls, a):
    return head(filter(lambda x: f(x[1]) == a, enumerat(lls)))[0]


This chapter reviews the Python , and shows how to apply these concepts to some data manipulation applications. Of course, we must deeply feel that functional programming is very close to data. It focuses on data rather than project structure, which is very different from object programming. Most object-based programming tutorials tend to outline the concepts of layering and structure, really because this is where object-based programming is good.

In the teaching project I implemented fppy (click here to go to github ) , I implemented a LazyList class with the built-in python module, which can be used to complete all the above examples in chain writing:

power1 = lambda x, n: LazyList.from_iter(x)(lambda xx: x * x).take(n).last
power2 = lambda x, n: LazyList.from_iter(x)(lambda _: x).take(n).reduce(lambda xx, yy: xx * yy)

find_a_in_lazylist = lambda f, lls, a: LazyList(lls)\
    .zip_with(LazyList.from_iter(0)(lambda x: x + 1))\
    .filter(lambda x: f(x[1]) == a)\

λ and τ
1.2k 声望
101 粉丝
0 条评论
最好用的 python 库合集
🎈 分词 - jieba优秀的中文分词库,依靠中文词库,利用词库确定汉子之间关联的概率,形成分词结果 {代码...} 🎈 词云库 - wordcloud对数据中出现频率较高的 关键词 生成的一幅图像,予以视觉上的突出 {代码...} 🎈 ...

tiny极客11阅读 3.4k评论 2

使用python做web开发面临的一个最大的问题就是性能,在解决C10K问题上显的有点吃力。有些异步框架Tornado、Twisted、Gevent 等就是为了解决性能问题。这些框架在性能上有些提升,但是也出现了各种古怪的问题难以...

jysong6阅读 3.9k评论 3

你是不是也有这样的操作,比如你需要使用「电子邮箱正则表达式」,首先想到的就是直接百度上搜索一个,然后采用 CV 大法神奇地接入到你的代码中?

良许4阅读 2.2k


良许5阅读 1.8k


二毛erma02阅读 10.1k评论 3

大家好,我是良许。从去年 12 月开始,我已经在视频号、抖音等主流视频平台上连续更新视频到现在,并得到了不错的评价。每个视频都花了很多时间精力用心制作,欢迎大家关注哦~考虑到有些小伙伴没有看过我的视频,...

良许3阅读 1.8k


Harpsichord12073阅读 7.3k

1.2k 声望
101 粉丝