Comparison of Python coroutine and JavaScript coroutine

Preface

I didn't have much contact with the front end before and didn't understand the asynchronous operation of JavaScript. Now I have a little understanding and found that the development history of python and JavaScript's coroutine is almost the same!
Here is a general horizontal comparison and summary to facilitate the understanding and absorption of newcomers who are interested in these two languages.

Common appeal

• With the multi-core CPU, it is necessary to implement concurrent functions due to its own historical reasons (single-threaded environment)
• Simplify code, avoid callback hell, keyword support
• Effective use of operating system resources and hardware: Compared with threads, coroutines take up less resources and have faster contexts

What is a coroutine

To sum up, a coroutine is a function that satisfies the following conditions:

• Execution can be paused (the paused expression is called the pause point)
• Can be restored from the suspension point (retaining its original parameters and local variables)
• The event loop is the underlying cornerstone of asynchronous programming

Chaotic history

The evolution of Python coroutines

• In Python2.2, generators were introduced for the first time
• In Python 2.5, the yield keyword was added to the grammar
• In Python 3.4, there is yield from (yield from is approximately equal to yield + exception handling + send), and the experimentally introduced asynchronous I/O framework asyncio (PEP 3156)
• Added async/await syntax in Python3.5 (PEP 492)
• The asyncio library in Python 3.6 is "regulated" (the official document will be much clearer afterwards)

During the development of the main line, there have also been many branch-line coroutine implementations such as Gevent.

def foo():
    print("foo start")
    a = yield 1
    print("foo a", a)
    yield 2
    yield 3
    print("foo end")


gen = foo()
# print(gen.next())
# gen.send("a")
# print(gen.next())
# print(foo().next())
# print(foo().next())

# 在python3.x版本中,python2.x的g.next()函数已经更名为g.__next__(),使用next(g)也能达到相同效果。
# next()跟send()不同的地方是,next()只能以None作为参数传递,而send()可以传递yield的值.

print(next(gen))
print(gen.send("a"))
print(next(gen))
print(next(foo()))
print(next(foo()))

list(foo())

"""
foo start
1
foo a a
2
3
foo start
1
foo start
1
foo start
foo a None
foo end
"""

The evolution of JavaScript coroutines

• Sync code
• Asynchronous JavaScript: callback hell
• ES6 introduces Promise/a+, Generators (syntax function* foo()() can give function execution pause/save context/resume execution status), the new keyword yield makes the generator function pause.
• ES7 introduces async function/await syntactic sugar, async can declare an asynchronous function (wrap the Generator function and the automatic executor in a function), this function needs to return a Promise object. await can wait for a Promise object to resolve and get the result,

Callback functions are also used in Promises. A callback function is passed in both the then and catch methods, which are executed when the Promise is satisfied and rejected, so that it can be chained to complete a series of tasks.
In short, it is to turn the nested callbacks into .then().then()..., so as to make the code writing and reading more intuitive

The underlying implementation mechanism of Generator is Coroutine.

function* foo() {
    console.log("foo start")
    a = yield 1;
    console.log("foo a", a)
    yield 2;
    yield 3;
    console.log("foo end")
}

const gen = foo();
console.log(gen.next().value); // 1
// gen.send("a") // http://www.voidcn.com/article/p-syzbwqht-bvv.html SpiderMonkey引擎支持 send 语法
console.log(gen.next().value); // 2
console.log(gen.next().value); // 3
console.log(foo().next().value); // 1
console.log(foo().next().value); // 1

/*
foo start
1
foo a undefined
2
3
foo start
1
foo start
1
*/

Python coroutine mature body

Awaitable objects can be used in await statements. There are three main types of awaitable objects: coroutine, task and Future.

Coroutine:

• Coroutine function: a function defined in the form of async def;
• Coroutine object: The object returned by calling the coroutine function.
• Old-style generator-based coroutine

Task (Task object):

• Tasks are used to "parallel" scheduling coroutines. When a coroutine is encapsulated as a task through functions such as asyncio.create_task(), the coroutine will be automatically scheduled for execution.
• The Task object is used to run the coroutine in the event loop. If a coroutine is waiting for a Future object, the Task object will suspend the execution of the coroutine and wait for the Future object to complete. When the Future object is completed, the packaged coroutine will resume execution.
• The event loop uses collaborative scheduling: an event loop runs a Task object at a time. A Task object will wait for a Future object to complete, and the event loop will run other Tasks, callbacks, or perform IO operations.
• asyncio.Task inherits all its APIs except Future.set_result() and Future.set_exception() from Future.

Future:

• The Future object is used to link the low-level callback code with the high-level asynchronous/waiting code.
• After writing asynchronous code without the callback method, in order to obtain the result of the asynchronous call, a Future object is introduced. Future encapsulates the interaction behavior with loop. The add_done_callback method registers a callback function with epoll. When the result property gets the return value, it will run the previously registered callback function and pass it up to the coroutine.

Several event loops (event loop):

• libevent/libev: The network library used by Gevent (greenlet + early libevent, later libev), widely used;
• tornado: IOLOOP implemented by the tornado framework itself;
• picoev: The network library used by meinheld (greenlet+picoev) is small and lightweight. Compared with libevent, it has improved data structure and event detection model, so it is faster. But from github it seems to have fallen into disrepair for a long time, and not many people use it.
• uvloop: A rising star in the Python3 era. Guido created the asyncio library. Asyncio can be configured with a pluggable event loop, but it needs to meet related API requirements. uvloop inherits from libuv and wraps some low-level structures and functions with Python objects. Currently Sanic framework is based on this library

example

import asyncio
import time


async def exec():
    await asyncio.sleep(2)
    print('exec')

# 这种会和同步效果一直
# async def go():
#     print(time.time())
#     c1 = exec()
#     c2 = exec()
#     print(c1, c2)
#     await c1
#     await c2
#     print(time.time())

# 正确用法
async def go():
    print(time.time())
    await asyncio.gather(exec(),exec()) # 加入协程组统一调度
    print(time.time())

if __name__ == "__main__":
    asyncio.run(go())

JavaScript coroutine mature body

Promise continues to be used

Promise is essentially a state machine, used to represent the final completion (or failure) of an asynchronous operation, and its result value. It has three states:

• pending: The initial state, which is neither a success nor a failure state.
• fulfilled: means that the operation completed successfully.
• rejected: means the operation failed.

In the end, Promise will have two states, one success and one failure. When pending changes, the Promise object will call different processing functions according to the final state.

async, await syntactic sugar

Async and await are the encapsulation of the combination of Generator and Promise, which makes the original asynchronous code closer to the writing of synchronous code in form, and is more friendly to error handling/conditional branching/exception stack/debugging and other operations.

The operating mechanism of js asynchronous execution

1) All tasks are executed on the main thread, forming an execution stack.
2) In addition to the main thread, there is also a "task queue" (task queue). As long as the asynchronous task has a running result, an event is placed in the "task queue".
3) Once all synchronization tasks in the "execution stack" are executed, the system will read the "task queue". Those corresponding asynchronous tasks end the waiting state, enter the execution stack and start execution.

When encountering synchronous tasks, they are directly executed, and when encountering asynchronous tasks, they are classified into macro-tasks and micro-tasks.
When the execution of the current execution stack is completed, all events in the micro task queue will be processed immediately, and then an event will be taken out of the macro task queue. In the same event loop, the micro task is always executed before the macro task.

example

var sleep = function (time) {
    console.log("sleep start")
    return new Promise(function (resolve, reject) {
        setTimeout(function () {
            resolve();
        }, time);
    });
};

async function exec() {
    await sleep(2000);
    console.log("sleep end")
}

async function go() {
    console.log(Date.now())
    c1 = exec()
    console.log("-------1")
    c2 = exec()
    console.log(c1, c2)
    await c1;
    console.log("-------2")
    await c2;
    console.log(c1, c2)
    console.log(Date.now())
}

go();

The event loop divides the task:

• The main thread loops to read events from the "task queue"
• Macro task (macro task) js synchronized code blocks, setTimeout, setInterval, XMLHttprequest, setImmediate, I/O, UI rendering, etc., are essentially tasks that participate in the event loop.
• Micro task Promise, process.nextTick (node environment), Object.observe, MutationObserver, etc., are essentially tasks that are executed directly in the Javascript engine and do not participate in the event loop.

Extended reading Node.js

Summary and comparison

Description	python	JavaScript	Reviews
process	Single process	Single process	Unanimous
Interruption/Resume	yield ,yield from,next,send	yield ,next	Basically the same, but JavaScript has no need for send
Future object (callback wrapper)	Futures	Promise	Solve the callback, the same idea
Builder	generator	Generator	Encapsulate yield as a coroutine Coroutine, the idea is the same
Keywords after maturity	async、await	async、await	Keyword support, the same as a dime
Event loop	The core of the asyncio application. The event loop will run asynchronous tasks and callbacks, perform network IO operations, and run child processes. The asyncio library supports many APIs and is highly controllable	The browser environment is basically a black box, and the outside is basically uncontrollable. There are priority classifications for tasks, and there are differences in scheduling methods.	There is a big difference here, the operating environment is different, and the scheduling of tasks is different. Python may be more comparable to Node.js about the event loop, and you need to continue to learn here.

It's basically over here. After reading it, I don't know what you will think. If there are errors, please feel free to enlighten me.
By the way, the official documentation of Python's asyncio library is now much clearer, rather than the documentation before Python 3.5 that makes people incomprehensible and impossible to start. I hope that more third-party libraries can follow up and develop more. Support asynchronous library. After all, there is still a gap compared with Go native coroutine.

reference

• python asyncio five stars
• From event loop to async await to understand the event loop mechanism five stars
• JS's Event Loop mechanism five stars
• JavaScript Five stars
• JavaScript yield next send
• JavaScript iterators and generators
• JavaScript Promise
• Python asynchronous history
• The evolution of Python coroutine technology
• JS
• Node.js event loop, timer and process.nextTick()