关于monkey.patch_all 的问题,我网上看到的这个,这里的pool.Pool(10)是10个线程吗?

开多线程时monkey会阻塞住线程的继续执行,需要对monkey.patch_all进行处理,解决方案有2种:

1. monkey.patch_all(thread=False) 
2. 直接用gevent.sleep()

from bs4 import BeautifulSoup
import requests
import gevent
import time
from gevent import monkey, pool
from threading import Thread
monkey.patch_all()
jobs = []
links = []
p = pool.Pool(10)

url = [
    'http://www.baidu.com',

]
def get_links(url):
    # time.sleep(2)
    r = requests.get(url)
    if r.status_code == 200:
        soup = BeautifulSoup(r.text, 'lxml')
        print('>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>')
        links + soup.find_all('a')

def start_process():
    time.sleep(2)
    # for url in urls:
    #     jobs.append(p.spawn(get_links, url))
    # gevent.joinall(jobs)
if __name__ == '__main__':
    t = Thread(target=start_process, )
    t.start()
    t.join()

阅读 2.6k
撰写回答
你尚未登录,登录后可以
  • 和开发者交流问题的细节
  • 关注并接收问题和回答的更新提醒
  • 参与内容的编辑和改进,让解决方法与时俱进
推荐问题