python 中关于多线程的两个问题

第一个问题:

import queue
import threading
import time

for i in range(20):
    a = threading.Thread(target = worker)
    a.setDaemon(True)
    a.start()
    

我想知道,在这个for循环里,这里设置的setDaemon里所绑定的主线程都是一样的吗?还是说因为for循环把每个子线程的守护线程都绑定为了上一个子线程?

第二个问题:

import queue
import threading
import time

aqueue = queue.Queue()
for i in range(10):
    aqueue.put(i)

def worker():
    while True:
        time.sleep(5)
        item = aqueue.get()
        print(item)

for i in range(20):
    a = threading.Thread(target = worker)
    a.setDaemon(True)
    a.start()
    
aqueue.join()    

在添加queue模块后,让我疑惑的是,程序在跑完aqueue里的所有项后却没有退出,我查了一下官网queue.join()的定义是Blocks until all items in the queue have been gotten and processed.这里子线程和主线程都执行完了,为什么没有退出呢?

阅读 7.6k
3 个回答

我也刚看,试了好久,初步知道怎么回事了.
先回答你的两个问题:
1、setDaemon所绑定的都是主线程,都是一样的,即运行py文件第一次创建的那个线程(也是主进程),有且只有一个
2、queue.join()如果单独使用会被无限挂起,字面翻译为等待队列为空,再执行别的操作.但是他等待的队列不是我们创建的aqueue,而是一个与aqueue长度相等的一个"需要完成的子线程"队列,他判断的很可能是这个列表,当这个列表为空时queue.join()才成立,所以如楼上的那位所说queue.join()要结合aqueue.task_done()函数来使用,aqueue.task_done()的意思是每task_done一次 就从"需要完成的子线程"队列里删掉一个元素,这样在最后join的时候根据队列长度是否为零来判断队列是否结束,aqueue.task_done()用在子线程函数中,这样queue.join()才知道完成了多少个子线程,才不会无限挂起,也就是为什么你没退出的原因

但是你的程序也有问题:
主要的问题是,守护线程机制:主线程运行完毕去结束子线程时,由于有大量的子线程还在time.sleep(5),结束这些子线程会触发异常:

Fatal Python error: could not acquire lock for <_io.BufferedWriter name='<stdout>'> at interpreter shutdown, possibly due to daemon threads

Thread 0x000019cc (most recent call first):
  File "test.py", line 13 in worker
  File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 864 in run
  File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 916 in _bootstrap_inner
  File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 884 in _bootstrap

这是因为:
下面的一些说法是错误的,可以看我后面的补充,这里就不改了,是一种猜测,但是应该确实不是这样
喂不饱的子线程:你aqueue.put(i)了是10个元素,但是却运行了20个子线程来"吃"这10个元素,又因为这20个线程都是守护进程,当10个吃饱了,这个10个卡在了item = aqueue.get()等待接受元素,另外的那些很有可能卡在了time.sleep(5)上,而主线程运行完毕,准备结束所有的守护进程,我判断,结束卡在item = aqueue.get()的守护进程不会报错,而结束其他程序,如time.sleep会触发异常,这也是为啥楼上那位吧你的for i in range(20):改成了for i in range(1):的原因.

还有两种退出的方式:
1、是在子线程里判断队列是否为空:

if aqueue.empty() == True:
    break

为空就退出
2、主线程通过queue向子线程发送退出命令
其实这里一般用pipe管道的,Queue队列不好针对特定的线程进行退出操作,大概的伪码为:

主线程:

aqueue.put('quit')

子线程""

item = aqueue.get()
if item == 'quit':
    return
else:
    # "do something"

所以多进程,多线程还是要控制好程序逻辑,控制好调度.

我没用过守护进程,我一般用的是普通的用户进程,这样主线程跑完会等待所有子线程运行完毕再结束

其实还没完=_=:
后面的都是我的猜测,算是附加的,选修:
这程序的细节也值得我们关注:
这程序如果真去实践的话很容易炸,以下都是我的猜测
如上所述,我们修改程序,注意注释

import queue
import threading
import time

aqueue = queue.Queue()
# 发送了0到9的十个数据
for i in range(10):
    aqueue.put(i)

def worker():
    while True:
        # 输出当前线程,和主线程(可以看到主线程都一样)
        print('thread %s is running...\nmain_thread %s is running...\n\n' % (threading.current_thread().name,threading.main_thread()))
        time.sleep(1)
        item = aqueue.get()
        print(item)
        aqueue.task_done()
        print('thread end')


n = 1
"""注意这!!!!!!!!!!!!!!!!!!"""
# 运行了10个线程来处理这十个数据
for i in range(10):
    print("run thread Counts: ",n)
    n += 1
    a = threading.Thread(target = worker)
    a.setDaemon(True)
    a.start()

aqueue.join()
print('after to do')
print('end')

10个数据10个线程,time.cleep(5)秒结果会怎么样?运行正常?
很不幸运的炸了23333333
有一定的几率报错,和上面一样:

Fatal Python error: could not acquire lock for <_io.BufferedWriter name='<stdout>'> at interpreter shutdown, possibly due to daemon threads

Thread 0x0000063c (most recent call first):
  File "test.py", line 13 in worker
  File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 864 in run
  File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 916 in _bootstrap_inner
  File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 884 in _bootstrap

我电脑上大部分运行正常,只有几次会这样,为什么会这样?
换成10个数据9个线程就不会出错了
我猜测的情况是,初步认为:
直接看完整的输出,注意看有注释的地方:

run thread Counts:  1
thread Thread-1 is running...
main_thread <_MainThread(MainThread, started 6680)> is running...

run thread Counts:  2

thread Thread-2 is running...
main_thread <_MainThread(MainThread, started 6680)> is running...

run thread Counts:  3

thread Thread-3 is running...
main_thread <_MainThread(MainThread, started 6680)> is running...

run thread Counts:  4

thread Thread-4 is running...
main_thread <_MainThread(MainThread, started 6680)> is running...

run thread Counts:  5

thread Thread-5 is running...
main_thread <_MainThread(MainThread, started 6680)> is running...

run thread Counts:  6

thread Thread-6 is running...
main_thread <_MainThread(MainThread, started 6680)> is running...

run thread Counts:  7

thread Thread-7 is running...
main_thread <_MainThread(MainThread, started 6680)> is running...

run thread Counts:  8

thread Thread-8 is running...
main_thread <_MainThread(MainThread, started 6680)> is running...

run thread Counts:  9

thread Thread-9 is running...
main_thread <_MainThread(MainThread, started 6680)> is running...

run thread Counts:  10

thread Thread-10 is running...
main_thread <_MainThread(MainThread, started 6680)> is running...


0
1
3
4
thread end
thread end
5
2
thread Thread-2 is running...
main_thread <_MainThread(MainThread, started 6680)> is running...

thread end
thread end
thread end
thread Thread-3 is running...
main_thread <_MainThread(MainThread, started 6680)> is running...

thread end

thread Thread-4 is running...
main_thread <_MainThread(MainThread, started 6680)> is running...

thread Thread-1 is running...
main_thread <_MainThread(MainThread, started 6680)> is running...

thread Thread-5 is running...
main_thread <_MainThread(MainThread, started 6680)> is running...





6
8
thread Thread-6 is running...
main_thread <_MainThread(MainThread, started 6680)> is running...

thread end
thread end

thread Thread-7 is running...
main_thread <_MainThread(MainThread, started 6680)> is running...

7
9
thread Thread-9 is running...
main_thread <_MainThread(MainThread, started 6680)> is running...
"""到这里主线程已经完毕"""
thread end
after to do
"""
这里其实系统在调用守护线程的结束程序来结束所有子线程
"""
"""但是在结束守护进程程序后,又运行了一个不该运行的子线程,这就是为什么会报错的原因"""
thread Thread-10 is running...
main_thread <_MainThread(MainThread, started 6680)> is running...

end
thread end


thread Thread-8 is running...
main_thread <_MainThread(MainThread, started 6680)> is running...

Fatal Python error: could not acquire lock for <_io.BufferedWriter name='<stdout>'> at interpreter shutdown, possibly due to daemon threads

Thread 0x00000424 (most recent call first):
  File "test.py", line 14 in worker
  File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 864 in run
  File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 916 in _bootstrap_inner
  File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 884 in _bootstrap

Thread 0x00000f04 (most recent call first):
  File "test.py", line 14 in worker
  File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 864 in run
  File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 916 in _bootstrap_inner
  File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 884 in _bootstrap

Thread 0x0000165c (most recent call first):
  File "test.py", line 13 in worker
  File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 864 in run
  File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 916 in _bootstrap_inner
  File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 884 in _bootstrap

Thread 0x000018b0 (most recent call first):
  File "test.py", line 14 in worker
  File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 864 in run
  File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 916 in _bootstrap_inner
  File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 884 in _bootstrap

Thread 0x00000db4 (most recent call first):
  File "test.py", line 14 in worker
  File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 864 in run
  File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 916 in _bootstrap_inner
  File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 884 in _bootstrap

Thread 0x00001a88 (most recent call first):
  File "test.py", line 14 in worker
  File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 864 in run
  File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 916 in _bootstrap_inner
  File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 884 in _bootstrap

Thread 0x0000111c (most recent call first):
  File "test.py", line 14 in worker
  File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 864 in run
  File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 916 in _bootstrap_inner
  File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 884 in _bootstrap

Thread 0x0000177c (most recent call first):
  File "test.py", line 14 in worker
  File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 864 in run
  File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 916 in _bootstrap_inner
  File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 884 in _bootstrap

Thread 0x000008e4 (most recent call first):
  File "test.py", line 14 in worker
  File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 864 in run
  File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 916 in _bootstrap_inner
  File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 884 in _bootstrap

Thread 0x00001788 (most recent call first):
  File "test.py", line 14 in worker
  File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 864 in run
  File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 916 in _bootstrap_inner
  File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 884 in _bootstrap

Current thread 0x00001a18 (most recent call first):

看主要的几句

"""到这里主线程已经完毕"""
    thread end
    after to do
    """
    这里其实系统在调用守护线程的结束程序来结束所有子线程
    """
    """但是在结束守护进程程序后,又运行了一个不该运行的子线程,这就是为什么会报错的原因"""
    thread Thread-10 is running...
    main_thread <_MainThread(MainThread, started 6680)> is running...
    
    end
    thread end

先这样吧,可以自己运行下程序看看

看了另一个答主的,使用了a.join()也是一种选择

改成发送10个数据,用9个线程跑,效率是最高的(理论)

import queue
import threading
import time

aqueue = queue.Queue()
# 发送了0到9的十个数据
for i in range(10):
    aqueue.put(i)

def worker():
    while True:
        # 输出当前线程,和主线程(可以看到主线程都一样)
        print('thread %s is running...\nmain_thread %s is running...\n\n' % (threading.current_thread().name,threading.main_thread()))
        time.sleep(5)
        item = aqueue.get()
        print(item)
        aqueue.task_done()
        print('thread end')


n = 1
"""注意这!!!!!!!!!!!!!!!!!!"""
for i in range(9):
    print("run thread Counts: ",n)
    n += 1
    a = threading.Thread(target = worker)
    a.setDaemon(True)
    a.start()

aqueue.join()
print('after to do')
print('end')

import Queue
import threading
import time

aqueue = Queue.Queue()
for i in range(10):
    aqueue.put(i)

def worker():
    while True:
        time.sleep(1)
        item = aqueue.get()
        print(item)
        aqueue.task_done()

for i in range(1):
    a = threading.Thread(target = worker)
    a.setDaemon(True)
    a.start()

aqueue.join()
print 'the main thread is already over'

用 aqueue.task_done() 去通知阻塞的 join 是一种方法

第一个问题 一样的
第二个问题
1、队列只有十个值,你开了20个线程,aqueue.get()在取不到值的时候会使线程暂停,后面的线程取不到值程序当然不会退出。
2、while True:也会使线程不停止。
3、aqueue.join()会优先于线程执行,并阻塞主进程继续

如果你是想主进程等待线程结束再停止可以这么写

import queue
import threading
import time

aqueue = queue.Queue()
for i in range(10):
    aqueue.put(i)


def worker():

    time.sleep(5)
    item = aqueue.get()
    print(item)


for i in range(20):
    a = threading.Thread(target=worker)
    a.setDaemon(True)
    a.start()
    
    a.join()

# aqueue.join()
撰写回答
你尚未登录,登录后可以
  • 和开发者交流问题的细节
  • 关注并接收问题和回答的更新提醒
  • 参与内容的编辑和改进,让解决方法与时俱进
推荐问题