第一個問題:
import queue
import threading
import time
for i in range(20):
a = threading.Thread(target = worker)
a.setDaemon(True)
a.start()
我想知道,在這個for循環(huán)里,這里設置的setDaemon里所綁定的主線程都是一樣的嗎?還是說因為for循環(huán)把每個子線程的守護線程都綁定為了上一個子線程?
第二個問題:
import queue
import threading
import time
aqueue = queue.Queue()
for i in range(10):
aqueue.put(i)
def worker():
while True:
time.sleep(5)
item = aqueue.get()
print(item)
for i in range(20):
a = threading.Thread(target = worker)
a.setDaemon(True)
a.start()
aqueue.join()
在添加queue模塊后,讓我疑惑的是,程序在跑完aqueue里的所有項后卻沒有退出,我查了一下官網(wǎng)queue.join()的定義是Blocks until all items in the queue have been gotten and processed.這里子線程和主線程都執(zhí)行完了,為什么沒有退出呢?
我也剛看,試了好久,初步知道怎么回事了.
先回答你的兩個問題:
1、setDaemon所綁定的都是主線程,都是一樣的,即運行py文件第一次創(chuàng)建的那個線程(也是主進程),有且只有一個
2、queue.join()如果單獨使用會被無限掛起,字面翻譯為等待隊列為空,再執(zhí)行別的操作.但是他等待的隊列不是我們創(chuàng)建的aqueue,而是一個與aqueue長度相等的一個"需要完成的子線程"隊列,他判斷的很可能是這個列表,當這個列表為空時queue.join()才成立,所以如樓上的那位所說queue.join()要結合aqueue.task_done()函數(shù)來使用,aqueue.task_done()的意思是每task_done一次 就從"需要完成的子線程"隊列里刪掉一個元素,這樣在最后join的時候根據(jù)隊列長度是否為零來判斷隊列是否結束,aqueue.task_done()用在子線程函數(shù)中,這樣queue.join()才知道完成了多少個子線程,才不會無限掛起,也就是為什么你沒退出的原因
但是你的程序也有問題:
主要的問題是,守護線程機制:主線程運行完畢去結束子線程時,由于有大量的子線程還在time.sleep(5),結束這些子線程會觸發(fā)異常:
Fatal Python error: could not acquire lock for <_io.BufferedWriter name='<stdout>'> at interpreter shutdown, possibly due to daemon threads
Thread 0x000019cc (most recent call first):
File "test.py", line 13 in worker
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 864 in run
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 916 in _bootstrap_inner
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 884 in _bootstrap
這是因為:
下面的一些說法是錯誤的,可以看我后面的補充,這里就不改了,是一種猜測,但是應該確實不是這樣
喂不飽的子線程:你aqueue.put(i)了是10個元素,但是卻運行了20個子線程來"吃"這10個元素,又因為這20個線程都是守護進程,當10個吃飽了,這個10個卡在了item = aqueue.get()等待接受元素,另外的那些很有可能卡在了time.sleep(5)上,而主線程運行完畢,準備結束所有的守護進程,我判斷,結束卡在item = aqueue.get()的守護進程不會報錯,而結束其他程序,如time.sleep會觸發(fā)異常,這也是為啥樓上那位吧你的for i in range(20):改成了for i in range(1):的原因.
還有兩種退出的方式:
1、是在子線程里判斷隊列是否為空:
if aqueue.empty() == True:
break
為空就退出
2、主線程通過queue向子線程發(fā)送退出命令
其實這里一般用pipe管道的,Queue隊列不好針對特定的線程進行退出操作,大概的偽碼為:
主線程:
aqueue.put('quit')
子線程""
item = aqueue.get()
if item == 'quit':
return
else:
# "do something"
所以多進程,多線程還是要控制好程序邏輯,控制好調度.
我沒用過守護進程,我一般用的是普通的用戶進程,這樣主線程跑完會等待所有子線程運行完畢再結束
其實還沒完=_=:
后面的都是我的猜測,算是附加的,選修:
這程序的細節(jié)也值得我們關注:
這程序如果真去實踐的話很容易炸,以下都是我的猜測
如上所述,我們修改程序,注意注釋
import queue
import threading
import time
aqueue = queue.Queue()
# 發(fā)送了0到9的十個數(shù)據(jù)
for i in range(10):
aqueue.put(i)
def worker():
while True:
# 輸出當前線程,和主線程(可以看到主線程都一樣)
print('thread %s is running...\nmain_thread %s is running...\n\n' % (threading.current_thread().name,threading.main_thread()))
time.sleep(1)
item = aqueue.get()
print(item)
aqueue.task_done()
print('thread end')
n = 1
"""注意這!!!!!!!!!!!!!!!!!!"""
# 運行了10個線程來處理這十個數(shù)據(jù)
for i in range(10):
print("run thread Counts: ",n)
n += 1
a = threading.Thread(target = worker)
a.setDaemon(True)
a.start()
aqueue.join()
print('after to do')
print('end')
10個數(shù)據(jù)10個線程,time.cleep(5)秒結果會怎么樣?運行正常?
很不幸運的炸了23333333
有一定的幾率報錯,和上面一樣:
Fatal Python error: could not acquire lock for <_io.BufferedWriter name='<stdout>'> at interpreter shutdown, possibly due to daemon threads
Thread 0x0000063c (most recent call first):
File "test.py", line 13 in worker
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 864 in run
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 916 in _bootstrap_inner
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 884 in _bootstrap
我電腦上大部分運行正常,只有幾次會這樣,為什么會這樣?
換成10個數(shù)據(jù)9個線程就不會出錯了
我猜測的情況是,初步認為:
直接看完整的輸出,注意看有注釋的地方:
run thread Counts: 1
thread Thread-1 is running...
main_thread <_MainThread(MainThread, started 6680)> is running...
run thread Counts: 2
thread Thread-2 is running...
main_thread <_MainThread(MainThread, started 6680)> is running...
run thread Counts: 3
thread Thread-3 is running...
main_thread <_MainThread(MainThread, started 6680)> is running...
run thread Counts: 4
thread Thread-4 is running...
main_thread <_MainThread(MainThread, started 6680)> is running...
run thread Counts: 5
thread Thread-5 is running...
main_thread <_MainThread(MainThread, started 6680)> is running...
run thread Counts: 6
thread Thread-6 is running...
main_thread <_MainThread(MainThread, started 6680)> is running...
run thread Counts: 7
thread Thread-7 is running...
main_thread <_MainThread(MainThread, started 6680)> is running...
run thread Counts: 8
thread Thread-8 is running...
main_thread <_MainThread(MainThread, started 6680)> is running...
run thread Counts: 9
thread Thread-9 is running...
main_thread <_MainThread(MainThread, started 6680)> is running...
run thread Counts: 10
thread Thread-10 is running...
main_thread <_MainThread(MainThread, started 6680)> is running...
0
1
3
4
thread end
thread end
5
2
thread Thread-2 is running...
main_thread <_MainThread(MainThread, started 6680)> is running...
thread end
thread end
thread end
thread Thread-3 is running...
main_thread <_MainThread(MainThread, started 6680)> is running...
thread end
thread Thread-4 is running...
main_thread <_MainThread(MainThread, started 6680)> is running...
thread Thread-1 is running...
main_thread <_MainThread(MainThread, started 6680)> is running...
thread Thread-5 is running...
main_thread <_MainThread(MainThread, started 6680)> is running...
6
8
thread Thread-6 is running...
main_thread <_MainThread(MainThread, started 6680)> is running...
thread end
thread end
thread Thread-7 is running...
main_thread <_MainThread(MainThread, started 6680)> is running...
7
9
thread Thread-9 is running...
main_thread <_MainThread(MainThread, started 6680)> is running...
"""到這里主線程已經(jīng)完畢"""
thread end
after to do
"""
這里其實系統(tǒng)在調用守護線程的結束程序來結束所有子線程
"""
"""但是在結束守護進程程序后,又運行了一個不該運行的子線程,這就是為什么會報錯的原因"""
thread Thread-10 is running...
main_thread <_MainThread(MainThread, started 6680)> is running...
end
thread end
thread Thread-8 is running...
main_thread <_MainThread(MainThread, started 6680)> is running...
Fatal Python error: could not acquire lock for <_io.BufferedWriter name='<stdout>'> at interpreter shutdown, possibly due to daemon threads
Thread 0x00000424 (most recent call first):
File "test.py", line 14 in worker
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 864 in run
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 916 in _bootstrap_inner
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 884 in _bootstrap
Thread 0x00000f04 (most recent call first):
File "test.py", line 14 in worker
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 864 in run
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 916 in _bootstrap_inner
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 884 in _bootstrap
Thread 0x0000165c (most recent call first):
File "test.py", line 13 in worker
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 864 in run
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 916 in _bootstrap_inner
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 884 in _bootstrap
Thread 0x000018b0 (most recent call first):
File "test.py", line 14 in worker
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 864 in run
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 916 in _bootstrap_inner
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 884 in _bootstrap
Thread 0x00000db4 (most recent call first):
File "test.py", line 14 in worker
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 864 in run
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 916 in _bootstrap_inner
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 884 in _bootstrap
Thread 0x00001a88 (most recent call first):
File "test.py", line 14 in worker
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 864 in run
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 916 in _bootstrap_inner
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 884 in _bootstrap
Thread 0x0000111c (most recent call first):
File "test.py", line 14 in worker
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 864 in run
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 916 in _bootstrap_inner
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 884 in _bootstrap
Thread 0x0000177c (most recent call first):
File "test.py", line 14 in worker
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 864 in run
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 916 in _bootstrap_inner
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 884 in _bootstrap
Thread 0x000008e4 (most recent call first):
File "test.py", line 14 in worker
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 864 in run
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 916 in _bootstrap_inner
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 884 in _bootstrap
Thread 0x00001788 (most recent call first):
File "test.py", line 14 in worker
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 864 in run
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 916 in _bootstrap_inner
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\threading.py", line 884 in _bootstrap
Current thread 0x00001a18 (most recent call first):
看主要的幾句
"""到這里主線程已經(jīng)完畢"""
thread end
after to do
"""
這里其實系統(tǒng)在調用守護線程的結束程序來結束所有子線程
"""
"""但是在結束守護進程程序后,又運行了一個不該運行的子線程,這就是為什么會報錯的原因"""
thread Thread-10 is running...
main_thread <_MainThread(MainThread, started 6680)> is running...
end
thread end
先這樣吧,可以自己運行下程序看看
看了另一個答主的,使用了a.join()也是一種選擇
改成發(fā)送10個數(shù)據(jù),用9個線程跑,效率是最高的(理論)
import queue
import threading
import time
aqueue = queue.Queue()
# 發(fā)送了0到9的十個數(shù)據(jù)
for i in range(10):
aqueue.put(i)
def worker():
while True:
# 輸出當前線程,和主線程(可以看到主線程都一樣)
print('thread %s is running...\nmain_thread %s is running...\n\n' % (threading.current_thread().name,threading.main_thread()))
time.sleep(5)
item = aqueue.get()
print(item)
aqueue.task_done()
print('thread end')
n = 1
"""注意這!!!!!!!!!!!!!!!!!!"""
for i in range(9):
print("run thread Counts: ",n)
n += 1
a = threading.Thread(target = worker)
a.setDaemon(True)
a.start()
aqueue.join()
print('after to do')
print('end')
第一個問題 一樣的
第二個問題
1、隊列只有十個值,你開了20個線程,aqueue.get()在取不到值的時候會使線程暫停,后面的線程取不到值程序當然不會退出。
2、while True:也會使線程不停止。
3、aqueue.join()會優(yōu)先于線程執(zhí)行,并阻塞主進程繼續(xù)
如果你是想主進程等待線程結束再停止可以這么寫
import queue
import threading
import time
aqueue = queue.Queue()
for i in range(10):
aqueue.put(i)
def worker():
time.sleep(5)
item = aqueue.get()
print(item)
for i in range(20):
a = threading.Thread(target=worker)
a.setDaemon(True)
a.start()
a.join()
# aqueue.join()import Queue
import threading
import time
aqueue = Queue.Queue()
for i in range(10):
aqueue.put(i)
def worker():
while True:
time.sleep(1)
item = aqueue.get()
print(item)
aqueue.task_done()
for i in range(1):
a = threading.Thread(target = worker)
a.setDaemon(True)
a.start()
aqueue.join()
print 'the main thread is already over'
用 aqueue.task_done() 去通知阻塞的 join 是一種方法
北大青鳥APTECH成立于1999年。依托北京大學優(yōu)質雄厚的教育資源和背景,秉承“教育改變生活”的發(fā)展理念,致力于培養(yǎng)中國IT技能型緊缺人才,是大數(shù)據(jù)專業(yè)的國家
達內教育集團成立于2002年,是一家由留學海歸創(chuàng)辦的高端職業(yè)教育培訓機構,是中國一站式人才培養(yǎng)平臺、一站式人才輸送平臺。2014年4月3日在美國成功上市,融資1
北大課工場是北京大學校辦產(chǎn)業(yè)為響應國家深化產(chǎn)教融合/校企合作的政策,積極推進“中國制造2025”,實現(xiàn)中華民族偉大復興的升級產(chǎn)業(yè)鏈。利用北京大學優(yōu)質教育資源及背
博為峰,中國職業(yè)人才培訓領域的先行者
曾工作于聯(lián)想擔任系統(tǒng)開發(fā)工程師,曾在博彥科技股份有限公司擔任項目經(jīng)理從事移動互聯(lián)網(wǎng)管理及研發(fā)工作,曾創(chuàng)辦藍懿科技有限責任公司從事總經(jīng)理職務負責iOS教學及管理工作。
浪潮集團項目經(jīng)理。精通Java與.NET 技術, 熟練的跨平臺面向對象開發(fā)經(jīng)驗,技術功底深厚。 授課風格 授課風格清新自然、條理清晰、主次分明、重點難點突出、引人入勝。
精通HTML5和CSS3;Javascript及主流js庫,具有快速界面開發(fā)的能力,對瀏覽器兼容性、前端性能優(yōu)化等有深入理解。精通網(wǎng)頁制作和網(wǎng)頁游戲開發(fā)。
具有10 年的Java 企業(yè)應用開發(fā)經(jīng)驗。曾經(jīng)歷任德國Software AG 技術顧問,美國Dachieve 系統(tǒng)架構師,美國AngelEngineers Inc. 系統(tǒng)架構師。