在线观看不卡亚洲电影_亚洲妓女99综合网_91青青青亚洲娱乐在线观看_日韩无码高清综合久久

鍍金池/ 問答/人工智能  Python/ scrapy 重試出現(xiàn)問題,采集不出數(shù)據(jù),不知是什么原因?

scrapy 重試出現(xiàn)問題,采集不出數(shù)據(jù),不知是什么原因?

一些數(shù)據(jù)量較大,重試次數(shù)多的網(wǎng)站,采集一部分數(shù)據(jù)后面就會一直重試,采集不出數(shù)據(jù),不知是什么原因

2018-04-25 08:44:42 [scrapy.downloadermiddlewares.retry] DEBUG: Retrying <GET https://www.autozi.com/goods/search.html?carModelId=1711281226033381&_=1524472973719&categoryId=148000000000000&categoryLevel=1> (failed 3 times): User timeout caused connection failure: Getting https://www.autozi.com/goods/search.html?carModelId=1711281226033381&_=1524472973719&categoryId=148000000000000&categoryLevel=1 took longer than 20.0 seconds..
2018-04-25 08:44:42 [scrapy.downloadermiddlewares.retry] DEBUG: Retrying <GET https://www.autozi.com/goods/search.html?carModelId=1711281226033381&_=1524472973719&categoryId=144000000000000&categoryLevel=1> (failed 3 times): User timeout caused connection failure: Getting https://www.autozi.com/goods/search.html?carModelId=1711281226033381&_=1524472973719&categoryId=144000000000000&categoryLevel=1 took longer than 20.0 seconds..
IP列表 ['114.99.25.27:52482', '60.17.232.159:51434', '59.38.241.117:61029', '223.246.157.253:19908', '218.66.144.152:15117']
通過IP 60.17.232.159:51434
2018-04-25 08:44:42 [scrapy.downloadermiddlewares.retry] DEBUG: Retrying <POST https://www.autozi.com/goods/ajaxQueryGoodsList.do> (failed 3 times): User timeout caused connection failure.
IP列表 ['114.99.25.27:52482', '60.17.232.159:51434', '59.38.241.117:61029', '223.246.157.253:19908', '218.66.144.152:15117']
通過IP 59.38.241.117:61029
2018-04-25 08:44:43 [scrapy.downloadermiddlewares.retry] DEBUG: Retrying <GET https://www.autozi.com/goods/search.html?carModelId=1711281226033375&_=1524472973719&categoryId=142000000000000&categoryLevel=1> (failed 3 times): User timeout caused connection failure: Getting https://www.autozi.com/goods/search.html?carModelId=1711281226033375&_=1524472973719&categoryId=142000000000000&categoryLevel=1 took longer than 20.0 seconds..
IP列表 ['114.99.25.27:52482', '60.17.232.159:51434', '59.38.241.117:61029', '223.246.157.253:19908', '218.66.144.152:15117']
通過IP 60.17.232.159:51434
2018-04-25 08:44:43 [scrapy.downloadermiddlewares.retry] DEBUG: Retrying <GET https://www.autozi.com/goods/search.html?carModelId=1711281226033375&_=1524472973719&categoryId=125000000000000&categoryLevel=1> (failed 3 times): User timeout caused connection failure.
IP列表 ['114.99.25.27:52482', '60.17.232.159:51434', '59.38.241.117:61029', '223.246.157.253:19908', '218.66.144.152:15117']
通過IP 59.38.241.117:61029
2018-04-25 08:44:43 [scrapy.downloadermiddlewares.retry] DEBUG: Retrying <GET https://www.autozi.com/goods/search.html?carModelId=1711281226033375&_=1524472973719&categoryId=124000000000000&categoryLevel=1> (failed 3 times): User timeout caused connection failure.
IP列表 ['114.99.25.27:52482', '60.17.232.159:51434', '59.38.241.117:61029', '223.246.157.253:19908', '218.66.144.152:15117']
通過IP 114.99.25.27:52482
2018-04-25 08:44:45 [scrapy.downloadermiddlewares.retry] DEBUG: Retrying <GET https://www.autozi.com/goods/search.html?carModelId=1711281226033375&_=1524472973719&categoryId=140000000000000&categoryLevel=1> (failed 3 times): User timeout caused connection failure.
回答
編輯回答
傻叼

你的IP應該被封了吧

2017年5月2日 07:57