一些數(shù)據(jù)量較大,重試次數(shù)多的網(wǎng)站,采集一部分數(shù)據(jù)后面就會一直重試,采集不出數(shù)據(jù),不知是什么原因
2018-04-25 08:44:42 [scrapy.downloadermiddlewares.retry] DEBUG: Retrying <GET https://www.autozi.com/goods/search.html?carModelId=1711281226033381&_=1524472973719&categoryId=148000000000000&categoryLevel=1> (failed 3 times): User timeout caused connection failure: Getting https://www.autozi.com/goods/search.html?carModelId=1711281226033381&_=1524472973719&categoryId=148000000000000&categoryLevel=1 took longer than 20.0 seconds..
2018-04-25 08:44:42 [scrapy.downloadermiddlewares.retry] DEBUG: Retrying <GET https://www.autozi.com/goods/search.html?carModelId=1711281226033381&_=1524472973719&categoryId=144000000000000&categoryLevel=1> (failed 3 times): User timeout caused connection failure: Getting https://www.autozi.com/goods/search.html?carModelId=1711281226033381&_=1524472973719&categoryId=144000000000000&categoryLevel=1 took longer than 20.0 seconds..
IP列表 ['114.99.25.27:52482', '60.17.232.159:51434', '59.38.241.117:61029', '223.246.157.253:19908', '218.66.144.152:15117']
通過IP 60.17.232.159:51434
2018-04-25 08:44:42 [scrapy.downloadermiddlewares.retry] DEBUG: Retrying <POST https://www.autozi.com/goods/ajaxQueryGoodsList.do> (failed 3 times): User timeout caused connection failure.
IP列表 ['114.99.25.27:52482', '60.17.232.159:51434', '59.38.241.117:61029', '223.246.157.253:19908', '218.66.144.152:15117']
通過IP 59.38.241.117:61029
2018-04-25 08:44:43 [scrapy.downloadermiddlewares.retry] DEBUG: Retrying <GET https://www.autozi.com/goods/search.html?carModelId=1711281226033375&_=1524472973719&categoryId=142000000000000&categoryLevel=1> (failed 3 times): User timeout caused connection failure: Getting https://www.autozi.com/goods/search.html?carModelId=1711281226033375&_=1524472973719&categoryId=142000000000000&categoryLevel=1 took longer than 20.0 seconds..
IP列表 ['114.99.25.27:52482', '60.17.232.159:51434', '59.38.241.117:61029', '223.246.157.253:19908', '218.66.144.152:15117']
通過IP 60.17.232.159:51434
2018-04-25 08:44:43 [scrapy.downloadermiddlewares.retry] DEBUG: Retrying <GET https://www.autozi.com/goods/search.html?carModelId=1711281226033375&_=1524472973719&categoryId=125000000000000&categoryLevel=1> (failed 3 times): User timeout caused connection failure.
IP列表 ['114.99.25.27:52482', '60.17.232.159:51434', '59.38.241.117:61029', '223.246.157.253:19908', '218.66.144.152:15117']
通過IP 59.38.241.117:61029
2018-04-25 08:44:43 [scrapy.downloadermiddlewares.retry] DEBUG: Retrying <GET https://www.autozi.com/goods/search.html?carModelId=1711281226033375&_=1524472973719&categoryId=124000000000000&categoryLevel=1> (failed 3 times): User timeout caused connection failure.
IP列表 ['114.99.25.27:52482', '60.17.232.159:51434', '59.38.241.117:61029', '223.246.157.253:19908', '218.66.144.152:15117']
通過IP 114.99.25.27:52482
2018-04-25 08:44:45 [scrapy.downloadermiddlewares.retry] DEBUG: Retrying <GET https://www.autozi.com/goods/search.html?carModelId=1711281226033375&_=1524472973719&categoryId=140000000000000&categoryLevel=1> (failed 3 times): User timeout caused connection failure.北大青鳥APTECH成立于1999年。依托北京大學優(yōu)質(zhì)雄厚的教育資源和背景,秉承“教育改變生活”的發(fā)展理念,致力于培養(yǎng)中國IT技能型緊缺人才,是大數(shù)據(jù)專業(yè)的國家
達內(nèi)教育集團成立于2002年,是一家由留學海歸創(chuàng)辦的高端職業(yè)教育培訓機構(gòu),是中國一站式人才培養(yǎng)平臺、一站式人才輸送平臺。2014年4月3日在美國成功上市,融資1
北大課工場是北京大學校辦產(chǎn)業(yè)為響應國家深化產(chǎn)教融合/校企合作的政策,積極推進“中國制造2025”,實現(xiàn)中華民族偉大復興的升級產(chǎn)業(yè)鏈。利用北京大學優(yōu)質(zhì)教育資源及背
博為峰,中國職業(yè)人才培訓領(lǐng)域的先行者
曾工作于聯(lián)想擔任系統(tǒng)開發(fā)工程師,曾在博彥科技股份有限公司擔任項目經(jīng)理從事移動互聯(lián)網(wǎng)管理及研發(fā)工作,曾創(chuàng)辦藍懿科技有限責任公司從事總經(jīng)理職務負責iOS教學及管理工作。
浪潮集團項目經(jīng)理。精通Java與.NET 技術(shù), 熟練的跨平臺面向?qū)ο箝_發(fā)經(jīng)驗,技術(shù)功底深厚。 授課風格 授課風格清新自然、條理清晰、主次分明、重點難點突出、引人入勝。
精通HTML5和CSS3;Javascript及主流js庫,具有快速界面開發(fā)的能力,對瀏覽器兼容性、前端性能優(yōu)化等有深入理解。精通網(wǎng)頁制作和網(wǎng)頁游戲開發(fā)。
具有10 年的Java 企業(yè)應用開發(fā)經(jīng)驗。曾經(jīng)歷任德國Software AG 技術(shù)顧問,美國Dachieve 系統(tǒng)架構(gòu)師,美國AngelEngineers Inc. 系統(tǒng)架構(gòu)師。