在线观看不卡亚洲电影_亚洲妓女99综合网_91青青青亚洲娱乐在线观看_日韩无码高清综合久久

鍍金池/ 問答/Python/ scrapy piplines拿不到數(shù)據(jù)?

scrapy piplines拿不到數(shù)據(jù)?

從管道中總是拿不到數(shù)據(jù),但是如果直接在執(zhí)行時(shí)使用:

$ scrapy crawl pythonname -o -t 

是可以拿到數(shù)據(jù)的,不知道問題出在哪百度半天無果,請(qǐng)各位指教,代碼如下:

douban.py

# -*- coding: utf-8 -*-
import scrapy
from  text3_scrapydouban.items import Text3Item

class DoubanSpider(scrapy.Spider):
    name = 'douban'
    # allowed_domains = ['https://book.tuicool.com']
    count=0
    urls= 'https://book.douban.com/tag/小說?start='
    start_urls = [ urls + str(count)+ '&type=T']
    
    def parse(self, response):
        items = []
        node_list = response.xpath("http://div[@class='info']")#拿到最外層div
        for node in node_list:
            item = Text3Item() # itemd 的類用來存儲(chǔ)信息            
            # extract 將 Xpath 對(duì)象轉(zhuǎn)換為 Unicode方便解讀
            bookname = node.xpath("./h2/a/text()").extract()[0].strip() # rstrip()將右邊空格去掉
            title = node.xpath("./div[1]/text()").extract()[0].split("/")
            author = title[0].strip()
            publishTime = title[-2]
            bookprice = title[-1].strip()
            grade = node.xpath("./div[2]/span[2]/text()").extract()[0] # 等級(jí)
            evaluate = node.xpath("./p/text()").extract()[0]
            item['bookname'] = bookname
            item["author"] = author
            item["time"] = publishTime
            item["price"] = bookprice
            a = "分"
            item["grade"] = grade
            item["evaluate"] = evaluate
            items.append(item)
        return items # 引擎自動(dòng)判斷此為item數(shù)據(jù),使用會(huì)自動(dòng)交給管道

piplines.py

class Text3ScrapydoubanPipeline(object):
    def __init__(self):
        self.file = open('dou.txt', 'wb')

    def process_item(self, item, spider):
        self.file.write(item)
        return item

    def spider_closed(self, spider):
        self.file.close()

items.py

import scrapy

class Text3Item(scrapy.Item):
    # define the fields for your item here like:
    # name = scrapy.Field()
    bookname = scrapy.Field()
    author = scrapy.Field()
    time = scrapy.Field()
    price = scrapy.Field()
    grade = scrapy.Field()    # 豆瓣口碑
    evaluate = scrapy.Field() # 簡述

settings 設(shè)置是正確的

運(yùn)行結(jié)果:

F:\GitHubCode\Code\Python\text3_scrapydouban>scrapy crawl douban
2018-02-03 14:39:49 [scrapy.utils.log] INFO: Scrapy 1.5.0 started (bot: text3_scrapydouban)
2018-02-03 14:39:49 [scrapy.utils.log] INFO: Versions: lxml 4.1.1.0, libxml2 2.9.5, cssselect 1.0.3, parsel 1.3.1, w3lib 1.19.0, Twisted 17.9.0, Python 3.6.4 (v3.6.4:d48eceb, Dec 19 2017, 06:54:40) [MSC v.1900 64 bit (AMD64)], pyOpenSSL 17.5.0 (OpenSSL 1.1.0g  2 Nov 2017), cryptography 2.1.4, Platform Windows-10-10.0.14393-SP0
2018-02-03 14:39:49 [scrapy.crawler] INFO: Overridden settings: {'BOT_NAME': 'text3_scrapydouban', 'NEWSPIDER_MODULE': 'text3_scrapydouban.spiders', 'SPIDER_MODULES': ['text3_scrapydouban.spiders'], 'USER_AGENT': 'Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; AcooBrowser; .NET CLR 1.1.4322; .NET CLR 2.0.50727)'}
2018-02-03 14:39:49 [scrapy.middleware] INFO: Enabled extensions:
['scrapy.extensions.corestats.CoreStats',
 'scrapy.extensions.telnet.TelnetConsole',
 'scrapy.extensions.logstats.LogStats']
2018-02-03 14:39:49 [scrapy.middleware] INFO: Enabled downloader middlewares:
['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
 'scrapy.downloadermiddlewares.retry.RetryMiddleware',
 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
 'scrapy.downloadermiddlewares.stats.DownloaderStats']
2018-02-03 14:39:49 [scrapy.middleware] INFO: Enabled spider middlewares:
['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
 'scrapy.spidermiddlewares.referer.RefererMiddleware',
 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
 'scrapy.spidermiddlewares.depth.DepthMiddleware']
2018-02-03 14:39:49 [scrapy.middleware] INFO: Enabled item pipelines:
['text3_scrapydouban.pipelines.Text3ScrapydoubanPipeline']
2018-02-03 14:39:49 [scrapy.core.engine] INFO: Spider opened
2018-02-03 14:39:49 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2018-02-03 14:39:49 [scrapy.extensions.telnet] DEBUG: Telnet console listening on 127.0.0.1:6023
2018-02-03 14:39:50 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://book.douban.com/tag/%E5%B0%8F%E8%AF%B4?start=0&type=T> (referer: None)
2018-02-03 14:39:50 [scrapy.core.scraper] ERROR: Error processing {'author': '[日] 東野圭吾',
 'bookname': '解憂雜貨店',
 'evaluate': '現(xiàn)代人內(nèi)心流失的東西,這家雜貨店能幫你找回——\n'
             '僻靜的街道旁有一家雜貨店,只要寫下煩惱投進(jìn)卷簾門的投信口,第二天就會(huì)在店后的牛奶箱里得到回答。\n'
             '因男友身患絕... ',
 'grade': '8.6',
 'price': '39.50元',
 'time': ' 2014-5 '}
Traceback (most recent call last):
  File "e:\software\python36\lib\site-packages\twisted\internet\defer.py", line 653, in _runCallbacks
    current.result = callback(current.result, *args, **kw)
  File "F:\GitHubCode\Code\Python\text3_scrapydouban\text3_scrapydouban\pipelines.py", line 16, in process_item
    self.file.write(item)
TypeError: a bytes-like object is required, not 'Text3Item'
2018-02-03 14:39:50 [scrapy.core.scraper] ERROR: Error processing {'author': '[英] 肯·福萊特',
 'bookname': '巨人的隕落',
 'evaluate': '在第一次世界大戰(zhàn)的硝煙中,每一個(gè)邁向死亡的生命都在熱烈地生長——威爾士的礦工少年、剛失戀的美國法律系大學(xué)生、窮困潦倒的俄國兄弟、富有英俊的英格蘭伯爵,以及癡... ',
 'grade': '8.9',
 'price': '129.80元',
 'time': ' 2016-5-1 '}
Traceback (most recent call last):
  File "e:\software\python36\lib\site-packages\twisted\internet\defer.py", line 653, in _runCallbacks
    current.result = callback(current.result, *args, **kw)
  File "F:\GitHubCode\Code\Python\text3_scrapydouban\text3_scrapydouban\pipelines.py", line 16, in process_item
    self.file.write(item)
TypeError: a bytes-like object is required, not 'Text3Item'
2018-02-03 14:39:50 [scrapy.core.scraper] ERROR: Error processing {'author': '亦舒',
 'bookname': '我的前半生',
 'evaluate': '一個(gè)三十幾歲的美麗女人子君,在家做全職家庭主婦。卻被一個(gè)平凡女人奪走丈夫,一段婚姻的失敗,讓女主角不得不堅(jiān)強(qiáng),變得更美麗,有了事業(yè),最終遇見一個(gè)更值得愛的男... ',
 'grade': '7.9',
 'price': '22.00元',
 'time': ' 2007-8 '}
Traceback (most recent call last):
  File "e:\software\python36\lib\site-packages\twisted\internet\defer.py", line 653, in _runCallbacks
    current.result = callback(current.result, *args, **kw)
  File "F:\GitHubCode\Code\Python\text3_scrapydouban\text3_scrapydouban\pipelines.py", line 16, in process_item
    self.file.write(item)
TypeError: a bytes-like object is required, not 'Text3Item'
2018-02-03 14:39:50 [scrapy.core.scraper] ERROR: Error processing {'author': '[哥倫比亞] 加西亞·馬爾克斯',
 'bookname': '百年孤獨(dú)',
 'evaluate': '《百年孤獨(dú)》是魔幻現(xiàn)實(shí)主義文學(xué)的代表作,描寫了布恩迪亞家族七代人的傳奇故事,以及加勒比海沿岸小鎮(zhèn)馬孔多的百年興衰,反映了拉丁美洲一個(gè)世紀(jì)以來風(fēng)云變幻的歷史。... ',
 'grade': '9.2',
 'price': '39.50元',
 'time': ' 2011-6 '}
Traceback (most recent call last):
  File "e:\software\python36\lib\site-packages\twisted\internet\defer.py", line 653, in _runCallbacks
    current.result = callback(current.result, *args, **kw)
  File "F:\GitHubCode\Code\Python\text3_scrapydouban\text3_scrapydouban\pipelines.py", line 16, in process_item
    self.file.write(item)
TypeError: a bytes-like object is required, not 'Text3Item'
2018-02-03 14:39:50 [scrapy.core.scraper] ERROR: Error processing {'author': '[美] 卡勒德·胡賽尼',
 'bookname': '追風(fēng)箏的人',
 'evaluate': '12歲的阿富汗富家少爺阿米爾與仆人哈桑情同手足。然而,在一場風(fēng)箏比賽后,發(fā)生了一件悲慘不堪的事,阿米爾為自己的懦弱感到自責(zé)和痛苦,逼走了哈桑,不久,自己也跟... ',
 'grade': '8.9',
 'price': '29.00元',
 'time': ' 2006-5 '}
Traceback (most recent call last):
  File "e:\software\python36\lib\site-packages\twisted\internet\defer.py", line 653, in _runCallbacks
    current.result = callback(current.result, *args, **kw)
  File "F:\GitHubCode\Code\Python\text3_scrapydouban\text3_scrapydouban\pipelines.py", line 16, in process_item
    self.file.write(item)
TypeError: a bytes-like object is required, not 'Text3Item'
2018-02-03 14:39:50 [scrapy.core.scraper] ERROR: Error processing {'author': '林奕含',
 'bookname': '房思琪的初戀樂園',
 'evaluate': '令人心碎卻無能為力的真實(shí)故事。\n'
             '性、權(quán)力、升學(xué)主義──青澀的表皮、變態(tài)社會(huì)的日常\n'
             '如果這件事情正在發(fā)生,我們要怎麼假裝世界上沒有人以強(qiáng)暴小女孩為樂?\n'
             '「我下... ',
 'grade': '8.8',
 'price': 'NTD320',
 'time': ' 2017-2-7 '}
Traceback (most recent call last):
  File "e:\software\python36\lib\site-packages\twisted\internet\defer.py", line 653, in _runCallbacks
    current.result = callback(current.result, *args, **kw)
  File "F:\GitHubCode\Code\Python\text3_scrapydouban\text3_scrapydouban\pipelines.py", line 16, in process_item
    self.file.write(item)
TypeError: a bytes-like object is required, not 'Text3Item'
2018-02-03 14:39:50 [scrapy.core.scraper] ERROR: Error processing {'author': '[意] 埃萊娜·費(fèi)蘭特',
 'bookname': '新名字的故事',
 'evaluate': '只有你身為女人 才會(huì)知道這些丑陋的秘密\n'
             '兩個(gè)女人 50年的友誼和戰(zhàn)爭\n'
             '如何成為那個(gè)更強(qiáng)大的她,又不被她戰(zhàn)勝\n'
             '那不勒斯四部曲NO.2\n'
             '《我的天才女友》后續(xù), ... ',
 'grade': '9.1',
 'price': '59.00元',
 'time': ' 2017-4 '}
Traceback (most recent call last):
  File "e:\software\python36\lib\site-packages\twisted\internet\defer.py", line 653, in _runCallbacks
    current.result = callback(current.result, *args, **kw)
  File "F:\GitHubCode\Code\Python\text3_scrapydouban\text3_scrapydouban\pipelines.py", line 16, in process_item
    self.file.write(item)
TypeError: a bytes-like object is required, not 'Text3Item'
2018-02-03 14:39:50 [scrapy.core.scraper] ERROR: Error processing {'author': '[美] 戴維·伽特森',
 'bookname': '雪落香杉樹',
 'evaluate': '編輯推薦:\n'
             '1995福克納獎(jiǎng)得主\n'
             '占據(jù)《紐約時(shí)報(bào)》暢銷榜首數(shù)月\n'
             '美國書商協(xié)會(huì)年度之選\n'
             '被譯成30余種文字,全球暢銷500萬冊(cè)\n'
             '當(dāng)代美國文學(xué)史上罕見的超級(jí)暢銷... ',
 'grade': '8.5',
 'price': '52.00元',
 'time': ' 2017-6 '}
Traceback (most recent call last):
  File "e:\software\python36\lib\site-packages\twisted\internet\defer.py", line 653, in _runCallbacks
    current.result = callback(current.result, *args, **kw)
  File "F:\GitHubCode\Code\Python\text3_scrapydouban\text3_scrapydouban\pipelines.py", line 16, in process_item
    self.file.write(item)
TypeError: a bytes-like object is required, not 'Text3Item'
2018-02-03 14:39:50 [scrapy.core.scraper] ERROR: Error processing {'author': '[哥倫比亞] 加西亞·馬爾克斯',
 'bookname': '霍亂時(shí)期的愛情',
 'evaluate': '★馬爾克斯唯一正式授權(quán),首次完整翻譯\n'
             '★《霍亂時(shí)期的愛情》是我最好的作品,是我發(fā)自內(nèi)心的創(chuàng)作。——加西亞?馬爾克斯\n'
             '★這部光芒閃耀、令人心碎的作品是人類有史... ',
 'grade': '9.0',
 'price': '39.50元',
 'time': ' 2012-9-1 '}
Traceback (most recent call last):
  File "e:\software\python36\lib\site-packages\twisted\internet\defer.py", line 653, in _runCallbacks
    current.result = callback(current.result, *args, **kw)
  File "F:\GitHubCode\Code\Python\text3_scrapydouban\text3_scrapydouban\pipelines.py", line 16, in process_item
    self.file.write(item)
TypeError: a bytes-like object is required, not 'Text3Item'
2018-02-03 14:39:50 [scrapy.core.scraper] ERROR: Error processing {'author': '[俄] 維克托·阿斯塔菲耶夫',
 'bookname': '魚王',
 'evaluate': '長篇小說《魚王》是阿斯塔菲耶夫最具個(gè)性的一部代表作,俄羅斯當(dāng)代文學(xué)的經(jīng)典。全書由十三個(gè)內(nèi)容相對(duì)獨(dú)立的“敘事短篇小說”組成,全部圍繞著人與自然的關(guān)系,深入細(xì)致... ',
 'grade': '9.0',
 'price': '78.00元',
 'time': ' 2017-4 '}
Traceback (most recent call last):
  File "e:\software\python36\lib\site-packages\twisted\internet\defer.py", line 653, in _runCallbacks
    current.result = callback(current.result, *args, **kw)
  File "F:\GitHubCode\Code\Python\text3_scrapydouban\text3_scrapydouban\pipelines.py", line 16, in process_item
    self.file.write(item)
TypeError: a bytes-like object is required, not 'Text3Item'
2018-02-03 14:39:50 [scrapy.core.scraper] ERROR: Error processing {'author': '[英] 喬治·奧威爾',
 'bookname': '1984',
 'evaluate': '★村上春樹以《1Q84》向本書致敬\n'
             '★著名學(xué)者劉紹銘經(jīng)典譯本內(nèi)地首次出版\n'
             '★62種文字風(fēng)靡110個(gè)國家,全球銷量超過5000萬冊(cè)\n'
             '★《時(shí)代周刊》“最好的10... ',
 'grade': '9.3',
 'price': '28.00',
 'time': ' 2010-4-1 '}
Traceback (most recent call last):
  File "e:\software\python36\lib\site-packages\twisted\internet\defer.py", line 653, in _runCallbacks
    current.result = callback(current.result, *args, **kw)
  File "F:\GitHubCode\Code\Python\text3_scrapydouban\text3_scrapydouban\pipelines.py", line 16, in process_item
    self.file.write(item)
TypeError: a bytes-like object is required, not 'Text3Item'
2018-02-03 14:39:50 [scrapy.core.scraper] ERROR: Error processing {'author': '[英] 肯·福萊特',
 'bookname': '世界的凜冬',
 'evaluate': '我親眼目睹,每一個(gè)邁向死亡的生命都在熱烈地生長。\n'
             '全球讀者平均3個(gè)通宵讀完的超級(jí)小說《巨人的隕落》的續(xù)篇!\n'
             '火遍全球的20世紀(jì)人類史詩“世紀(jì)三部曲”第二部。... ',
 'grade': '9.0',
 'price': '132.00(全三冊(cè))',
 'time': ' 2017-3-1 '}
Traceback (most recent call last):
  File "e:\software\python36\lib\site-packages\twisted\internet\defer.py", line 653, in _runCallbacks
    current.result = callback(current.result, *args, **kw)
  File "F:\GitHubCode\Code\Python\text3_scrapydouban\text3_scrapydouban\pipelines.py", line 16, in process_item
    self.file.write(item)
TypeError: a bytes-like object is required, not 'Text3Item'
2018-02-03 14:39:50 [scrapy.core.scraper] ERROR: Error processing {'author': '[英] 毛姆',
 'bookname': '月亮和六便士',
 'evaluate': '一個(gè)英國證券交易所的經(jīng)紀(jì)人,本已有牢靠的職業(yè)和地位、美滿的家庭,但卻迷戀上繪畫,像“被魔鬼附了體”,突然棄家出走,到巴黎去追求繪畫的理想。他的行徑?jīng)]有人能夠... ',
 'grade': '9.0',
 'price': '15.00元',
 'time': ' 2006-8 '}
Traceback (most recent call last):
  File "e:\software\python36\lib\site-packages\twisted\internet\defer.py", line 653, in _runCallbacks
    current.result = callback(current.result, *args, **kw)
  File "F:\GitHubCode\Code\Python\text3_scrapydouban\text3_scrapydouban\pipelines.py", line 16, in process_item
    self.file.write(item)
TypeError: a bytes-like object is required, not 'Text3Item'
2018-02-03 14:39:50 [scrapy.core.scraper] ERROR: Error processing {'author': '余華',
 'bookname': '活著',
 'evaluate': '地主少爺福貴嗜賭成性,終于賭光了家業(yè)一貧如洗,窮困之中的福貴因?yàn)槟赣H生病前去求醫(yī),沒想到半路上被國民黨部隊(duì)抓了壯丁,后被解放軍所俘虜,回到家鄉(xiāng)他才知道母親已... ',
 'grade': '9.1',
 'price': '12.00元',
 'time': ' 1998-5 '}
Traceback (most recent call last):
  File "e:\software\python36\lib\site-packages\twisted\internet\defer.py", line 653, in _runCallbacks
    current.result = callback(current.result, *args, **kw)
  File "F:\GitHubCode\Code\Python\text3_scrapydouban\text3_scrapydouban\pipelines.py", line 16, in process_item
    self.file.write(item)
TypeError: a bytes-like object is required, not 'Text3Item'
2018-02-03 14:39:50 [scrapy.core.scraper] ERROR: Error processing {'author': '[美] 哈珀·李',
 'bookname': '殺死一只知更鳥',
 'evaluate': '成長總是個(gè)讓人煩惱的命題。成長有時(shí)會(huì)很緩慢,如小溪般唱著叮咚的歌曲趟過,有時(shí)卻如此突如其來,如暴雨般劈頭蓋臉……三個(gè)孩子因?yàn)樾℃?zhèn)上的幾樁冤案經(jīng)歷了猝不及防的... ',
 'grade': '9.2',
 'price': '32.00元',
 'time': ' 2012-9 '}
Traceback (most recent call last):
  File "e:\software\python36\lib\site-packages\twisted\internet\defer.py", line 653, in _runCallbacks
    current.result = callback(current.result, *args, **kw)
  File "F:\GitHubCode\Code\Python\text3_scrapydouban\text3_scrapydouban\pipelines.py", line 16, in process_item
    self.file.write(item)
TypeError: a bytes-like object is required, not 'Text3Item'
2018-02-03 14:39:50 [scrapy.core.scraper] ERROR: Error processing {'author': '錢鍾書',
 'bookname': '圍城',
 'evaluate': '《圍城》是錢鐘書所著的長篇小說。第一版于1947年由上海晨光出版公司出版。1949年之后,由于政治等方面的原因,本書長期無法在中國大陸和臺(tái)灣重印,僅在香港出... ',
 'grade': '8.9',
 'price': '19.00',
 'time': ' 1991-2 '}
Traceback (most recent call last):
  File "e:\software\python36\lib\site-packages\twisted\internet\defer.py", line 653, in _runCallbacks
    current.result = callback(current.result, *args, **kw)
  File "F:\GitHubCode\Code\Python\text3_scrapydouban\text3_scrapydouban\pipelines.py", line 16, in process_item
    self.file.write(item)
TypeError: a bytes-like object is required, not 'Text3Item'
2018-02-03 14:39:50 [scrapy.core.scraper] ERROR: Error processing {'author': '[美] 庫爾特·馮內(nèi)古特',
 'bookname': '囚鳥',
 'evaluate': '我們都是受困于時(shí)代的“囚鳥”——既渴望逃離,又踟躕不前。\n'
             '聯(lián)邦最低限度安保措施成人改造所里,頹唐的小老頭兒瓦爾特?斯代布克正在等待領(lǐng)他出獄的獄卒。\n'
             '在他過去... ',
 'grade': '8.0',
 'price': '38.00元',
 'time': ' 2017-6 '}
Traceback (most recent call last):
  File "e:\software\python36\lib\site-packages\twisted\internet\defer.py", line 653, in _runCallbacks
    current.result = callback(current.result, *args, **kw)
  File "F:\GitHubCode\Code\Python\text3_scrapydouban\text3_scrapydouban\pipelines.py", line 16, in process_item
    self.file.write(item)
TypeError: a bytes-like object is required, not 'Text3Item'
2018-02-03 14:39:50 [scrapy.core.scraper] ERROR: Error processing {'author': '[美] 約翰·威廉斯',
 'bookname': '斯通納',
 'evaluate': '《斯通納》講述了生命中最重要的部分:愛,認(rèn)同,憐憫,志業(yè),傲骨,信任與死亡。\n'
             '一個(gè)勇者有過的失敗不失意的人生:即使不能擁有完美的生活,所幸追求過完整的自我。... ',
 'grade': '8.8',
 'price': '39.00元',
 'time': ' 2016-1 '}
Traceback (most recent call last):
  File "e:\software\python36\lib\site-packages\twisted\internet\defer.py", line 653, in _runCallbacks
    current.result = callback(current.result, *args, **kw)
  File "F:\GitHubCode\Code\Python\text3_scrapydouban\text3_scrapydouban\pipelines.py", line 16, in process_item
    self.file.write(item)
TypeError: a bytes-like object is required, not 'Text3Item'
2018-02-03 14:39:50 [scrapy.core.scraper] ERROR: Error processing {'author': '[意] 埃萊娜·費(fèi)蘭特',
 'bookname': '我的天才女友',
 'evaluate': '只有你身為女人才會(huì)知道這些丑陋的秘密\n'
             '兩個(gè)女人,50年的友誼和戰(zhàn)爭\n'
             '過去五年,幾乎所有歐美讀者都在談?wù)撍?、\n'
             '全球暢銷百萬冊(cè) 被翻譯成40種語言\n'
             '《金融時(shí)報(bào)》... ',
 'grade': '8.5',
 'price': '42.00元',
 'time': ' 2017-1 '}
Traceback (most recent call last):
  File "e:\software\python36\lib\site-packages\twisted\internet\defer.py", line 653, in _runCallbacks
    current.result = callback(current.result, *args, **kw)
  File "F:\GitHubCode\Code\Python\text3_scrapydouban\text3_scrapydouban\pipelines.py", line 16, in process_item
    self.file.write(item)
TypeError: a bytes-like object is required, not 'Text3Item'
2018-02-03 14:39:50 [scrapy.core.scraper] ERROR: Error processing {'author': '路遙',
 'bookname': '平凡的世界(全三部)',
 'evaluate': '《平凡的世界》是一部現(xiàn)實(shí)主義小說,也是一部小說形式的家族史。作者濃縮了中國西北農(nóng)村的歷史變遷過程,在小說中全景式地表現(xiàn)了中國當(dāng)代城鄉(xiāng)的社會(huì)生活。在近十年的廣... ',
 'grade': '9.0',
 'price': '64.00元',
 'time': ' 2005-1 '}
Traceback (most recent call last):
  File "e:\software\python36\lib\site-packages\twisted\internet\defer.py", line 653, in _runCallbacks
    current.result = callback(current.result, *args, **kw)
  File "F:\GitHubCode\Code\Python\text3_scrapydouban\text3_scrapydouban\pipelines.py", line 16, in process_item
    self.file.write(item)
TypeError: a bytes-like object is required, not 'Text3Item'
2018-02-03 14:39:50 [scrapy.core.engine] INFO: Closing spider (finished)
2018-02-03 14:39:50 [scrapy.statscollectors] INFO: Dumping Scrapy stats:
{'downloader/request_bytes': 324,
 'downloader/request_count': 1,
 'downloader/request_method_count/GET': 1,
 'downloader/response_bytes': 13916,
 'downloader/response_count': 1,
 'downloader/response_status_count/200': 1,
 'finish_reason': 'finished',
 'finish_time': datetime.datetime(2018, 2, 3, 6, 39, 50, 457633),
 'log_count/DEBUG': 2,
 'log_count/ERROR': 20,
 'log_count/INFO': 7,
 'response_received_count': 1,
 'scheduler/dequeued': 1,
 'scheduler/dequeued/memory': 1,
 'scheduler/enqueued': 1,
 'scheduler/enqueued/memory': 1,
 'start_time': datetime.datetime(2018, 2, 3, 6, 39, 49, 703712)}
2018-02-03 14:39:50 [scrapy.core.engine] INFO: Spider closed (finished)
回答
編輯回答
苦妄

報(bào)了這個(gè)異常

TypeError: a bytes-like object is required, not 'Text3Item'

主要是這一行報(bào)錯(cuò):

self.file.write(item)

item 類型要求是 bytes

可以改成這樣:

self.file.write(json.dumps(item))

主要代碼:

import json


class Text3ScrapydoubanPipeline(object):
    def __init__(self):
        self.file = open('dou.txt', 'wb')

    def process_item(self, item, spider):
        self.file.write(json.dumps(item))
        return item

    def spider_closed(self, spider):
        self.file.close()
2017年6月19日 12:40