在线观看不卡亚洲电影_亚洲妓女99综合网_91青青青亚洲娱乐在线观看_日韩无码高清综合久久

鍍金池/ 問答/Python  HTML/ 關(guān)于python3爬蟲post json 無法爬取數(shù)據(jù)?

關(guān)于python3爬蟲post json 無法爬取數(shù)據(jù)?

問題描述

python版本為3.6,按照網(wǎng)上的模板來寫該代碼,發(fā)現(xiàn)無法爬取。不知道錯(cuò)在哪里,有大神出來指點(diǎn)一下嗎?

問題出現(xiàn)的環(huán)境背景及自己嘗試過哪些方法

圖片描述圖片描述

相關(guān)代碼

// 請(qǐng)把代碼文本粘貼到下方(請(qǐng)勿用圖片代替代碼)

import urllib.request
import urllib.parse
import json
url = r'XXXXXXX'
headers = {'Accept':'application/json, text/javascript, */*; q=0.01',
    'Accept-Language':'zh-CN,zh;q=0.8',
    'Content-Length':'2',
    'Content-Type':'application/json',
    'Cookie':'XXXXX',
    'Host':'XXXX',
    'Origin':'XXXXX',
    'Proxy-Connection':'keep-alive',
    'Referer':'XXXXXX',
    'User-Agent':'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like             Gecko) Chrome/55.0.2883.87 Safari/537.36'
    }
data = {'requestType':'3',
    'quoteBeginTime':'2018-07-12',
    'quoteEndTime':'2018-07-13',
    'supplierSign':'false',
    'newQuality':'false',
    'newAutoQuote':'false',
    'inquiryTimesDesc':'false',
    'page':'1',
    'pageSize':'20',
    'orderBy':'0',
    'ordered':'false'
    }
data=json.dumps(data)
data=data.encode()
wed_content=urllib.request.Request(url,headers=headers,data=data)
wed_content=urllib.request.urlopen(wed_content,timeout=120)
wed_content=wed_content.read()
wed_content=wed_content.decode('utf-8')
print(wed_content)



你期待的結(jié)果是什么?實(shí)際看到的錯(cuò)誤信息又是什么?


報(bào)錯(cuò)內(nèi)容是urllib.error.HTTPError: HTTP Error 400: Bad Request
期待的結(jié)果是返回JSON的數(shù)據(jù)
第二次報(bào)錯(cuò)內(nèi)容:Traceback (most recent call last):
  File "E:/pg/batulu/4.py", line 31, in <module>
    wed_content=urllib.request.urlopen(wed_content,timeout=120)
  File "C:\Users\hasee\AppData\Local\Programs\Python\Python36\lib\urllib\request.py", line 223, in urlopen
    return opener.open(url, data, timeout)
  File "C:\Users\hasee\AppData\Local\Programs\Python\Python36\lib\urllib\request.py", line 532, in open
    response = meth(req, response)
  File "C:\Users\hasee\AppData\Local\Programs\Python\Python36\lib\urllib\request.py", line 642, in http_response
    'http', request, response, code, msg, hdrs)
  File "C:\Users\hasee\AppData\Local\Programs\Python\Python36\lib\urllib\request.py", line 570, in error
    return self._call_chain(*args)
  File "C:\Users\hasee\AppData\Local\Programs\Python\Python36\lib\urllib\request.py", line 504, in _call_chain
    result = func(*args)
  File "C:\Users\hasee\AppData\Local\Programs\Python\Python36\lib\urllib\request.py", line 650, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
    urllib.error.HTTPError: HTTP Error 400: Bad Request

回答
編輯回答
不歸路

你確定 JSON 數(shù)據(jù)里都是字符串類型嗎?試試用回原生類型:

data = {'requestType':'3',
    'quoteBeginTime':'2018-07-12',
    'quoteEndTime':'2018-07-13',
    'supplierSign': False,
    'newQuality': False,
    'newAutoQuote': False,
    'inquiryTimesDesc': False,
    'page': 1,
    'pageSize': 20,
    'orderBy': 0,
    'ordered': False
    }
2018年7月21日 07:16