網(wǎng)頁時html，但是返回的結(jié)果是xml，且未返回內(nèi)容

import re
import random
import requests

rendom_num = random.random()
post_url = 'http://navi.cnki.net/knavi/Common/Search/NPaper'
pages = 1

post_data = {
    'SearchStateJson':{"StateID":"","Platfrom":"","QueryTime":"","Account":"knavi","ClientToken":"","Language":"","CNode":{"PCode":"CCND","SMode":"","OperateT":""},"QNode":{"SelectT":"","Select_Fields":"","S_DBCodes":"","QGroup":[{"Key":"Navi","Logic":1,"Items":[],"ChildItems":[{"Key":"NPaper","Logic":1,"Items":[{"Key":1,"Title":"","Logic":1,"Name":"地區(qū)代碼","Operate":"","Value":"0001?","ExtendType":0,"ExtendValue":"","Value2":""}],"ChildItems":[]}]}],"OrderBy":"RT|","GroupBy":"","Additon":""}},
    'displaymode':1,
    'pageindex':4,
    'pagecount':21,
    'index':pages,
    'random':rendom_num
}

header = {
    'Cookie':'UM_distinctid=160cf4141da790-0ba2bdda54b9fd-454f032b-ff000-160cf4141db7f0; Ecp_IpLoginFail=18010761.190.32.52; cnkiUserKey=0a9ae521-a984-c34f-9439-d2d12fa85751; ASP.NET_SessionId=133ge3vm32yrtztdrvlyycb1; SID_navi=120161; KNET_SSO_COOKIE_CHECK=2018-01-07 15:23:59; LID=admin$9A4hF_YAuvQ5obgVAqNKPCYcEjKensW4ggI8Fm4gTkoUKaID8j8gFw!!',
    'Referer':'http://navi.cnki.net/KNavi/NPaper.html',
    'User-Agent':'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.108 Safari/537.36',
    'X-Requested-With':'XMLHttpRequest'
}

content = requests.post(url=post_url, data=post_data, headers=header).text
print(content)

以上，返回結(jié)果是 <?xml version="1.0" encoding="utf-16"?> 我看了請求頭信息，網(wǎng)頁編碼是 text/html
但是返回結(jié)果為啥是 xml 的呢？還就返回這一句話，其它啥都沒有，會不會是隨機值的原因？

回答

編輯回答

囍槑

你這個問題在post_data里面

這一塊的數(shù)據(jù)應(yīng)該寫成字符串格式,否則requests對post_data數(shù)據(jù)進行解析的時候會出偏差
或者你可以用data = urllib.parse.urlencode(post_data)這樣預(yù)處理一下再post

2017年11月29日 15:15

編輯回答

吃藕丑

給你一個例子：

post = {'source': 'index_nav',
     'form_password': 'your password',
     'form_email': 'your username'
     }
post_data = urllib.parse.urlencode(post).encode('utf8')

post編碼的規(guī)則需要遵從RFC標準，而urlencode就是干這個的，把你post的數(shù)據(jù)轉(zhuǎn)化成標準格式。但是進入urlencode之前post也有相應(yīng)的數(shù)據(jù)類型，可以是字典或者是元組。

具體POST如何使用可參考Python爬蟲之urllib庫—進階篇

2017年3月30日 01:19