1.通過對知網(wǎng)期刊的鏈接去解析包含內(nèi)容的詳情頁鏈接,但是知網(wǎng)不存在頁面狀態(tài)碼依舊是 200, 所以轉(zhuǎn)而想用網(wǎng)頁存在某些內(nèi)容來決定取舍,但是判斷條件好像對知網(wǎng)內(nèi)容不起作用,起不到篩查的效果.

2.完整代碼:

import requests
from bs4 import BeautifulSoup
import time

def get_url(years,month,num):#month用兩位數(shù)表示，num用三位數(shù)表示。
    times = 'GJXW'+str(years)+str(month).zfill(2)+str(num).zfill(3)
    url='http://kns.cnki.net/kcms/detail/detail.aspx?dbcode=CJFD&filename={}'.format(times)

    headers={
        'User-Agent':'Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.104 Safari/537.36 Core/1.53.4033.400 QQBrowser/9.6.12624.400',
        'Cookie':'Ecp_notFirstLogin=ezBGYf; SID=120162; ASP.NET_SessionId=pj040hrhxe43fsrsa2mk24ep; Ecp_ClientId=3171112155604138976; c_m_LinID=LinID=WEEvREcwSlJHSldRa1Fhb09jMjVzQmVYUXBzNTNlbzJreWFVTU5NeVRmST0=$9A4hF_YAuvQ5obgVAqNKPCYcEjKensW4IQMovwHtwkF4VYPoHbKxJw!!&ot=11/12/2017 16:16:52; c_m_expire=2017-11-12 16:16:52; Ecp_session=1; LID=WEEvREcwSlJHSldRa1Fhb09jMjVzQmVYUXBzNTNlbzJreWFVTU5NeVRmST0=$9A4hF_YAuvQ5obgVAqNKPCYcEjKensW4IQMovwHtwkF4VYPoHbKxJw!!; Ecp_LoginStuts=%7B%22IsAutoLogin%22%3Afalse%2C%22UserName%22%3A%22gz0289%22%2C%22ShowName%22%3A%22%25E5%2590%2589%25E6%259E%2597%25E5%25A4%25A7%25E5%25AD%25A6%25E7%258F%25A0%25E6%25B5%25B7%25E5%25AD%25A6%25E9%2599%25A2%22%2C%22UserType%22%3A%22bk%22%2C%22r%22%3A%22ezBGYf%22%7D'
    }
    html=requests.get(url,headers=headers)
    soup = BeautifulSoup(html.text,'lxml')
    if soup.find('body > div:nth-child(1) > div.sorry > p'): #body > div:nth-child(1) > div.sorry > p
        pass
    else:
        print(url)

def main():
    years='2017'
    for month in range(1,13):
        for num in range(1,15):
            get_url(years, month, num)
            time.sleep(2)

if __name__ == '__main__':
    main()

3.判斷條件是根據(jù)這部分選擇

預(yù)想結(jié)果是會根據(jù) main 函數(shù)里的內(nèi)容進(jìn)行循環(huán)判斷, 但是 201701013 和 14 后綴的連接仍舊會跳轉(zhuǎn)到上面的部分, 但是會被作為結(jié)果返回.!!!

4.我進(jìn)行的處理是反其道而行之, 設(shè)想通過判斷存在某部分內(nèi)容來選擇, 但依舊沒有任何效果.