獲取響應(yīng)信息
import requests
response = requests.get('http://www.baidu.com')
print(response.status_code) # 狀態(tài)碼
print(response.url) # 請(qǐng)求url
print(response.headers) # 響應(yīng)頭信息
print(response.cookies) # cookie信息
print(response.content) # bytes形式的響應(yīng)內(nèi)容
print(response.encoding) # 獲取響應(yīng)內(nèi)容編碼
response.encoding=”utf-8” # 指定響應(yīng)內(nèi)容編碼
print(response.text) # 文本形式的響應(yīng)內(nèi)容,response.content編碼后的結(jié)果
發(fā)送Get請(qǐng)求
不帶參數(shù)的Get請(qǐng)求
response = requests.get('http://www.baidu.com')
print(response.text)
帶參數(shù)的Get請(qǐng)求
直接寫(xiě)在url后面
在url后面用?表示帶上參數(shù),每對(duì)參數(shù)用&分隔。如下url:
https://www.bilibili.com/vide...
注意:url最長(zhǎng)2048字節(jié),且數(shù)據(jù)透明不安全
作為字典參數(shù)傳入
data = {'name': 'xiaoming', 'age': 26}
response = requests.get('http://www.abcd.com', params=data)
print(response.text)
發(fā)送post請(qǐng)求
只能作為字典參數(shù)傳入,注意參數(shù)名字是data而不是params
data = {'name': 'xiaoming', 'age': 26}
response = requests.post('http://www.abcd.com', data=data)
print(response.text)
添加headers
heads = {}
heads['User-Agent'] = 'Mozilla/5.0 ' /
'(Macintosh; U; Intel Mac OS X 10_6_8; en-us) AppleWebKit/534.50 ' /
'(KHTML, like Gecko) Version/5.1 Safari/534.50'
response = requests.get('http://www.baidu.com',headers=headers)
使用代理
proxy = {'http': '49.89.84.106:9999', 'https': '49.89.84.106:9999'}
heads = {}
heads['User-Agent'] = 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.221 Safari/537.36 SE 2.X MetaSr 1.0'
req = requests.get(url, proxies=proxy, headers=heads)
print(req.text)
使用加密代理
from requests.auth import HTTPProxyAuth
proxies= {'http': '127.0.0.1:8888', 'https': '127.0.0.1:8888'}
auth = HTTPProxyAuth('user', 'pwd')
requests.get(url, proxies=proxies, auth=auth)
也可以這樣
proxies = {"http": "http://user:pass@10.10.1.10:3128/",}
req = requests.get(url, proxies=proxy, headers=heads)
Cookie
獲取Cookie
import requests
response = requests.get("http://www.baidu.com")
print(type(response.cookies))
# 把cookiejar對(duì)象轉(zhuǎn)化為字典
cookies = requests.utils.dict_from_cookiejar(response.cookies)
print(cookies)
使用Cookie
cookie = {"Cookie":"xxxxxxxx"}
response = requests.get(url,cookies=cookie)
Session
session = requests.Session()
session.get('http://httpbin.org/cookies/set/number/12345')
response = session.get('http://httpbin.org/cookies')
print(response.text)
限定響應(yīng)時(shí)間
from requests.exceptions import ReadTimeout
try:
response = requests.get('https://www.baidu.com', timeout=1)
print(response.status_code)
except :
print('給定時(shí)間內(nèi)未響應(yīng)')
解析JSON格式的響應(yīng)內(nèi)容
通過(guò)response.json()方法可以將為JSON格式的響應(yīng)內(nèi)容轉(zhuǎn)變?yōu)?a href="http://www.brongaenegriffin.com/tags/python/" target="_blank">Python的對(duì)象,json.loads(response.text)也能起到同樣的作用
response = requests.get('http://www.abcd.com')
print(response.text)
print(response.json())
print(type(response.json()))
想進(jìn)一步了解編程開(kāi)發(fā)相關(guān)知識(shí),與我一同成長(zhǎng)進(jìn)步,請(qǐng)關(guān)注我的公眾號(hào)“松果倉(cāng)庫(kù)”,共同分享宅&程序員的各類(lèi)資源,謝謝?。?!
審核編輯 黃昊宇
-
python
+關(guān)注
關(guān)注
56文章
4827瀏覽量
86702 -
爬蟲(chóng)
+關(guān)注
關(guān)注
0文章
83瀏覽量
7502
發(fā)布評(píng)論請(qǐng)先 登錄
Python數(shù)據(jù)爬蟲(chóng)學(xué)習(xí)內(nèi)容
Python爬蟲(chóng)與Web開(kāi)發(fā)庫(kù)盤(pán)點(diǎn)
Python爬蟲(chóng)初學(xué)者需要準(zhǔn)備什么?
0基礎(chǔ)入門(mén)Python爬蟲(chóng)實(shí)戰(zhàn)課
Python爬蟲(chóng)簡(jiǎn)介與軟件配置
python網(wǎng)絡(luò)爬蟲(chóng)概述
詳細(xì)用Python寫(xiě)網(wǎng)絡(luò)爬蟲(chóng)

完全自學(xué)指南Python爬蟲(chóng)BeautifulSoup詳解

評(píng)論