针对部分网站使用python请求出现403等,强制使用tls1.2发包的场景
如某盛优选小程序,fiddler重新发包正常,用程序请求失败
1.urllib版本
context = ssl.SSLContext(ssl.PROTOCOL_TLSv1_2) response = urllib.request.urlopen(url=urllib.request.Request(url=url, headers=headers), data=data, context=context) response = response.read().decode() print(response)
2.requests版本
import ssl import requests from requests.adapters import HTTPAdapter, PoolManager class MyAdapter(HTTPAdapter): # 重写init_poolmanager方法 def init_poolmanager(self, connections, maxsize, block=False, **pool_kwargs): self.poolmanager = PoolManager(num_pools=connections, maxsize=maxsize, block=block, ssl_version=ssl.PROTOCOL_TLSv1_2) s = requests.Session() s.mount('https://', MyAdapter()) res = s.post(url, data=data, headers=headers) text = res.text
3.在scrapy中强制使用TLS1.2的话需要在settings.py中设置以下代码
参考scrapy文档: https://docs.scrapy.org/en/latest/topics/settings.html#std-setting-DOWNLOADER_CLIENT_TLS_CIPHERS
DOWNLOADER_CLIENT_TLS_METHOD="TLSv1.2"
4.直接使用httpx请求
import httpx res = httpx.post(url, data=data, headers=headers) text = res.text