关于python3爬虫post json 无法爬取数据？

Question

问题描述

python版本为3.6，按照网上的模板来写该代码，发现无法爬取。不知道错在哪里，有大神出来指点一下吗？

问题出现的环境背景及自己尝试过哪些方法

相关代码

// 请把代码文本粘贴到下方（请勿用图片代替代码）

import urllib.request
import urllib.parse
import json
url = r'XXXXXXX'
headers = {'Accept':'application/json, text/javascript, */*; q=0.01',
    'Accept-Language':'zh-CN,zh;q=0.8',
    'Content-Length':'2',
    'Content-Type':'application/json',
    'Cookie':'XXXXX',
    'Host':'XXXX',
    'Origin':'XXXXX',
    'Proxy-Connection':'keep-alive',
    'Referer':'XXXXXX',
    'User-Agent':'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like             Gecko) Chrome/55.0.2883.87 Safari/537.36'
    }
data = {'requestType':'3',
    'quoteBeginTime':'2018-07-12',
    'quoteEndTime':'2018-07-13',
    'supplierSign':'false',
    'newQuality':'false',
    'newAutoQuote':'false',
    'inquiryTimesDesc':'false',
    'page':'1',
    'pageSize':'20',
    'orderBy':'0',
    'ordered':'false'
    }
data=json.dumps(data)
data=data.encode()
wed_content=urllib.request.Request(url,headers=headers,data=data)
wed_content=urllib.request.urlopen(wed_content,timeout=120)
wed_content=wed_content.read()
wed_content=wed_content.decode('utf-8')
print(wed_content)

你期待的结果是什么？实际看到的错误信息又是什么？

报错内容是urllib.error.HTTPError: HTTP Error 400: Bad Request

期待的结果是返回JSON的数据

第二次报错内容：Traceback (most recent call last):
  File "E:/pg/batulu/4.py", line 31, in <module>
    wed_content=urllib.request.urlopen(wed_content,timeout=120)
  File "C:\Users\hasee\AppData\Local\Programs\Python\Python36\lib\urllib\request.py", line 223, in urlopen
    return opener.open(url, data, timeout)
  File "C:\Users\hasee\AppData\Local\Programs\Python\Python36\lib\urllib\request.py", line 532, in open
    response = meth(req, response)
  File "C:\Users\hasee\AppData\Local\Programs\Python\Python36\lib\urllib\request.py", line 642, in http_response
    'http', request, response, code, msg, hdrs)
  File "C:\Users\hasee\AppData\Local\Programs\Python\Python36\lib\urllib\request.py", line 570, in error
    return self._call_chain(*args)
  File "C:\Users\hasee\AppData\Local\Programs\Python\Python36\lib\urllib\request.py", line 504, in _call_chain
    result = func(*args)
  File "C:\Users\hasee\AppData\Local\Programs\Python\Python36\lib\urllib\request.py", line 650, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
    urllib.error.HTTPError: HTTP Error 400: Bad Request

Answer 1

你确定 JSON 数据里都是字符串类型吗？试试用回原生类型：

data = {'requestType':'3',
    'quoteBeginTime':'2018-07-12',
    'quoteEndTime':'2018-07-13',
    'supplierSign': False,
    'newQuality': False,
    'newAutoQuote': False,
    'inquiryTimesDesc': False,
    'page': 1,
    'pageSize': 20,
    'orderBy': 0,
    'ordered': False
    }