pyspider报can't use a string pattern on a bytes-like object的错误

Question

pyspider processor 重启失败, 报出了can't use a string pattern on a bytes-like object

[E 151118 10:20:19 processor:225] can't use a string pattern on a bytes-like object

Traceback (most recent call last):
  File "/root/anaconda3/lib/python3.4/site-packages/pyspider/processor/processor.py", line 218, in run
    self.on_task(task, response)
  File "/root/anaconda3/lib/python3.4/site-packages/pyspider/processor/processor.py", line 151, in on_task
    'encoding': response.encoding,
  File "/root/anaconda3/lib/python3.4/site-packages/pyspider/libs/response.py", line 81, in encoding
    encoding = get_encodings_from_content(utils.pretty_unicode(self.content[:100]))
  File "/root/anaconda3/lib/python3.4/site-packages/requests/utils.py", line 304, in get_encodings_from_content
    pragma_re.findall(content) +
TypeError: can't use a string pattern on a bytes-like object

Answer 1

经过查找, 在github.com上找到了答案:
luoq referenced this issue on 30 Jul

所以我升级了一下新的版本0.3.6 ,问题解决了

Answer 2

content 的类型是 bytes，而 pragma_re 是从 str 构建的，只能应用于 str。

应当把 content 用正确的编码 decode 一下再找。