Pyspider webui调试错误

新手上路,请多包涵

1.在Ubuntu上搭建好Pyspider环境后,用Pyspider all命令启动,然后在webui中单步调试测试时,遇到下面异常:


Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/pyspider/run.py", line 360, in <lambda>
    app.config['fetch'] = lambda x: webui_fetcher.fetch(x)
  File "/usr/local/lib/python2.7/dist-packages/pyspider/fetcher/tornado_fetcher.py", line 114, in fetch
    return self.async_fetch(task, callback).result()
  File "/usr/lib/python2.7/dist-packages/tornado/concurrent.py", line 129, in result
    raise_exc_info(self.__exc_info)
  File "/usr/lib/python2.7/dist-packages/tornado/gen.py", line 227, in wrapper
    runner.run()
  File "/usr/lib/python2.7/dist-packages/tornado/gen.py", line 531, in run
    yielded = self.gen.send(next)
  File "/usr/local/lib/python2.7/dist-packages/pyspider/fetcher/tornado_fetcher.py", line 136, in async_fetch
    result = self.handle_error(type, url, task, e)
TypeError: handle_error() takes exactly 6 arguments (5 given)

测试所用的python脚本是github上官方demo程序;然后又使用本人在Windows平台上已经测试能正常运行的程序,同样抛出以上异常

终端报错信息是:

[E 160817 11:00:53 tornado_fetcher:135] 'module' object has no attribute 'maybe_future'
Traceback (most recent call last):
    File "/usr/local/lib/python2.7/dist-packages/pyspider/fetcher/tornado_fetcher.py", line 127, in async_fetch
    result = yield gen.maybe_future(self.data_fetch(url, task))
    AttributeError: 'module' object has no attribute 'maybe_future'

本人测试的程序是:

from pyspider.libs.base_handler import *


class Handler(BaseHandler):
    crawl_config = {
    }

    @every(minutes=24 * 60)
    def on_start(self):
        self.crawl('http://scrapy.org/', callback=self.index_page)

    @config(age=10 * 24 * 60 * 60)
    def index_page(self, response):
        for each in response.doc('a[href^="http"]').items():
            self.crawl(each.attr.href, callback=self.detail_page)

    def detail_page(self, response):
        return {
            "url": response.url,
            "title": response.doc('title').text(),
        }

本人刚接触Pyspider,想研究源码但是心有余力不足,所以在这儿虚心向各位大神请教,望得到大神指点!

阅读 4.7k
1 个回答
新手上路,请多包涵

我在Github上提交问题后,Pyspider作者,Binux大神问我tornado版本,这个提醒了我,原tornado版本是3.2,于是我用命令

pip install -U tornado

升级了tornado版本,问题解决了,多谢足兆叉虫,您应该就是Binux吧?

撰写回答
你尚未登录,登录后可以
  • 和开发者交流问题的细节
  • 关注并接收问题和回答的更新提醒
  • 参与内容的编辑和改进,让解决方法与时俱进