利用scrapy爬取出现SyntaxError: invalid syntax

Question

问题：出现SyntaxError: invalid syntax
报错代码：

shiyanlou:louspider/ $ scrapy crawl mylouspider -o courses.json      [11:00:46]
Traceback (most recent call last):
  File "/usr/local/bin/scrapy", line 11, in <module>
    sys.exit(execute())
  File "/usr/local/lib/python3.5/dist-packages/scrapy/cmdline.py", line 148, in execute
    cmd.crawler_process = CrawlerProcess(settings)
  File "/usr/local/lib/python3.5/dist-packages/scrapy/crawler.py", line 243, in __init__
    super(CrawlerProcess, self).__init__(settings)
  File "/usr/local/lib/python3.5/dist-packages/scrapy/crawler.py", line 134, in __init__
    self.spider_loader = _get_spider_loader(settings)
  File "/usr/local/lib/python3.5/dist-packages/scrapy/crawler.py", line 330, in _get_spider_loader
    return loader_cls.from_settings(settings.frozencopy())
  File "/usr/local/lib/python3.5/dist-packages/scrapy/spiderloader.py", line 61, in from_settings
    return cls(settings)
  File "/usr/local/lib/python3.5/dist-packages/scrapy/spiderloader.py", line 25, in __init__
    self._load_all_spiders()
  File "/usr/local/lib/python3.5/dist-packages/scrapy/spiderloader.py", line 47, in _load_all_spiders
    for module in walk_modules(name):
  File "/usr/local/lib/python3.5/dist-packages/scrapy/utils/misc.py", line 71, in walk_modules
    submod = import_module(fullpath)
  File "/usr/lib/python3.5/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 986, in _gcd_import
  File "<frozen importlib._bootstrap>", line 969, in _find_and_load
  File "<frozen importlib._bootstrap>", line 958, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 673, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 669, in exec_module
  File "<frozen importlib._bootstrap_external>", line 775, in get_code
  File "<frozen importlib._bootstrap_external>", line 735, in source_to_code
  File "<frozen importlib._bootstrap>", line 222, in _call_with_frames_removed
  File "/home/shiyanlou/louspider/louspider/spiders/mylouspider.py", line 16
    item['image'] = course.xpath('.//div[@class="course-img"]/img/@src').extract()[0].strip()
       ^
SyntaxError: invalid syntax
shiyanlou:louspider/ $ ls                                            [11:01:00]
items.py  louspider  scrapy.cfg
shiyanlou:louspider/ $ cd louspider                                  [11:03:38]
shiyanlou:louspider/ $ ls                                            [11:08:16]
__init__.py  middlewares.py  __pycache__  spiders
items.py     pipelines.py    settings.py
shiyanlou:louspider/ $ vi items.py                                   [11:08:17]
shiyanlou:louspider/ $ scrapy crawl mylouspider -o courses.json      [11:08:40]
Traceback (most recent call last):
  File "/usr/local/bin/scrapy", line 11, in <module>
    sys.exit(execute())
  File "/usr/local/lib/python3.5/dist-packages/scrapy/cmdline.py", line 148, in execute
    cmd.crawler_process = CrawlerProcess(settings)
  File "/usr/local/lib/python3.5/dist-packages/scrapy/crawler.py", line 243, in __init__
    super(CrawlerProcess, self).__init__(settings)
  File "/usr/local/lib/python3.5/dist-packages/scrapy/crawler.py", line 134, in __init__
    self.spider_loader = _get_spider_loader(settings)
  File "/usr/local/lib/python3.5/dist-packages/scrapy/crawler.py", line 330, in _get_spider_loader
    return loader_cls.from_settings(settings.frozencopy())
  File "/usr/local/lib/python3.5/dist-packages/scrapy/spiderloader.py", line 61, in from_settings
    return cls(settings)
  File "/usr/local/lib/python3.5/dist-packages/scrapy/spiderloader.py", line 25, in __init__
    self._load_all_spiders()
  File "/usr/local/lib/python3.5/dist-packages/scrapy/spiderloader.py", line 47, in _load_all_spiders
    for module in walk_modules(name):
  File "/usr/local/lib/python3.5/dist-packages/scrapy/utils/misc.py", line 71, in walk_modules
    submod = import_module(fullpath)
  File "/usr/lib/python3.5/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 986, in _gcd_import
  File "<frozen importlib._bootstrap>", line 969, in _find_and_load
  File "<frozen importlib._bootstrap>", line 958, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 673, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 669, in exec_module
  File "<frozen importlib._bootstrap_external>", line 775, in get_code
  File "<frozen importlib._bootstrap_external>", line 735, in source_to_code
  File "<frozen importlib._bootstrap>", line 222, in _call_with_frames_removed
  File "/home/shiyanlou/louspider/louspider/spiders/mylouspider.py", line 16
    item['image'] = course.xpath('.//div[@class="course-img"]/img/@src').extract()[0].strip()
       ^
SyntaxError: invalid syntax

mylouspider.py

  1 import scrapy
  2 from louspider.items import LouSpiderItem
  3 from scrapy.selector import Selector
  4 
  5 class LouSpider(scrapy.Spider):
  6     name = "mylouspider"
  7     allowed_domains = ["shiyanlou.com"]
  8     start_urls = ['https://www.shiyanlou.com/courses/?category=all&course_ty
    pe=all&tag=all&fee=free']
  9     def parse(self, response):
 10         hxs = Selector(response)
 11         courses = hxs.xpath('//div[@class="col-md-3 col-sm-6 course"]')
 12         for course in courses: 
 13             item = CourseItem()
 14             item['name'] = course.xpath('.//div[@class="course-name"]/text(    )').extract()[0].strip() 
 15             item['learned'] = course.xpath('.//span[@class="course-per-num p    ull-left"]/text()').extract([1].strip() 
 16             item['image'] = course.xpath('.//div[@class="course-img"]/img/@s    rc').extract()[0].strip()
 17             yield item

items.py

import scrapy


class LouSpiderItem(scrapy.Item):
    # define the fields for your item here like:
    name = scrapy.Field()
    image = scrapy.Field()
    learned = scrapy.Field()

利用ipython通过scrap shell测试xpath可以抓取
但是整体运行就报错
新手求大神解答 感谢！！！！

Answer 1

你在spider里面用的CourseItem()貌似没有定义吧