start_urls = ["http://xxx.com/xxx]
def parse(self, response):
yield Request(url,headers=self.headers,callback=self.parse2)
其实我想在第二层触发自定义RedirectMiddleware,但是start_urls也会触发(进入这层url,不需要触发重定向),如何控制?
--------------------补充RedirectMiddleware------------------------------------
class RedirectMiddleware(object):
def process_request(self, request, spider):
print '111111111111111'
print request.url
Reference:
1.Scrapy1.0文档-下载中间件(downloadmiddleware)
2.Scrapy1.0文档-request and response