问题描述
scrapy对于同一个url用不同的方法给parse,有的能执行,有的就不能
问题出现的环境背景及自己尝试过哪些方法
相关代码
urls=['http://maoyan.com/xseats/201809300156836?movieId=342166&cinemaId=16887','http://maoyan.com/xseats/201809300237849?movieId=1210830&cinemaId=2285','http://maoyan.com/xseats/201809300013842?movieId=1217402&cinemaId=13199']
for url in urls:
yield Request(url=url,callback=self.parse)
这个parse就能执行
# # 从数据库中获得城市的id
myclient = pymongo.MongoClient("mongodb://localhost:27017/")
mydb = myclient["scrapy"]
mycol_movie = mydb["movieTime"]
movies=list(mycol_movie.find().sort([("movieTime",1)]))
for movie in movies:
now = float(datetime.datetime.now().strftime('%H.%M'))
if (now > movie['movieTime']):
print(str(movie['movieTime']) + " 过时了")
else:
while True:
if (movie['movieTime'] < now + 0.07):
now = float(datetime.datetime.now().strftime('%H.%M'))
print(str(now)+" "+str(movie['movieTime']))
url="http://maoyan.com" + str(movie['movieUrl'])
print(url)
try:
yield Request(url=url, callback=self.parse)
except:
print("抛出失败!!!")
finally:
print('抛出了反正')
print(" ")
break
else:
print("暂停45秒,,")
time.sleep(60)
now = float(datetime.datetime.now().strftime('%H.%M'))
这个就不能执行,
你期待的结果是什么?实际看到的错误信息又是什么?
让复杂的能够执行。