写了小爬虫,用到asyncio做异步,多次调用async with 和 await爬虫操作,但是在其中没调用异步的数据库操作没有成功,代码下:
async with session.get(restaurant_url) as restaurant_rep:
restaurant_rep = await restaurant_rep.json()
rating_url = RES_URL.format(restaurant_id=restaurant_id)
async with session.get(rating_url) as rating_rep:
rating_rep = await rating_rep.json()
overall_rating, service_rating, food_rating = format_rating_data(rating_rep)
add_restaurant_data(restaurant_rep, overall_rating, service_rating, food_rating)
food_url = constants.ELE_FOOD_JSON.format(restaurant_id=restaurant_id)
async with session.get(food_url) as food_rep:
food_rep = await food_rep.json()
for food_type in food_rep:
for food in food_type['foods']:
food_id = food['specfoods'][0]['food_id']
if food_id not in food_set:
food_set.add(food_id)
add_food_data(food)
add_*函数 解析response并写入数据库, 但是8行写操作并没有写成功, 函数没问题, 19行写数据库操作可以,求问一下为什么
顺便问一下这样写有什么不好?