1、书籍附带的文件,python网络爬虫实例教程4.1爬q房,最简单的一个爬虫未显示错误,练习很久了,视频也看了,书也一条一条对照,不管怎么调试,print什么内容,检查URL和Xpath都没有错误,但是没内容?
2、
from lxml import etree
import requests
import csv
import time
def data_writer(item):
with open('qfang.csv', 'a',encoding='utf-8',newline='') as csvfile:
writer = csv.writer(csvfile)
writer.writerow(item)
def spider():
headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2490.80 Safari/537.36'}
pre_url = 'http://shenzhen.qfang.com/sale/f'
for x in range(1,11):
html = requests.get(pre_url + str(x), headers=headers)
time.sleep(1)
selector = etree.HTML(html.text)
house_list = selector.xpath("//*[@id='cycleListings']/ul/li")
for house in house_list:
xiaoqu = house.xpath("div[1]/p[1]/a/text()")[0]
huxing = house.xpath("div[1]/p[2]/span[2]/text()")[0]
mianji = house.xpath("div[1]/p[2]/span[4]/text()")[0]
quyu = house.xpath("div[1]/p[3]/span[2]/a[1]/text()")[0]
zongjia = house.xpath("div[2]/span[1]/text()")[0]
item = [xiaoqu, huxing, mianji, quyu, zongjia]
data_writer(item)
print('正在抓取', xiaoqu)
if name == '__main__':
spider()
3、C:UserspcPycharmProjectsuntitledvenvScriptspython.exe "C:/Users/pc/Desktop/python学习/爬虫电子书/Python爬虫实例教程-代码/4.1 qfang.py"
h
进程完成,退出码 0
4、notebooks、Geany、Visual Studio Code、JetBrains PyCharm 2019.1.1 x64都试了,不缺第三方库、也print不出内容,CSV也没生成。