cURL爬取淘宝时无法获取数据,正常访问时却有数据?

Question

cURL爬取淘宝时无法获取数据,正常访问时却有数据?

发布于
2017-07-18

今天想爬爬淘宝数据时,发现淘宝的反爬虫又加强了
下面是我curl的配置

        $ch = curl_init();
        //设置选项
        curl_setopt($ch,CURLOPT_URL,'https://mochagushi.tmall.com');
        curl_setopt($ch,CURLOPT_RETURNTRANSFER,1);
        curl_setopt($ch,CURLOPT_HEADER,0);
        curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE);
        curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, FALSE);
        curl_setopt($ch,CURLOPT_FOLLOWLOCATION,true);
        curl_setopt($ch, CURLOPT_REFERER, $url);
        $header = array();
        $header[] = 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8';
        $header[] = 'Accept-Language: zh-CN,zh;q=0.8';
        $header[] = 'Cache-Control: max-age=0';
        $header[] = 'Connection: keep-alive';
        $header[] = 'Cookie: uc3=nk2=0M56YNYN%2BA4%3D&id2=W8rr4uUwfV5Z&vt3=F8dAT%2B%2BMdeVt9sjBn%2Bk%3D&lg2=WqG3DMC9VAQiUQ%3D%3D; lgc=%5Cu695A%5Cu5929%5Cu5B9D%5Cu9676; tracknick=%5Cu695A%5Cu5929%5Cu5B9D%5Cu9676; t=45643498fa8ff940a0534abf468c496b; _tb_token_=5touPdVYykOo; cookie2=9cc3ace3ba141f7e0e950da7b4782eae; tt=segmentfault.com; res=scroll%3A1349*6222-client%3A1349*667-offset%3A1349*6222-screen%3A1366*768; pnm_cku822=055UW5TcyMNYQwiAiwQRHhBfEF8QXtHcklnMWc%3D%7CUm5Ockt1T3BEf0N%2BSnJPciQ%3D%7CU2xMHDJ7G2AHYg8hAS8WKQcnCVU0Uj5ZJ11zJXM%3D%7CVGhXd1llXGJYZ1NoVGldZVhlUm9NeEd9SHFFfElyR35Be0N%2FR2k%2F%7CVWldfS0TMwY8ACAcKQkncVU4XSNKGn9AZE8BUDxQCW4PIXch%7CVmhIGCUFOBgkGiMXNwwzDDMTLxEqETELMAUlGSccJwc9AjdhNw%3D%3D%7CV25Tbk5zU2xMcEl1VWtTaUlwJg%3D%3D; cq=ccp%3D1; l=AikpB1ewOIvOADHsO87Aa9qtuduD9B0s; otherx=e%3D1%26p%3D*%26s%3D0%26c%3D0%26f%3D0%26g%3D0%26t%3D0; swfstore=142738; isg=4C03295B995DB1CB7ADB1CC702592A48';
        curl_setopt($ch, CURLOPT_HTTPHEADER, $header);
        //执行并获取HTML文档内容
        $data = curl_exec($ch);
        curl_close($ch);

以前的话是没有vdata.htm这个xml文件的,只需要获取asynSearch.htm这个文件即可
图片描述