极客观点
项目管理
HarmonyOS
按照网上的模板自己写了类似的代码爬取微博，可是response回来的html是登录界面的html，应该是没有成功登陆微博，但是和网上的代码是基本一样的
from bs4 import BeautifulSoup
import pandas as pd
import requests
url = "https://weibo.cn/2610622321/info"
cook = {"Cookies":"SUB=_2A25xtQZjDeRhGeVM6VQV8CvIyjuIHXVTWaorrDV6PUJbkdAKLW-ikW1NTRxe3ZqwS1KogOneF6bXUXz7lkhb7SGJ; SUHB=0QSpzHMgP-YPEV; SCF=AisDqT2yt7Pkh8gy_7C80NumATyJngEuCniBti09uOwx-8_0IHEkgoGo9HSTyrsBm2J-UMjtDw_DbAEbO-jH2xA.; _T_WM=2901aedc3b6c36f819fe83923328067d"}
html = requests.get(url,cookies = cook).content
soup = BeautifulSoup(html,'lxml')
print(soup)可结果获取的html是这样的
<!DOCTYPE html>
<html style="height:100%">
<head>
<meta charset="utf-8"/>
<link href="http://h5.sinaimg.cn" rel="dns-prefetch"/>
<link href="http://u1.sinaimg.cn" rel="dns-prefetch"/>
<link href="http://ww1.sinaimg.cn" rel="dns-prefetch"/>
<link href="http://ww2.sinaimg.cn" rel="dns-prefetch"/>
<link href="http://ww3.sinaimg.cn" rel="dns-prefetch"/>
<link href="http://ww4.sinaimg.cn" rel="dns-prefetch"/>
<meta content="yes" name="apple-mobile-web-app-capable"/>
<meta content="black" name="apple-mobile-web-app-status-bar-style"/>
<link href="http://u1.sinaimg.cn/upload/h5/img/apple-touch-icon.png" rel="apple-touch-icon"/>
<title>登录 - 新浪微博</title>
<meta content="width=device-width,initial-scale=1.0,minimum-scale=1.0,maximum-scale=1.0,minimal-ui" id="viewport" name="viewport"/>
<link href="/css/weibo/signin/login.css?id=20141028160201" rel="stylesheet"/>
</head>
<body style="height:100%">
<div class="login-wrapper" id="loginWrapper" style="display:none">
<!-- 登陆成功跳转页 -->
<input id="loginSuccessAddress" name="url" type="hidden" value="https%3A%2F%2Fweibo.cn"/>
<!-- 登陆rf标识 -->
<input id="loginRF" name="rf" type="hidden"/>
<input id="loginRFCAL" name="rfcal" type="hidden"/>
<input id="oldUserName" type="hidden"/>
<input id="loginfrom" type="hidden" value=""/>
<input id="featurecode" type="hidden" value=""/>
<input id="hff" name="hff" type="hidden" value=""/>
<input id="hfp" name="hfp" type="hidden" value=""/>
<a class="close" href="javascript:history.go(-1);">关闭</a>
<section class="avatar-wrapper" id="avatarWrapper">
<!-- 头像 -->
<img onerror="this.src='/images/weibo/signin/default-avatar_2x.png';" src=""/>
</section>
<form>
<section class="box">
<div class="input-wrapper">
<i class="icon forName"></i>
<!-- 用户名 -->
<p class="input-box">
<input id="loginName" placeholder="邮箱/手机号" type="text"/>
<!-- 清除用户名小叉 -->
<a class="input-clear" href="javascript:;" id="loginnameclear" style="display:none"></a>
</p>
</div>
<div class="input-wrapper">
<i class="icon forPwd"></i>
<p class="input-box"><input id="loginPassword" placeholder="请输入密码" type="password"/></p>
</div>
</section>
<!-- 要隐藏添加 hid -->
<div id="verifyCodeWrapper" style="display:none">
<p class="vcode">
<img alt="" id="verifyCodeImage" src=""/>
<a href="javascript:;" id="changeVerifyCode">换一张</a>
</p>
<!-- 要隐藏添加 hid -->
<div class="box">
<input id="loginVCode" placeholder="请输入验证码" type="text"/>
</div>
</div>
<!-- 要隐藏添加 hid -->
<div id="dVerifyCodeWrapper" style="display:none">
<p class="shield-tit">动态码</p>
<!-- 要隐藏添加 hid -->
<div class="box">
<input id="loginDVCode" name="shieldCode" placeholder="请输入微盾动态码" type="text"/>
</div>
</div>
<div class="error-label" id="errorMsg" style="display:none">手机号不能为空</div>
<a class="btn btnRed" href="javascript:;" id="loginAction">登录</a>
</form>
<p class="label">
<a href="https://passport.weibo.cn/signin/other?r=https%3A%2F%2Fweibo.cn">第三方帐号</a>
</p>
<footer class="footer">
<a href="https://m.weibo.cn/reg/index?&vt=4&wm=3349&wentry=&backURL=https%3A%2F%2Fweibo.cn">注册帐号</a><a href="https://passport.weibo.cn/forgot/forgot?entry=wapsso&from=0&r=https%3A%2F%2Fm.weibo.cn%2F">忘记密码</a>
</footer>
</div>
<!-- 要隐藏添加 hid -->
<div class="popup" id="errorDialog" style="display:none">
<article class="wrapper" id="errorDialogPanel" style="top:180px;">
<header class="title"></header>
<span class="info" id="errorDialogMsg">帐号或密码错误，你也可以选择短信验证码方式登录微博。</span>
<div class="action">
<a class="btn cancel" href="javascript:;" id="errorDialogBtnF">取消</a>
<a class="btn confirm" href="javascript:;" id="errorDialogBtnT">验证码登录</a>
</div>
</article>
</div>
<div class="account-wrapper" id="accountWrapper" style="display:none">
<a class="close" href="javascript:history.go(-1);">关闭</a>
<section class="avatar-wrapper">
<!-- 头像 -->
<img alt="" id="ucavatar" onerror="this.src='/images/weibo/signin/default-avatar_2x.png';" src="/images/weibo/signin/default-avatar_2x.png"/>
</section>
<p class="label" id="uctext">使用<span id="ucname"></span>的身份登录网页版微博</p>
<p class="label" id="weibotext">检测到您已在微博客户端登录<br/>是否获取该登录状态？</p>
<a class="btn btnRed" href="javascript:;" id="ucORweiboLogin">确认</a>
<a class="btn btnWhite" href="javascript:;" id="changeLogin">使用其他帐号登录</a>
</div>
<form action="https://open.weibo.cn/oauth2/authorize" id="postform" method="post">
<button style="display:none" type="submit"></button>
</form>
<script type="text/javascript">
var fEntry = '';
var fWentry = '';
var fBackUrl = '';
fEntry = 'mweibo';
fBackUrl = 'https%3A%2F%2Fweibo.cn';
var fClientid = '';
var fCode = '';
var fQq = '';
</script>
<!-- 登录wifi -->
<script charset="gb2312" src="/js/signin/req.js?v=20181220" type="text/javascript"></script>
<script charset="gb2312" src="/js/signin/weibologin.js?v=20181220" type="text/javascript"></script>
<script type="text/javascript">
//记住用户名的时候需要把mode设置为1
//这里是需要微盾
</script>
</body>
</html>请过来人解答。是需要模拟登陆吗？可是我看网上的教程都不需要模拟登陆啊
python
过来人回答一波，这是前年我用过的爬虫，不知道还有没有用了，你可以试下，爬微博最好用u端web，网页简单好爬。