如何实现类似requests的Session的功能

Question

抓取http://www.blogbar.cc/首页右侧的最新文章，翻页是一个ajax，headers里带有一个X-CSRFToken，用requests的Session实现了，不知道用pyspider如何实现

#!/usr/bin/env python
# coding:utf-8

import requests
from pyquery import PyQuery as pq

s = requests.Session()

def get_csrftoken():
    content = s.get('http://www.blogbar.cc').content
    csrftoken = pq(content)('meta[name="csrf-token"]').attr.content
    return csrftoken


def get_lastest_posts(page):
    headers = {
        'X-CSRFToken': get_csrftoken(),
    }
    
    payload = {
        'page': page,
    }
    content = s.post('http://www.blogbar.cc/load_posts', data=payload, headers=headers).content
    return content

if __name__ == "__main__":
    print get_lastest_posts(2)

类似的用pyspider写了下，失败了

from pyspider.libs.base_handler import *


class Handler(BaseHandler):
    
    crawl_config = {
    }

    @every(minutes=24 * 60)
    def on_start(self):
        self.crawl('http://www.blogbar.cc/', callback=self.index_page)

    @config(age=10 * 24 * 60 * 60)
    def index_page(self, response):
        csrftoken = response.doc('meta[name="csrf-token"]').attr.content
        headers = {'X-CSRFToken':csrftoken,}
        url = 'http://www.blogbar.cc/load_posts'    
        
        self.crawl(url, method="POST", data={'page':2}, headers=headers, callback=self.detail_page)
            
    @config(priority=2)
    def detail_page(self, response):
        print response.content

Answer 1

CSRF在cookie中还有, response.cookies 读cookie再设置到下一个请求中.

Answer 2

同问“pyspider 如何实现类似requests的Session的功能”