Fabric - 动态生成主机列表和角色列表

在使用 Fabric 的过程中，一般我们常用的方式是手工填写主机列表或者是角色列表，但是这样当服务器数量超级多的时候，你会有想死的感觉的。正好公司有 cmdb 的话，就可以结合 CMDB 来做。

PS：如果公司没有开发 CMDB 系统，也自己尽量弄个简单的系统录入服务器数据吧，不要那么复杂，提供 API，能获取主机列表即可。

动态生成主机列表

通过参考 Fabric 的官方文档的 Using execute with dynamically-set host lists，其中有这么一段示例代码：

from fabric.api import run, execute, task

# For example, code talking to an HTTP API, or a database, or ...
from mylib import external_datastore

# This is the actual algorithm involved. It does not care about host
# lists at all.
def do_work():
    run("something interesting on a host")

# This is the user-facing task invoked on the command line.
@task
def deploy(lookup_param):
    # This is the magic you don't get with @hosts or @roles.
    # Even lazy-loading roles require you to declare available roles
    # beforehand. Here, the sky is the limit.
    host_list = external_datastore.query(lookup_param)
    # Put this dynamically generated host list together with the work to be
    # done.
    execute(do_work, hosts=host_list)

然后执行命令：

$ fab deploy:app
$ fab deploy:db

其生成主机列表的格式如下：

['10.2.5.1', '10.2.5.2', '10.2.5.3', '10.2.5.4', '10.2.5.5', '10.2.5.6', '10.2.5.7', '10.2.5.8', '10.2.5.9', '10.2.5.10']

现在我们就可以根据 CMDB 接口来动态生成主机列表了。具体见代码吧

方法一：

#!/usr/bin/env python
#encoding=utf-8
import sys
import os
import requests


from fabric.api import env
from fabric.api import run
from fabric.api import put
from fabric.api import execute
from fabric.api import roles
from fabric.api import parallel
from fabric.api import cd
from fabric.api import task

env.user = 'test'
env.password = 'test'



cmdburl = 'http://cmdb.test.com/test/listServer.do'


def find_ips_by_domain(domain_name):
     ips=[]
     payload={'domain':domain_name}
     res = requests.get(cmdburl, params=payload)
     hosts=res.json()['object'][0]['servers']
     for host in hosts:
         host_ip=host['ip']
         ips.append(host_ip)
     return ips 




def do_work():
    run('echo "Running stress test..."')

@task
def set_hosts(domain):
    # Update env.hosts instead of calling execute()
    host_list = find_ips_by_domain(domain)
    execute(do_work, hosts=host_list)

# 调用
fab set_hosts:app
fab set_hosts:db

方法二：

#!/usr/bin/env python
#encoding=utf-8
import sys
import os
import requests

from fabric.api import env
from fabric.api import run
from fabric.api import put
from fabric.api import execute
from fabric.api import roles
from fabric.api import parallel
from fabric.api import cd
from fabric.api import task

env.user = 'test'
env.password = 'test'



cmdburl = 'http://cmdb.test.com/test/listServer.do'


def find_ips_by_domain(domain_name):
     ips=[]
     payload={'domain':domain_name}
     res = requests.get(cmdburl, params=payload)
     hosts=res.json()['object'][0]['servers']
     for host in hosts:
         host_ip=host['ip']
         ips.append(host_ip)
     return ips 



@task
def do_work():
    run('echo "Running stress test..."')

@task
def set_hosts(domain):
    # Update env.hosts instead of calling execute()
    env.hosts = find_ips_by_domain(domain)

#调用
fab set_hosts:test.com do_work

上面两种方法的区别是，第二种方法更容易替换执行其他任务

动态生成角色列表

#!/usr/bin/env python
#encoding=utf-8
import sys
import os
import requests

from fabric.api import env
from fabric.api import run
from fabric.api import put
from fabric.api import execute
from fabric.api import roles
from fabric.api import parallel
from fabric.api import cd
from fabric.api import task

env.user = 'test'
env.password = 'test'


cmdburl = 'http://cmdb.test.com/test/listServer.do'

## 根据域名（服务名）查询该域的所有服务器列表
def find_ips_by_domain(domain_name):
     ips=[]
     payload={'domain':domain_name}
     res = requests.get(cmdburl, params=payload)
     hosts=res.json()['object'][0]['servers']
     for host in hosts:
         host_ip=host['ip']
         ips.append(host_ip)
     return ips 

@task
def gener_roles(domain_name):
    ips = find_ips_by_domain(domain_name)  
    ### 动态生成角色列表
    **env.roledefs['ips'] = map(lambda x: x, ips)**
    ### 根据生成的角色列表处理任务
    execute(do_work)

@roles('ips')
def do_work():
    run('echo "Running stress test..."')

执行任务的方式为：

 fab gener_roles:test.com

参考资料

• Using execute with dynamically-set host lists
• http://my.oschina.net/indestiny/blog/290239