用 keepalived 搭建高可用集群之 IP Failover

头像
edwingeng
    阅读 7 分钟

    运行环境

    • CentOS 7
    • 云主机：Linode

    配置节点

    在同一机房购买两个 Linode 节点，规格随意。安装 CentOS 7，然后去 Settings 页面设置 Linode Label，一个叫 node-alice，另一个叫 node-bob。

    获取内网 IP：打开两个节点的 Remote Access 页面，点 Add a Private IP。

    记下两个节点的 Private IP，下文分别称为 alice-private-ip 和 bob-private-ip，在配置中看到时请自行替换。

    配置 eth0

    去 Dashboard 启动两个节点，启动完成后，点 My CentOS 7 Profile 后面的 Edit，在最下方找到 Auto-configure Networking，选 No，然后 Save Changes。

    查看 eth0 配置（可选）

    cat /etc/sysconfig/network-scripts/ifcfg-eth0

    运行结果：

    # Generated by Linode Network Helper
# Sun Jun 11 12:51:56 2017 UTC
#
# This file is automatically generated on each boot with your Linode's
# current network configuration. If you need to modify this file, please
# first disable the 'Auto-configure Networking' setting within your Linode's
# configuration profile:
#  - https://manager.linode.com/linodes/config/node-bob?id=4317235
#
# For more information on Network Helper:
#  - https://www.linode.com/docs/platform/network-helper
#
# A backup of the previous config is at /etc/sysconfig/network-scripts/.ifcfg-eth0.linode-last
# A backup of the original config is at /etc/sysconfig/network-scripts/.ifcfg-eth0.linode-orig
#
# /etc/sysconfig/network-scripts/ifcfg-eth0

# For full descriptions of what these switches do,
# and what the interface's defaults are, see
# /usr/share/doc/initscripts-*/sysconfig.txt
DEVICE="eth0"
ONBOOT="yes"

# "bootp" and "dhcp" are for dhcp, anything else
# is for a static configuration. "none" is given
# by sysconfig.txt so we're using it.
BOOTPROTO="none"

# Use hardware-based IPv6 addresses, no privacy extensions.
IPV6_ADDR_GEN_MODE="eui64"

# Since we want a static configuration, we're specifying DNS
# addresses in this file for NetworkManager. "No" here tells
# NM to use them when BOOTPROTO!=dhcp.
PEERDNS="no"

DOMAIN=members.linode.com
GATEWAY0=96.126.123.1

# resolvconf doesn't recognize more than 3 nameservers.
DNS1=173.255.199.5
DNS2=66.228.53.5
DNS3=96.126.122.5


IPADDR0=96.126.123.152

# Sysconfig.txt says that PREFIX takes precedence over
# NETMASK when both are present. Since both aren't needed,
# we'll go with PREFIX since it seems to be preferred.
PREFIX0=24

IPADDR1=192.168.130.9
PREFIX1="17"

    申请 Floating IP

    打开 node-alice 的 Remote Access 页面，点 IP Add 链接跳至 Support 页面，Regarding 选 Linode: node-alice，Description 可以参照下面写，最后点 Open Ticket。

    Hi there,
    I need a floating IP for building a keepalived pair. Can I have that?
    Thank you for your help.

    Linode 工作人员回复后，回到 node-alice 的 Remote Access 页面，点 IP Add 链接跳至 IP 购买页面，价格是一美元一个月，买。

    再回到 node-alice 的 Remote Access 页面，你的第二个公网 IP 已经到位，记下这个 IP，下文用 the-floating-ip 表示。

    IP Failover

    打开 node-bob 的 Remote Access 页面，点 IP Failover，选中前面新买的 the-floating-ip，最后 Save Changes。

    CentOS

    下面这些事在 node-bob 上也要做，做的时候，用 bob 替换 alice。

    打开 Remote Access 页面，找到 SSH Access，复制后面的 SSH 命令，登录节点。

    更新系统

    yum update -y

    设置 hostname

    hostnamectl set-hostname alice

    vim /etc/hosts，添加如下内容

    127.0.0.1   alice

    设置公钥（可选）

    mkdir -p ~/.ssh && sudo chmod -R 700 ~/.ssh/
vim ~/.ssh/authorized_keys
...

    开启网络选项

    echo 'net.ipv4.ip_nonlocal_bind=1' >> /etc/sysctl.conf
echo 'net.ipv4.ip_forward=1' >> /etc/sysctl.conf
sysctl -p

    重启网络服务

    systemctl restart network

    安装工具软件

    yum groupinstall -y 'Development Tools'
yum install -y openssl-devel
yum install -y wget
yum install -y telnet

    Nginx

    下面这些事在 node-bob 上也要做，做的时候，用 bob 替换 alice。

    安装 Nginx

    yum install -y epel-release
yum install -y nginx

    vim /etc/nginx/nginx.conf，在 listen 后添加一行

    listen       the-floating-ip:80 default_server;

    启动 Nginx

    systemctl enable nginx
systemctl start  nginx

    vim /usr/share/nginx/html/index.html，把 Welcome to nginx 改成 Welcome to alice。

    Keepalived - alice

    安装 keepalived

    cd ~ && wget http://www.keepalived.org/software/keepalived-1.3.5.tar.gz
tar zxvf keepalived-* && cd keepalived-* && ./configure && make && sudo make install

    创建配置文件目录

    mkdir -p /etc/keepalived

    vim /etc/keepalived/keepalived.conf

    ! Configuration File for keepalived

global_defs {
   router_id LVS_DEVEL
}

vrrp_instance VI_1 {
    state BACKUP
    nopreempt
    interface eth0
    virtual_router_id 51
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 123456
    }
    unicast_src_ip alice-private-ip
    unicast_peer {
        bob-private-ip
    }
    virtual_ipaddress {
        the-floating-ip
    }
}

    查看 service 配置

    cat /usr/lib/systemd/system/keepalived.service

    运行结果：

    [Unit]
Description=LVS and VRRP High Availability Monitor
After=syslog.target network-online.target

[Service]
Type=forking
PIDFile=/usr/local/var/run/keepalived.pid
KillMode=process
EnvironmentFile=-/usr/local/etc/sysconfig/keepalived
ExecStart=/usr/local/sbin/keepalived $KEEPALIVED_OPTIONS
ExecReload=/bin/kill -HUP $MAINPID

[Install]
WantedBy=multi-user.target

    把上面的 PIDFile=/usr/local/var/run/keepalived.pid 改成 PIDFile=/run/keepalived.pid。

    启动 keepalived

    systemctl enable keepalived.service
systemctl start  keepalived.service

    重新加载配置（必要时）

    systemctl reload keepalived.service

    Keepalived - bob

    重复上面所有操作，除了 /etc/keepalived/keepalived.conf 不一样

    ! Configuration File for keepalived

global_defs {
   router_id LVS_DEVEL
}

vrrp_instance VI_1 {
    state BACKUP
    nopreempt
    interface eth0
    virtual_router_id 51
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 123456
    }
    unicast_src_ip bob-private-ip
    unicast_peer {
        alice-private-ip
    }
    virtual_ipaddress {
        the-floating-ip
    }
}

    测试

    用浏览器访问 http://the-floating-ip，此时可能显示 alice，也可能显示 bob，和服务启动时间有关。

    假设现在显示 alice，关闭 node-alice，刷新网页，网页显示改为 bob。

    启动 node-alice，刷新网页，应该还是 bob。

    接下来关闭 node-bob，刷新网页，网页应换回 alice。

    最后看看日志

    tail -50 /var/log/messages

    可以找到 bob 转为 MASTER 的记录

    Jun 11 09:16:59 bob Keepalived_vrrp[3502]: VRRP_Instance(VI_1) Transition to MASTER STATE
Jun 11 09:17:00 bob Keepalived_vrrp[3502]: VRRP_Instance(VI_1) Entering MASTER STATE
Jun 11 09:17:00 bob Keepalived_vrrp[3502]: VRRP_Instance(VI_1) setting protocol VIPs.

    监控 Nginx

    alice & bob - vim /root/keepalived-check-nginx.sh

    #!/usr/bin/env bash

if [[ `pidof nginx` == '' ]]; then
    flock -n /tmp/restart-keepalived.lock bash -c \
        "systemctl stop keepalived; sleep 30; systemctl start keepalived; sleep 3" &
fi

    alice - vim /etc/keepalived/keepalived.conf

    ! Configuration File for keepalived

global_defs {
   router_id LVS_DEVEL
}

vrrp_script check_nginx {
    script "/root/keepalived-check-nginx.sh"
    interval 1
}

vrrp_instance VI_1 {
    state BACKUP
    nopreempt
    interface eth0
    virtual_router_id 51
    priority 120
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 123456
    }
    unicast_src_ip alice-private-ip
    unicast_peer {
        bob-private-ip
    }
    virtual_ipaddress {
        the-floating-ip
    }
    track_script {
        check_nginx
    }
}

    只增加了两块内容：vrrp_script check_nginx 和 track_script。

    bob - vim /etc/keepalived/keepalived.conf

    ! Configuration File for keepalived

global_defs {
   router_id LVS_DEVEL
}

vrrp_script check_nginx {
    script "/root/keepalived-check-nginx.sh"
    interval 1
}

vrrp_instance VI_1 {
    state BACKUP
    nopreempt
    interface eth0
    virtual_router_id 51
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 123456
    }
    unicast_src_ip bob-private-ip
    unicast_peer {
        alice-private-ip
    }
    virtual_ipaddress {
        the-floating-ip
    }
    track_script {
        check_nginx
    }
}

    alice & bob - 重新加载配置

    systemctl reload keepalived

    关闭 Nginx 试试，看 Floating IP 会不会转到另一台机器上

    systemctl stop nginx

    结束

    写到这里，试验都做完了。你如果还没试过 Linode 主机，可以尝试一下，做 SOCKS5 代理、VPN 很好用，5 美元一个月，按小时计费，新人还送点钱。

    Linode 传送门：SSD Cloud Hosting & Linux Servers

    keepalived高可用
    edwingeng
    115 声望2 粉丝
    « 上一篇
    用 systemd 配置一个简单的自动重启服务
    下一篇 »
    一个比 UUID 快百倍的唯一 ID 生成器

    引用和评论

    推荐阅读
    头像
    doublejump - 快速、简洁的一致性哈希库，Google Jump 算法的改进版

    edwingeng阅读 3.9k

    头像
    Veritas InfoScale 8.0 (Unix, Linux, Windows) - 高可用性和灾难恢复解决方案

    sysin阅读 121

    0 条评论
    得票最新