HA_Cluster: corosync+pacemaker+drbd+mysql

本文主要讲述HA_Cluster: corosync+pacemaker+drbd+mysql 的CRM资源配置, 各个软件的安装配置不再介绍

CRM配置DRBD资源:

1.基本配置:

crm(live)configure# property stonith-enabled=false #关闭stonith
crm(live)configure# property no-quorum-policy=ignore #不具备法定票数时的行为: 忽略
crm(live)configure# rsc_defaults resource-stickiness=100 #默认资源粘性为100
crm(live)configure# verify
crm(live)configure# commit

2.添加DRBD资源:

crm(live)configure# primitive drbd_data ocf:linbit:drbd params drbd_resource="data" op monitor role="Master" interval="50" timeout="30" op monitor role="Slave" interval
interval-origin=  interval=        
crm(live)configure# primitive drbd_data ocf:linbit:drbd params drbd_resource=data op monitor role=Master interval=50 timeout=30 op monitor role=Slave interval=60 timeout=30 op start timeout=240 op stop timeout=100
crm(live)configure# verify
crm(live)configure# ms ms_drbd_data drbd_data meta master-max=1 master-node-max=1 clone-max=2 clone-node-max=1 notify=true
crm(live)configure# verify
crm(live)configure# commit

PS: heartbeat 和 linbit 都提供了drbd 的 RA ,本人实验时发现使用 heartbeat 提供的 RA 会出现无法选举 Master 节点的情况, 此处建议使用 linbit 提供的RA
如有人知道为什么 heartbeat 的 RA 选不出 Master 节点, 愿闻其详

测试DRBD服务:

root@drbd1:~# crm status
Last updated: Sun May 22 22:04:35 2016
Last change: Mon May 23 06:02:13 2016 via cibadmin on drbd1
Stack: corosync
Current DC: drbd1 (168495249) - partition with quorum
Version: 1.1.10-42f2063
2 Nodes configured
2 Resources configured
 
 
Online: [ drbd1 drbd2 ]
 
Master/Slave Set: ms_drbd_data [drbd_data]
     Masters: [ drbd1 ]
     Slaves: [ drbd2 ]
root@drbd1:~# crm node standby <strong>#离线drbd1</strong>
root@drbd1:~# crm status
Last updated: Mon May 23 06:07:22 2016
Last change: Mon May 23 06:07:14 2016 via crm_attribute on drbd1
Stack: corosync
Current DC: drbd1 (168495249) - partition with quorum
Version: 1.1.10-42f2063
2 Nodes configured
2 Resources configured
 
 
Node drbd1 (168495249): standby
Online: [ drbd2 ]
 
Master/Slave Set: ms_drbd_data [drbd_data]
     Masters: [ drbd2 ]
     Stopped: [ drbd1 ]
root@drbd1:~# crm node online <strong>#上线drbd1</strong>
root@drbd1:~# crm status
Last updated: Mon May 23 06:09:11 2016
Last change: Mon May 23 06:09:09 2016 via crm_attribute on drbd1
Stack: corosync
Current DC: drbd1 (168495249) - partition with quorum
Version: 1.1.10-42f2063
2 Nodes configured
2 Resources configured
 
 
Online: [ drbd1 drbd2 ]
 
Master/Slave Set: ms_drbd_data [drbd_data]
     Masters: [ drbd2 ]
     Slaves: [ drbd1 ]
root@drbd2:~# crm node standby #离线drbd2
root@drbd2:~# crm node online #上线drbd2
root@drbd2:~# crm status
Last updated: Sun May 22 22:11:47 2016
Last change: Mon May 23 06:11:18 2016 via crm_attribute on drbd2
Stack: corosync
Current DC: drbd1 (168495249) - partition with quorum
Version: 1.1.10-42f2063
2 Nodes configured
2 Resources configured
 
 
Online: [ drbd1 drbd2 ]
 
Master/Slave Set: ms_drbd_data [drbd_data]
     Masters: [ drbd1 ]
     Slaves: [ drbd2 ]
-------------正常---------------

配置Filesystem资源:

crm(live)configure# primitive mount ocf:heartbeat:Filesystem params device=/dev/drbd0 directory=/var/lib/mysql fstype=ext3 op start timeout=60 op stop timeout=60
crm(live)configure# verify
crm(live)configure# colocation mount_with_ms_drbd_data inf: mount ms_drbd_data:Master
crm(live)configure# order mount_after_ms_drbd_data Mandatory: ms_drbd_data:promote mount:start
crm(live)configure# verify
crm(live)configure# commit

测试Filesystem资源:

root@drbd2:~# crm status
Last updated: Mon May 23 20:28:19 2016
Last change: Mon May 23 20:27:37 2016 via crm_attribute on drbd1
Stack: corosync
Current DC: drbd1 (168495249) - partition with quorum
Version: 1.1.10-42f2063
2 Nodes configured
3 Resources configured
 
 
Online: [ drbd1 drbd2 ]
 
Master/Slave Set: ms_drbd_data [drbd_data]
     Masters: [ drbd2 ]
     Slaves: [ drbd1 ]
 mount    (ocf::heartbeat:Filesystem):    Started drbd2 #现在drbd0挂载在drbd2上
root@drbd2:~# crm node standby
root@drbd2:~# crm node online
root@drbd2:~# crm status
Last updated: Mon May 23 20:28:31 2016
Last change: Mon May 23 20:28:28 2016 via crm_attribute on drbd2
Stack: corosync
Current DC: drbd1 (168495249) - partition with quorum
Version: 1.1.10-42f2063
2 Nodes configured
3 Resources configured
 
 
Node drbd2 (168495262): standby
Online: [ drbd1 ]
 
Master/Slave Set: ms_drbd_data [drbd_data]
     Masters: [ drbd1 ]
     Stopped: [ drbd2 ]
 mount    (ocf::heartbeat:Filesystem):    Started drbd1 #成功转移到drbd1

配置mysql(此处直接使用apt-get安装)资源:

crm(live)configure# primitive mysqld lsb:mysql
crm(live)configure# primitive mysql_ip ocf:heartbeat:IPaddr params ip=10.11.8.200 cidr_netmask=23
crm(live)configure# verify
crm(live)configure# colocation mysql_ip_with_mount inf: mysql_ip mount
crm(live)configure# colocation mysqld_with_mount inf: mysqld mount
crm(live)configure# order mysqld_after_mount Mandatory: mount mysqld
crm(live)configure# verify
crm(live)configure# commit

查看CRM配置:

node $id="168495249" drbd1 \
        attributes standby="off"
node $id="168495262" drbd2 \
        attributes standby="off"
primitive drbd_data ocf:linbit:drbd \
        params drbd_resource="data" \
        op monitor role="Master" interval="50" timeout="30" \
        op monitor role="Slave" interval="60" timeout="30" \
        op start timeout="240" interval="0" \
        op stop timeout="100" interval="0"
primitive mount ocf:heartbeat:Filesystem \
primitive mount ocf:heartbeat:Filesystem \
        params device="/dev/drbd0" directory="/var/lib/mysql" fstype="ext3" \
        op start timeout="60" interval="0" \
        op stop timeout="60" interval="0" \
        meta target-role="Started"
primitive mysql_ip ocf:heartbeat:IPaddr \
        params ip="10.11.8.200" cidr_netmask="23"
primitive mysqld lsb:mysql
ms ms_drbd_data drbd_data \
        meta master-max="1" master-node-max="1" clone-max="2" clone-node-max="1" notify="true" target-role="Started"
colocation mount_with_ms_drbd_data inf: mount ms_drbd_data:Master
colocation mysql_ip_with_mount inf: mysql_ip mount
colocation mysqld_with_mount inf: mysqld mount
order mount_after_ms_drbd_data Mandatory: ms_drbd_data:promote mount:start
order mysqld_after_mount Mandatory: mount mysqld
property $id="cib-bootstrap-options" \
        dc-version="1.1.10-42f2063" \
        cluster-infrastructure="corosync" \
        stonith-enabled="false" \
        no-quorum-policy="ignore" \
        last-lrm-refresh="1464019270"
rsc_defaults $id="rsc-options" \
        resource-stickiness="100"

修改mysql监听所有地址:

root@drbd1:~# vim /etc/mysql/my.cnf
#bind-address           = 127.0.0.1 #注释这一行

测试服务:

root@drbd1:~# crm status
Last updated: Tue May 24 00:55:04 2016
Last change: Tue May 24 00:42:45 2016 via crmd on drbd1
Stack: corosync
Current DC: drbd1 (168495249) - partition with quorum
Version: 1.1.10-42f2063
2 Nodes configured
5 Resources configured
 
 
Online: [ drbd1 drbd2 ]
 
Master/Slave Set: ms_drbd_data [drbd_data]
     Masters: [ drbd1 ]
     Slaves: [ drbd2 ]
 mount    (ocf::heartbeat:Filesystem):    Started drbd1
 mysqld    (lsb:mysql):    Started drbd1
 mysql_ip    (ocf::heartbeat:IPaddr):    Started drbd1

在drbd2上连接测试:

root@drbd2:~# mysql -h10.11.8.200 -ushiina -p
Enter password:
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 40
Server version: 5.6.30-0ubuntu0.14.04.1 (Ubuntu)
 
Copyright (c) 2000, 2016, Oracle and/or its affiliates. All rights reserved.
 
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
 
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
 
mysql> show databases;
+---------------------+
| Database            |
+---------------------+
| information_schema  |
| #mysql50#lost+found |
| mysql               |
| performance_schema  |
+---------------------+
4 rows in set (0.01 sec)
 
mysql> create database ha_test;
Query OK, 1 row affected (0.00 sec)

离线drbd1后再drbd1上连接测试:

root@drbd1:~# crm status
Last updated: Mon May 23 21:13:24 2016
Last change: Mon May 23 21:13:22 2016 via cibadmin on drbd1
Stack: corosync
Current DC: drbd1 (168495249) - partition with quorum
Version: 1.1.10-42f2063
2 Nodes configured
5 Resources configured
 
 
Online: [ drbd1 drbd2 ]
 
Master/Slave Set: ms_drbd_data [drbd_data]
     Masters: [ drbd2 ]
     Slaves: [ drbd1 ]
 mount    (ocf::heartbeat:Filesystem):    Started drbd2
 mysql_ip    (ocf::heartbeat:IPaddr):    Started drbd2
 mysqld    (lsb:mysql):    Started drbd2
root@drbd1:~# mysql -h10.11.8.200 -ushiina -p
Enter password:
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 40
Server version: 5.6.30-0ubuntu0.14.04.1 (Ubuntu)
 
Copyright (c) 2000, 2016, Oracle and/or its affiliates. All rights reserved.
 
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
 
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
 
mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| ha_test            |
| lost+found         |
| mysql              |
| performance_schema |
+--------------------+
5 rows in set (0.00 sec)