There are two popular command line tools to manage cluster configuration. One is pcs and the other is crmsh. In this serial documents, it's based on pcs.
Most HA document and example on the web are web application based, use GFS2 as share filesystem and DRDB for content replication. What about other applications? Do they all have to use GFS2 and DRDB? My answer is no.
In this serial documents, I'll be focus on using pacemaker to manage SAN based filesystems as resource, on top of this, you can choose to manage applications like cluster filesystem, database, or applications use filesystems as , such as file server etc.. No GFS2 and DRDB.
The test bed I chose are three servers(4 CPUS, 8GB memory each), each server has its own RSA interface for fencing. Total 6 SAN storage LUNs available for 3 servers.
The OS is RHEL6.6, kernel : 2.6.32-504.23.4.el6.x86_64
# yum install pcs fence-agents pacemaker cman psmisc policycoreutils-python
Package psmisc-22.6-19.el6_5.x86_64 already installed and latest version
Package Arch Version Repository Size
cman x86_64 184.108.40.206-68.el6 sl 448 k
fence-agents x86_64 3.1.5-48.el6 sl 165 k
pacemaker x86_64 1.1.12-4.el6 sl 448 k
pcs x86_64 0.9.123-9.el6_6.2 sl-errata 5.7 M
policycoreutils-python x86_64 2.0.83-19.47.el6 sl 345 k
Installing for dependencies: x86_64 0.16.2-75.el6_6.1 sl-errata 51 k
clusterlib x86_64 220.127.116.11-68.el6 sl 104 k
corosync x86_64 1.4.7-1.el6 sl 211 k
corosynclib x86_64 1.4.7-1.el6 sl 188 k
modcluster x86_64 0.16.2-29.el6 sl 190 k
openais x86_64 1.1.1-7.el6 sl 191 k
openaislib x86_64 1.1.1-7.el6 sl 81 k
pacemaker-cli x86_64 1.1.12-4.el6 sl 190 k
pacemaker-cluster-libs x86_64 1.1.12-4.el6 sl 79 k
pacemaker-libs x86_64 1.1.12-4.el6 sl 459 k
resource-agents x86_64 3.9.5-12.el6 sl 355 k
ricci x86_64 0.16.2-75.el6_6.1 sl-errata 630 k
Install 17 Package(s)
Total download size: 9.7 M
Installed size: 25 M
Is this ok [y/N]: y
pcs is the command line tool to config and manage the cluster on Redhat, the alternative tool is crmsh
Preparation before start configuring
Newtorking, open port access between cluster nodes. For following ports,
For TCP: Ports 2224, 3121, 21064
For UDP: Ports, 5405
For eample, open port connection for ip subset 192.168.1, port 2224
# iptables -A INPUT -m state --state NEW -m tcp -p tcp -s 192.168.1.0/24 --dport 2224 -j ACCEPT
More iptable rules, examples, see Linux iptables useful examples
Cluster configuration files
Pacemaker use cib xml configuration file to store and manage the cluster configuration. But it's not recommanded to modify the xml file to manage pacemaker, actually in most of cases, you should not edit the xml configuration files. Use the pcs or pcsd interface.
file provides the cluster parameters used by corosync, the cluster manager that Pacemaker is built on.
The cib.xml file is an XML file that represents both the cluster’s configuration and current state of all resources in the cluster. This file is used by Pacemaker's Cluster Information Base (CIB). The contents of the the CIB are automatically kept in sync across the entire cluster
More cluster configuration
This is a serial hands on documents, you can check them by order from scratch, or just one sector.