Pacemaker is the central component to make a scalable High Available Resource Manager on Linux, primarily started on RedHat. Now available on most of Linux distros. It allows you to connect a group of nodes to work together as a cluster, therefore to achive detecting, recovering from node and resource-level failures by making use of the messaging and membership capabilitie.

There are two popular command line tools to manage cluster configuration. One is pcs and the other is crmsh. In this serial documents, it's based on pcs.

Most HA document and example on the web are web application based, use GFS2 as share filesystem and DRDB for content replication. What about other applications? Do they all have to use GFS2 and DRDB? My answer is no.


In this serial documents, I'll be focus on using pacemaker to manage SAN based filesystems as resource, on top of this, you can choose to manage applications like cluster filesystem, database, or applications use filesystems as , such as file server etc.. No GFS2 and DRDB.

During the testing and writing the document, I used the following as references, but not a copy or modification from any of them.

Custer environment

The test bed I chose are three servers(4 CPUS, 8GB memory each), each server has its own RSA interface for fencing. Total 6 SAN storage LUNs available for 3 servers.

The OS is RHEL6.6, kernel : 2.6.32-504.23.4.el6.x86_64

Package installation

# yum install pcs fence-agents pacemaker cman psmisc policycoreutils-python   
Package psmisc-22.6-19.el6_5.x86_64 already installed and latest version

 Package                        Arch        Version            Repository    Size
 cman                           x86_64     sl          448 k
 fence-agents                   x86_64      3.1.5-48.el6        sl          165 k
 pacemaker                      x86_64      1.1.12-4.el6        sl          448 k
 pcs                            x86_64      0.9.123-9.el6_6.2   sl-errata   5.7 M
 policycoreutils-python         x86_64      2.0.83-19.47.el6    sl          345 k
Installing for dependencies:    x86_64      0.16.2-75.el6_6.1   sl-errata   51 k
clusterlib                     x86_64     sl          104 k
 corosync                       x86_64      1.4.7-1.el6         sl          211 k
 corosynclib                    x86_64      1.4.7-1.el6         sl          188 k
 modcluster                     x86_64      0.16.2-29.el6       sl          190 k
 openais                        x86_64      1.1.1-7.el6         sl          191 k
 openaislib                     x86_64      1.1.1-7.el6         sl         81 k
 pacemaker-cli                  x86_64      1.1.12-4.el6        sl         190 k
 pacemaker-cluster-libs         x86_64      1.1.12-4.el6        sl          79 k
 pacemaker-libs                 x86_64      1.1.12-4.el6        sl          459 k
 resource-agents                x86_64      3.9.5-12.el6        sl          355 k
 ricci                          x86_64      0.16.2-75.el6_6.1   sl-errata  630 k
Transaction Summary====================================================================
Install      17 Package(s)

Total download size: 9.7 M
Installed size: 25 M
Is this ok [y/N]: y

pcs is the command line tool to config and manage the cluster on Redhat, the alternative tool is crmsh

Preparation before start configuring

Newtorking, open port access between cluster nodes. For following ports,

  • For TCP: Ports 2224, 3121, 21064
  • For UDP: Ports, 5405

For eample, open port connection for ip subset 192.168.1, port 2224

# iptables -A INPUT -m state --state NEW -m tcp -p tcp -s --dport 2224 -j ACCEPT

More iptable rules, examples, see Linux iptables useful examples

Cluster configuration files

Pacemaker use cib xml configuration file to store and manage the cluster configuration. But it's not recommanded to modify the xml file to manage pacemaker, actually in most of cases, you should not edit the xml configuration files. Use the pcs or pcsd interface.

The cluster.conf file provides the cluster parameters used by corosync, the cluster manager that Pacemaker is built on.

The cib.xml file is an XML file that represents both the cluster’s configuration and current state of all resources in the cluster. This file is used by Pacemaker's Cluster Information Base (CIB). The contents of the the CIB are automatically kept in sync across the entire cluster

More cluster configuration

This is a serial hands on documents, you can check them by order from scratch, or just one sector.

Cluster creation , deletion, add and remove a node from cluster

Cluster fencing

Cluster resource creation, management

Cluster command line pcs reference