Cluster Server and Clients

I configured our nodes with the following hostnames using these steps. Our server is:

3.buhpc.com

The clients are:

buhpc.com
buhpc.com
buhpc.com
buhpc.com
buhpc.com

Installation

On the server, inside the shared folder of our cluster, we will first download the latest version of ganglia. For our cluster, /nfs is the folder with our network file system.

cd /nfs
wget http://downloads.sourceforge.net/project/ganglia/ganglia%20monitoring%20core/3.7.2/ganglia-3.7.2.tar.gz

On the server, we will install dependencies and libconfuse.

yum install freetype-devel rpm-build php httpd libpng-devel libart_lgpl-devel python-devel pcre-devel autoconf automake libtool expat-devel rrdtool-devel apr-devel gcc-c++ make pkgconfig -y
yum install https://dl.fedoraproject.org/pub/epel/7/x86_64/l/libconfuse-2.7-7.el7.x86_64.rpm -y
yum install https://dl.fedoraproject.org/pub/epel/7/x86_64/l/libconfuse-devel-2.7-7.el7.x86_64.rpm -y

#建立rrd数据库
mkdir -p /var/lib/ganglia/rrds/
chown nobody:nobody -R /var/lib/ganglia/rrds/

Now, we will build the rpms from ganglia-3.7.2 on the server.

rpmbuild -tb ganglia-3.7.2.tar.gz

After running rpmbuild, /root/rpmbuild/RPMS/x86_64 contains the generated rpms:

cd /root/rpmbuild/RPMS/x86_64/
yum install *ganglia*.rpm -y

We will remove gmetad because we do not need it on the clients. Send the rest of the rpms to all the clients’ /tmp folder:

cd /root/rpmbuild/RPMS/x86_64/
rm -rf ganglia-gmetad*.rpm
scp *.rpm root@1.buhpc.com:/tmp
scp *.rpm root@2.buhpc.com:/tmp
scp *.rpm root@4.buhpc.com:/tmp
scp *.rpm root@5.buhpc.com:/tmp
scp *.rpm root@6.buhpc.com:/tmp

SSH onto every client and install the rpms that we will need:

ssh root@#.buhpc.com
yum install https://dl.fedoraproject.org/pub/epel/7/x86_64/l/libconfuse-2.7-7.el7.x86_64.rpm -y
yum install https://dl.fedoraproject.org/pub/epel/7/x86_64/l/libconfuse-devel-2.7-7.el7.x86_64.rpm -y
yum install /tmp/*ganglia*.rpm - y

Back on the server, we will adjust the gmetad configuration file:

cd /etc/ganglia
vim gmetad.conf

buhpc will be the name of our cluster. Find the following line and add the name of your cluster and ip address. I am using the subdomain instead of the ip address.

data_source "buhpc" 1 3.buhpc.com

Now, we edit the server’s gmond configuration file.

vim /etc/ganglia/gmond.conf

Make sure that these sections have the following and comment any extra lines you see that are within each section.

cluster {
  name = "buhpc"
  owner = "unspecified"
  latlong = "unspecified"
  url = "unspecified"
}

udp_send_channel {
  host = 1.buhpc.com
  port = 8649
  ttl = 1
}

udp_send_channel {
  host = 2.buhpc.com
  port = 8649
  ttl = 1
}

udp_send_channel {
  host = 3.buhpc.com
  port = 8649
  ttl = 1
}
udp_send_channel {
  host = 4.buhpc.com
  port = 8649
  ttl = 1
}

udp_send_channel {
  host = 5.buhpc.com
  port = 8649
  ttl = 1
}

udp_send_channel {
  host = 6.buhpc.com
  port = 8649
  ttl = 1
}

udp_recv_channel {
  port = 8649
  retry_bind = true
}

Now, SSH into each of the clients and do the following individually. On every client:

vim /etc/ganglia/gmond.conf

We will change the clients’ gmond.conf in the same way as the server’s. Make sure that these sections have the following lines and comment any extra lines you see that are within each section.

cluster {
  name = "buhpc"
  owner = "unspecified"
  latlong = "unspecified"
  url = "unspecified"
}

udp_send_channel {
  host = 1.buhpc.com
  port = 8649
  ttl = 1
}

udp_send_channel {
  host = 2.buhpc.com
  port = 8649
  ttl = 1
}

udp_send_channel {
  host = 3.buhpc.com
  port = 8649
  ttl = 1
}
udp_send_channel {
  host = 4.buhpc.com
  port = 8649
  ttl = 1
}

udp_send_channel {
  host = 5.buhpc.com
  port = 8649
  ttl = 1
}

udp_send_channel {
  host = 6.buhpc.com
  port = 8649
  ttl = 1
}

udp_recv_channel {
  port = 8649
  retry_bind = true
}

We will start gmond on the clients for monitoring.

chkconfig gmond on
systemctl start gmond

然后，安装ganglia-web 3.7.1

wget http://superb-sea2.dl.sourceforge.net/project/ganglia/ganglia-web/3.7.1/ganglia-web-3.7.1.tar.gz
tar zxvf  ganglia-web-3.7.1.tar.gz
cd  ganglia-web-3.7.1
vim Makefile
      # Location where gweb should be installed to (excluding conf, dwoo dirs).
      GDESTDIR = /var/www/html/ganglia

      # Gweb statedir (where conf dir and Dwoo templates dir are stored)
      GWEB_STATEDIR = /var/lib/ganglia-web

      # Gmetad rootdir (parent location of rrd folder)
      GMETAD_ROOTDIR = /var/lib/ganglia

      # User by which your webserver is running
      APACHE_USER =  apache

 make install

Next, we will want to disable SELinux. Change SELINUX inside /etc/sysconfig/selinux from enforcing to disabled. Then, restart the server node.

vim /etc/sysconfig/selinux
SELINUX=disabled
#如果 SELINUX本就是disable，不必reboot
reboot
Now, on the server, we’ll open the correct ports on the firewall.

#如果 firewall 没有打开，systemctl service firewalld
firewall-cmd --permanent --zone=public --add-service=http
firewall-cmd --permanent --zone=public --add-port=8649/udp
firewall-cmd --permanent --zone=public --add-port=8649/tcp
firewall-cmd --permanent --zone=public --add-port=8651/tcp
firewall-cmd --permanent --zone=public --add-port=8652/tcp
firewall-cmd --reload

On the server, we will now start httpd, gmetad, and gmond.

chkconfig httpd
chkconfig gmetad on
chkconfig gmond on
systemctl start httpd
systemctl start gmetad
systemctl start gmond

Visit http://3.buhpc.com/ganglia to see Ganglia’s monitoring. You should see something like this:

ganglia 安装

Cluster Server and Clients

Installation