Tag Archives: cluster


Example of GlusterFS configuration on 2 servers with Block Storage attached.

This setup is suggested for TESTING purposes only. In a production environment please verify performances.

Create a separate network and map IP/servers' names in /etc/hosts
Append to /etc/hosts
# GlusterFS     gfs01     gfs02

>> On BOTH nodes:

yum update
wget -P /etc/yum.repos.d http://download.gluster.org/pub/gluster/glusterfs/LATEST/CentOS/glusterfs-epel.repo
yum -y install parted lvm2 xfsprogs glusterfs glusterfs-fuse glusterfs-server
grep ^exclude /etc/yum.conf

[root@gfs01 sysconfig]# grep ^exclude /etc/yum.conf
(in [main] section)

parted -s -- /dev/xvdb mktable gpt
parted -s -- /dev/xvdb mkpart primary 2048s 100%
parted -s -- /dev/xvdb set 1 lvm on
partx -a /dev/xvdb
pvcreate /dev/xvdb1 
vgcreate vggfs01 /dev/xvdb1 

lvcreate -l 100%VG -n gbrick1 vggfs01
mkfs.xfs -i size=512 /dev/vggfs01/gbrick1
echo '/dev/vggfs01/gbrick1 /data/gluster/gvol0 xfs inode64,nobarrier 0 0' >> /etc/fstab
mkdir -p /data/gluster/gvol0
mount /data/gluster/gvol0
mkdir -p /data/gluster/gvol0/brick1

/bin/systemctl start glusterd.service
/bin/systemctl status glusterd.service
systemctl enable glusterd.service

>> On NODE2
gluster peer probe gfs01
gluster peer status
gluster pool list

>> On NODE1
gluster peer probe gfs02
gluster peer status
gluster pool list

gluster volume create gvol0 replica 2 transport tcp gfs01:/data/gluster/gvol0/brick1 gfs02:/data/gluster/gvol0/brick1
gluster volume start gvol0
gluster volume info gvol0

gluster volume set gvol0 performance.cache-refresh-timeout 30
gluster volume set gvol0 performance.io-thread-count 32
gluster volume set gvol0 performance.cache-size 1073741824
gluster volume info gvol0


TO MOUNT - Fuse (HA)
=> nodes need to be connected to the same Cloud Network

Append to /etc/hosts
# GlusterFS     gfs01     gfs02

yum -y install glusterfs glusterfs-fuse

modprobe fuse

echo 'gfs01:/gvol0 /mnt/gluster/gvol0 glusterfs defaults,backupvolfile-server=gfs02,_netdev 0 0' >> /etc/fstab

mkdir -p /mnt/gluster/gvol0

mount /mnt/gluster/gvol0

It seems that Debian 7 cannot have GlusterFS 3.7 but only from Debian 8.


Extra source: http://matty.digital/gluster

Extra commands
# gluster volume remove-brick gvol0 gfs02:/data/gluster/gvol0/brick1 force
# gluster peer detach gfs02
# gluster peer detach gfs01
# gluster peer probe gfs01

Puppet – Let’s start

Puppet is a quite powerful configuration manager tool which allows you to configure automatically hosts and keep configurations consistence.

I did some tests using 3 VMs:

  • puppetmaster (server)
  • puppetagent01 (client)
  • puppetagent02 (client)

Of course, most of the work is done on puppetmaster server. On the last two machines you will simply see the outcome of the configurations that you’re going do set on puppetmaster.

Important: all the machines have to be able to communicate between each others. Please make sure DNS is working or set local names/IPs in /etc/hosts file, and do some ping tests before proceeding.

Client setup

On each puppetagent machine, just install the package puppet

apt-get install puppet

By default, the client will look for a host called “puppet” on the network.
If your DNS/hosts file doesn’t have this entry, and it can’t be resolved, you can manually set the name of the puppetmaster in /etc/puppet/puppet.conf file, adding this line under [main] section:


Now, no more configuration is required from the client side. Just edit /etc/default/puppet to start at boot time and start the service.

# Defaults for puppet - sourced by /etc/init.d/puppet

# Start puppet on boot?

# Startup options


service puppet start

Starting the service, will make automatically a request to the server to be added under his control.

If you want to do some tests, you can eventually use the following command to run puppet only once. This will also force the polling updates, which by default runs every 30 minutes.

puppet agent --no-daemonize --onetime --verbose

You can repeat all these steps on the second client machine.

Server setup

apt-get install puppetmaster

Check if the service is running, otherwise, start it up.

Sign clients’ certificates on the server side

Puppet uses this client/server certificate sign system to add/remove hosts from being managed by the server.

To see who has requested to be “controlled” use this command:

puppet cert --list

This will show all the hosts waiting to be added under puppetmaster server.

puppet cert --sign

This command will add the host.

Puppetmaster configuration files

The main configuration file is /etc/puppet/manifests/site.pp

Inside manifests folder, I’ve created a subfolder called classes with extra definitions (content of these files is showed later in this post).

/etc/puppet/manifests# tree
|___ classes
|   |___ apache.pp
|   |___ mysite.pp
|   |___ ntpd.pp
|   |___ packages.pp
|___ site.pp


import 'classes/*.pp'
# This add all the custom .pp files into classes folder
class puppettools {
# Creates a file, setting permissions and content
        file { '/usr/local/sbin/puppet_once.sh':
                owner => root, group => root, mode => 755,
                content => "#!/bin/sh\npuppet agent --no-daemonize --onetime --verbose $1\n",
# Install (if not present) some puppet modules required for 'vimconf' class
        exec { "install_puppet_module":
        command => "puppet module install puppetlabs-stdlib",
        path => [ "/bin", "/sbin", "/usr/bin", "/usr/sbin",
              "/usr/local/bin", "/usr/local/sbin" ],
        onlyif  => "test `puppet module list | grep puppetlabs-stdlib | wc -l` -eq 0"

class vimconf {
# Modify vimrc conf file, enabling syntax on
        file_line { 'vim_syntax_on':
        path  => '/etc/vim/vimrc',
        match => '^.*syntax on.*$',
        line  => 'syntax on',

node  default {
# this will be applied to all nodes without specific node definitions
        include packages
        include vimconf
        include ntp
        include puppettools

node  'puppetagent01' inherits default {
# this specific node, gets all the default classes PLUS some extras
        include mysite

Here the content of the single files .pp in classes folder:

class apache {
	package { 'apache2-mpm-prefork':
		ensure => installed

	service { 'apache2':
		ensure => running,
		hasstatus => true,
		hasrestart => true,


class mysite {

	include apache

	file { '/etc/apache2/sites-available/mysite':
		owner => root, group => root, mode => 0644,
		source => "puppet:///files/mysite/mysite_apache.conf",

	file {'/var/www/mysite.localdomain':
		ensure => directory,

	file {'/var/www/mysite.localdomain/index.html':
                owner => root, group => www-data, mode => 0755,
                source => "puppet:///files/mysite/index.html",

	 exec {'/usr/sbin/a2dissite * ; /usr/sbin/a2ensite mysite':
            	onlyif => '/usr/bin/test -e /etc/apache2/sites-available/mysite',
		notify => Service['apache2'],


class ntp {
		package { ntp: ensure => present }
		file { "/etc/ntp.conf":
			owner	 => root,
			group	 => root,
			mode	=> 444,
			backup => false,
			source	=> "puppet:///files/etc/ntp.conf",
			require => Package["ntp"],
                        notify  => Service["ntp"],
		service { "ntp":
			enable => true ,
			ensure => running,
			subscribe => [Package[ntp], File["/etc/ntp.conf"],],


class packages  {
        Package { ensure => "installed" }

        package { "screen": }
        package { "dselect": }
        package { "vim": }
        package { "curl": }


It’s important to remember to NOT duplicate entries.
For example, in this case, we have a specific file where we have setup ntp service, including the required package. This means that we do NOT have to add this package in the list into packages.pp, otherwise you will get an error and configs won’t get pushed.

As I’m sure you’ve noted, there are references to some “files”.
Yes, we need some extra configuration, to tell puppet to run as file server as well and where files are located.

In our example we are storing our files in here:

mkdir -p /etc/puppet/files

Now we need to add the following in /etc/puppet/fileserver.conf

  path /etc/puppet/files
allow *

Last bit, is creating the subfolders and place the files required for our configuration:

mkdir -p /etc/puppet/files 
cd /etc/puppet/files 
mkdir mysite mkdir etc

Inside mysite create mysite_apache.conf and index.html files.

Example mysite_apache.conf

<VirtualHost *:80> 
  ServerName mysite.localdomain 
  DocumentRoot /var/www/mysite.localdomain 

For index.html, you can simply have some text, just for testing purposes.

In this example, we have also setup ntp to be installed and to have a custom ntp.conf file pushed.
For this reason, we need to make sure to have this file present into /etc/puppet/files/etc as declared into our .pp file.

After doing all these changes, you should restart your puppetmaster service on the server.

If all went well, you should have the following:

  • puppetagent02 host with screen, dselect, vim (installed and with syntax on), ntp (installed, running with custom ntp.conf file)
  • puppetagent01: with the same as puppetagent02 PLUS apache with a running website

Of course this is just a raw example and you can use template and other super features.
But I think it’s a good start 😉