Tag Archives: lsyncd

Lsyncd – basic setup

This is an example where you install Lsyncd on a CentOS master server and you sync the folder ‘data’ on a slave server with IP 10.0.0.3

First of all, your master server needs an SSH key setup AND the slave has to have it configured, to allow passwordless SSH connection

Here an article that tells you how to do it.

Configure Lsyncd

/etc/lsyncd.conf 

-- comments with "--"
settings {
logfile = "/var/log/lsyncd/lsyncd.log",
statusFile = "/var/log/lsyncd/lsyncd-status.log",
statusInterval = 20
}

sync {
default.rsync,
source="/data/",
target="10.0.0.3:/data/",
rsync = {
compress = true,
archive = true,
verbose = true,
rsh = "/usr/bin/ssh -p 22 -o StrictHostKeyChecking=no"
},
-- excludeFrom = "/etc/lsyncd.exclusions"
}

 

Add the service and enable it

chkconfig --add lsyncd
chkconfig lsyncd on

On CentoS7 use this:

systemctl enable lsyncd.service

 

Logrotate

Once installed, you also need to be sure that Lsyncd logs are managed by Logrotate.

Create/update this file: /etc/logrotate.d/lsyncd

/var/log/lsyncd/*log {
    missingok
    notifempty
    sharedscripts
    postrotate
    if [ -f /var/lock/lsyncd ]; then
      /sbin/service lsyncd restart > /dev/null 2>/dev/null || true
    fi
    endscript
}

 

On CentOS7, you need to use sistemctl instead service command:

/var/log/lsyncd/*log {
missingok
notifempty
sharedscripts
postrotate
if [ -f /var/lock/lsyncd ]; then
/bin/systemctl restart lsyncd.service > /dev/null 2>/dev/null || true
fi
endscript
}

Test the logrotate config

You can test this using the command:

logrotate -d /etc/logrotate.d/lsyncd

 

For more advanced Lsyncd configuration, check this article 🙂

Lsyncd – conf.d like setup on Ubuntu

On Ubuntu 14 (version 2.1.x)

Create configuration files

apt-get install lsyncd
grep "CONFIG=" /etc/init.d/lsyncd

-> it should be /etc/lsyncd/lsyncd.conf.lua

mkdir -p /etc/lsyncd/conf.d/

Backup the original file and create a new conf file

mv /etc/lsyncd/lsyncd.conf.lua{,.ORIG}
cat <<'EOF' > /etc/lsyncd/lsyncd.conf.lua 
-- DO NOT EDIT THIS FILE
settings {
logfile = "/var/log/lsyncd.log",
statusFile = "/var/log/lsyncd-status.log",
statusInterval = 5
}
-- conf.d style configs
package.path = "/etc/lsyncd/conf.d/?.lua;" .. package.path
local f = io.popen("ls /etc/lsyncd/conf.d/*.lua|xargs -n1 basename|sed 's/.lua//'") for mod in f:lines() do require(mod) end
-- // DO NOT EDIT THIS FILE
EOF

 

Create the config file for 2 web nodes called w01 and w02.
These 2 nodes have the following IPs:
10.180.3.201 w01
10.180.3.322 w02

cat <<'EOF' > /etc/lsyncd/conf.d/w0x.lua 
-- w01 and w02
servers = {
"10.180.3.201",
"10.180.3.322",
}

for _, server in ipairs(servers) do
sync {
default.rsync,
source="/var/www/vhosts/",
target=server..":/var/www/vhosts/",
rsync = {
compress = true,
archive = true,
verbose = true,
rsh = "/usr/bin/ssh -p 22 -o StrictHostKeyChecking=no"
},
excludeFrom = "/etc/lsyncd/conf.d/w0x.exclusions"
}
end
EOF

Now let’s create the exclusions file. This will be the list of paths that won’t be sync’d.

cat <<'EOF' > /etc/lsyncd/conf.d/w0x.exclusions
www.mytestsite.com/
EOF

NOTE! For exclusions, please remember to put the relative path, NOT the full path. In this case, it excludes www.mytestsite.com/ from /var/www/vhosts

Set up a logrotate conf file

cat > /etc/logrotate.d/lsyncd << EOF
/var/log/lsyncd/*log {
missingok
notifempty
sharedscripts
postrotate
if [ -f /var/lock/lsyncd ]; then
/sbin/service lsyncd restart > /dev/null 2>/dev/null || true
fi
endscript
}
EOF

 

Troubleshoot

Test Lsyncd

$ lsyncd --nodaemon -log Exec /etc/lsyncd/lsyncd.conf.lua

 

Error log – inotify issue

ERROR: Terminating since out of inotify watches//Consider increasing /proc/sys/fs/inotify/max_user_watches

Temporary fix:

# echo 100000 > /proc/sys/fs/inotify/max_user_watches

Permanent fix (ALSO write sysctl.conf):

# echo 100000 > /proc/sys/fs/inotify/max_user_watches
# echo "fs.inotify.max_user_watches = 100000" >> /etc/sysctl.conf

Lsync monitoring on Rackspace Cloud 

mkdir -p /usr/lib/rackspace-monitoring-agent/plugins/
cd /usr/lib/rackspace-monitoring-agent/plugins/
wget https://raw.githubusercontent.com/racker/rackspace-monitoring-agent-plugins-contrib/master/lsyncd-status.sh
chmod 755 lsyncd-status.sh

You can test the above script by calling it directly to see if it is working and reporting stats:

/usr/lib/rackspace-monitoring-agent/plugins/lsyncd-status.sh

 

Now, we need to create the alert itself.
[To get the token, you can use this]

curl -i -X POST \
-H 'X-Auth-Token: [AUTH_TOKEN]' \
-H 'Content-Type: application/json; charset=UTF-8' \
-H 'Accept: application/json' \
--data-binary \
'{"label": "Lsyncd", "type": "agent.plugin", "details": {"file": "lsyncd-status.sh","args": ["arg1","arg2"]}}' \
'https://monitoring.api.rackspacecloud.com/v1.0/[ACCOUNT_ID]/entities/[ENTITY_ID]/checks'

NOTE: ENTITY_ID is the Monitoring ID, NOT the server ID!!

Once the alert has been created, you can add the alarm manually via the Control Panel:

if (metric['lsyncd_status'] != 'running') {
return new AlarmStatus(CRITICAL, 'Lsyncd Service is NOT running.');
}
if (metric['lsyncd_status'] == 'running' && metric['percent_used_watches'] >= 80) {
return new AlarmStatus(WARNING, 'Lsyncd is running but the number of directories has reached 80% of notify watches.');
}
if (metric['lsyncd_status'] == 'running' && metric['percent_used_watches'] >= 95) {
return new AlarmStatus(CRITICAL, 'Lsyncd is running but the number of directories has reached 95% of notify watches.');
}
return new AlarmStatus(OK, 'Lsyncd Service is running.');

Make sure to test and save the alert.