Tag Archives: lsyncd

Lsyncd – basic setup

This is an example where you install Lsyncd on a CentOS master server and you sync the folder ‘data’ on a slave server with IP 10.0.0.3

First of all, your master server needs an SSH key setup AND the slave has to have it configured, to allow passwordless SSH connection

Here an article that tells you how to do it.

Configure Lsyncd

/etc/lsyncd.conf

-- comments with "--"
settings {
logfile = "/var/log/lsyncd/lsyncd.log",
statusFile = "/var/log/lsyncd/lsyncd-status.log",
statusInterval = 20
}

sync {
default.rsync,
source="/data/",
target="10.0.0.3:/data/",
rsync = {
compress = true,
archive = true,
verbose = true,
rsh = "/usr/bin/ssh -p 22 -o StrictHostKeyChecking=no"
},
-- excludeFrom = "/etc/lsyncd.exclusions"
}

-- comments with "--"

settings {

logfile = "/var/log/lsyncd/lsyncd.log",

statusFile = "/var/log/lsyncd/lsyncd-status.log",

statusInterval = 20

}

sync {

default.rsync,

source="/data/",

target="10.0.0.3:/data/",

rsync = {

compress = true,

archive = true,

verbose = true,

rsh = "/usr/bin/ssh -p 22 -o StrictHostKeyChecking=no"

-- excludeFrom = "/etc/lsyncd.exclusions"

}

Add the service and enable it

chkconfig --add lsyncd
chkconfig lsyncd on

1 2	chkconfig --add lsyncd chkconfig lsyncd on

On CentoS7 use this:

systemctl enable lsyncd.service

1	systemctl enable lsyncd.service

Logrotate

Once installed, you also need to be sure that Lsyncd logs are managed by Logrotate.

Create/update this file: /etc/logrotate.d/lsyncd

/var/log/lsyncd/*log {
    missingok
    notifempty
    sharedscripts
    postrotate
    if [ -f /var/lock/lsyncd ]; then
      /sbin/service lsyncd restart > /dev/null 2>/dev/null || true
    fi
    endscript
}

/var/log/lsyncd/*log {

missingok

notifempty

sharedscripts

postrotate

if [ -f /var/lock/lsyncd ]; then

/sbin/service lsyncd restart > /dev/null 2>/dev/null || true

endscript

}

On CentOS7, you need to use sistemctl instead service command:

/var/log/lsyncd/*log {
missingok
notifempty
sharedscripts
postrotate
if [ -f /var/lock/lsyncd ]; then
/bin/systemctl restart lsyncd.service > /dev/null 2>/dev/null || true
fi
endscript
}

/var/log/lsyncd/*log {

missingok

notifempty

sharedscripts

postrotate

if [ -f /var/lock/lsyncd ]; then

/bin/systemctl restart lsyncd.service > /dev/null 2>/dev/null || true

endscript

}

Test the logrotate config

You can test this using the command:

logrotate -d /etc/logrotate.d/lsyncd

1	logrotate -d /etc/logrotate.d/lsyncd

For more advanced Lsyncd configuration, check this article 🙂

Lsyncd – conf.d like setup on Ubuntu

On Ubuntu 14 (version 2.1.x)

Create configuration files

apt-get install lsyncd
grep "CONFIG=" /etc/init.d/lsyncd

1 2	apt-get install lsyncd grep "CONFIG=" /etc/init.d/lsyncd

-> it should be /etc/lsyncd/lsyncd.conf.lua

mkdir -p /etc/lsyncd/conf.d/

1	mkdir -p /etc/lsyncd/conf.d/

Backup the original file and create a new conf file

mv /etc/lsyncd/lsyncd.conf.lua{,.ORIG}
cat <<'EOF' > /etc/lsyncd/lsyncd.conf.lua 
-- DO NOT EDIT THIS FILE
settings {
logfile = "/var/log/lsyncd.log",
statusFile = "/var/log/lsyncd-status.log",
statusInterval = 5
}
-- conf.d style configs
package.path = "/etc/lsyncd/conf.d/?.lua;" .. package.path
local f = io.popen("ls /etc/lsyncd/conf.d/*.lua|xargs -n1 basename|sed 's/.lua//'") for mod in f:lines() do require(mod) end
-- // DO NOT EDIT THIS FILE
EOF

mv /etc/lsyncd/lsyncd.conf.lua{,.ORIG}

cat <<'EOF' > /etc/lsyncd/lsyncd.conf.lua

-- DO NOT EDIT THIS FILE

settings {

logfile = "/var/log/lsyncd.log",

statusFile = "/var/log/lsyncd-status.log",

statusInterval = 5

}

-- conf.d style configs

package.path = "/etc/lsyncd/conf.d/?.lua;" .. package.path

local f = io.popen("ls /etc/lsyncd/conf.d/*.lua|xargs -n1 basename|sed 's/.lua//'") for mod in f:lines() do require(mod) end

-- // DO NOT EDIT THIS FILE

EOF

Create the config file for 2 web nodes called w01 and w02.
These 2 nodes have the following IPs:
10.180.3.201 w01
10.180.3.322 w02

cat <<'EOF' > /etc/lsyncd/conf.d/w0x.lua 
-- w01 and w02
servers = {
"10.180.3.201",
"10.180.3.322",
}

for _, server in ipairs(servers) do
sync {
default.rsync,
source="/var/www/vhosts/",
target=server..":/var/www/vhosts/",
rsync = {
compress = true,
archive = true,
verbose = true,
rsh = "/usr/bin/ssh -p 22 -o StrictHostKeyChecking=no"
},
excludeFrom = "/etc/lsyncd/conf.d/w0x.exclusions"
}
end
EOF

cat <<'EOF' > /etc/lsyncd/conf.d/w0x.lua

-- w01 and w02

servers = {

"10.180.3.201",

"10.180.3.322",

}

for _, server in ipairs(servers) do

sync {

default.rsync,

source="/var/www/vhosts/",

target=server..":/var/www/vhosts/",

rsync = {

compress = true,

archive = true,

verbose = true,

rsh = "/usr/bin/ssh -p 22 -o StrictHostKeyChecking=no"

excludeFrom = "/etc/lsyncd/conf.d/w0x.exclusions"

}

end

EOF

Now let’s create the exclusions file. This will be the list of paths that won’t be sync’d.

cat <<'EOF' > /etc/lsyncd/conf.d/w0x.exclusions
www.mytestsite.com/
EOF

cat <<'EOF' > /etc/lsyncd/conf.d/w0x.exclusions

www.mytestsite.com/

EOF

NOTE! For exclusions, please remember to put the relative path, NOT the full path. In this case, it excludes www.mytestsite.com/ from /var/www/vhosts

Set up a logrotate conf file

cat > /etc/logrotate.d/lsyncd << EOF
/var/log/lsyncd/*log {
missingok
notifempty
sharedscripts
postrotate
if [ -f /var/lock/lsyncd ]; then
/sbin/service lsyncd restart > /dev/null 2>/dev/null || true
fi
endscript
}
EOF

cat > /etc/logrotate.d/lsyncd << EOF

/var/log/lsyncd/*log {

missingok

notifempty

sharedscripts

postrotate

if [ -f /var/lock/lsyncd ]; then

/sbin/service lsyncd restart > /dev/null 2>/dev/null || true

endscript

}

EOF

Troubleshoot

Test Lsyncd

$ lsyncd --nodaemon -log Exec /etc/lsyncd/lsyncd.conf.lua

1	$ lsyncd --nodaemon -log Exec /etc/lsyncd/lsyncd.conf.lua

Error log – inotify issue

ERROR: Terminating since out of inotify watches//Consider increasing /proc/sys/fs/inotify/max_user_watches

Temporary fix:

# echo 100000 > /proc/sys/fs/inotify/max_user_watches

1	# echo 100000 > /proc/sys/fs/inotify/max_user_watches

Permanent fix (ALSO write sysctl.conf):

# echo 100000 > /proc/sys/fs/inotify/max_user_watches
# echo "fs.inotify.max_user_watches = 100000" >> /etc/sysctl.conf

1 2	# echo 100000 > /proc/sys/fs/inotify/max_user_watches # echo "fs.inotify.max_user_watches = 100000" >> /etc/sysctl.conf

Lsync monitoring on Rackspace Cloud

mkdir -p /usr/lib/rackspace-monitoring-agent/plugins/
cd /usr/lib/rackspace-monitoring-agent/plugins/
wget https://raw.githubusercontent.com/racker/rackspace-monitoring-agent-plugins-contrib/master/lsyncd-status.sh
chmod 755 lsyncd-status.sh

mkdir -p /usr/lib/rackspace-monitoring-agent/plugins/

cd /usr/lib/rackspace-monitoring-agent/plugins/

wget https://raw.githubusercontent.com/racker/rackspace-monitoring-agent-plugins-contrib/master/lsyncd-status.sh

chmod 755 lsyncd-status.sh

You can test the above script by calling it directly to see if it is working and reporting stats:

/usr/lib/rackspace-monitoring-agent/plugins/lsyncd-status.sh

1	/usr/lib/rackspace-monitoring-agent/plugins/lsyncd-status.sh

Now, we need to create the alert itself.
[To get the token, you can use this]

curl -i -X POST \
-H 'X-Auth-Token: [AUTH_TOKEN]' \
-H 'Content-Type: application/json; charset=UTF-8' \
-H 'Accept: application/json' \
--data-binary \
'{"label": "Lsyncd", "type": "agent.plugin", "details": {"file": "lsyncd-status.sh","args": ["arg1","arg2"]}}' \
'https://monitoring.api.rackspacecloud.com/v1.0/[ACCOUNT_ID]/entities/[ENTITY_ID]/checks'

curl -i -X POST \

-H 'X-Auth-Token: [AUTH_TOKEN]' \

-H 'Content-Type: application/json; charset=UTF-8' \

-H 'Accept: application/json' \

--data-binary \

'{"label": "Lsyncd", "type": "agent.plugin", "details": {"file": "lsyncd-status.sh","args": ["arg1","arg2"]}}' \

'https://monitoring.api.rackspacecloud.com/v1.0/[ACCOUNT_ID]/entities/[ENTITY_ID]/checks'

NOTE: ENTITY_ID is the Monitoring ID, NOT the server ID!!

Once the alert has been created, you can add the alarm manually via the Control Panel:

if (metric['lsyncd_status'] != 'running') {
return new AlarmStatus(CRITICAL, 'Lsyncd Service is NOT running.');
}
if (metric['lsyncd_status'] == 'running' && metric['percent_used_watches'] >= 80) {
return new AlarmStatus(WARNING, 'Lsyncd is running but the number of directories has reached 80% of notify watches.');
}
if (metric['lsyncd_status'] == 'running' && metric['percent_used_watches'] >= 95) {
return new AlarmStatus(CRITICAL, 'Lsyncd is running but the number of directories has reached 95% of notify watches.');
}
return new AlarmStatus(OK, 'Lsyncd Service is running.');

if (metric['lsyncd_status'] != 'running') {

return new AlarmStatus(CRITICAL, 'Lsyncd Service is NOT running.');

}

if (metric['lsyncd_status'] == 'running' && metric['percent_used_watches'] >= 80) {

return new AlarmStatus(WARNING, 'Lsyncd is running but the number of directories has reached 80% of notify watches.');

}

if (metric['lsyncd_status'] == 'running' && metric['percent_used_watches'] >= 95) {

return new AlarmStatus(CRITICAL, 'Lsyncd is running but the number of directories has reached 95% of notify watches.');

}

return new AlarmStatus(OK, 'Lsyncd Service is running.');

Make sure to test and save the alert.

My notepad

Chris' IT notes…

Tag Archives: lsyncd

Lsyncd – basic setup

Configure Lsyncd

Logrotate

Test the logrotate config

Lsyncd – conf.d like setup on Ubuntu

Create configuration files

Set up a logrotate conf file

Troubleshoot

Test Lsyncd

Error log – inotify issue

Lsync monitoring on Rackspace Cloud