Category Archives: Linux

Lsyncd – basic setup

This is an example where you install Lsyncd on a CentOS master server and you sync the folder ‘data’ on a slave server with IP 10.0.0.3

First of all, your master server needs an SSH key setup AND the slave has to have it configured, to allow passwordless SSH connection

Here an article that tells you how to do it.

Configure Lsyncd

/etc/lsyncd.conf 

-- comments with "--"
settings {
logfile = "/var/log/lsyncd/lsyncd.log",
statusFile = "/var/log/lsyncd/lsyncd-status.log",
statusInterval = 20
}

sync {
default.rsync,
source="/data/",
target="10.0.0.3:/data/",
rsync = {
compress = true,
archive = true,
verbose = true,
rsh = "/usr/bin/ssh -p 22 -o StrictHostKeyChecking=no"
},
-- excludeFrom = "/etc/lsyncd.exclusions"
}

 

Add the service and enable it

chkconfig --add lsyncd
chkconfig lsyncd on

On CentoS7 use this:

systemctl enable lsyncd.service

 

Logrotate

Once installed, you also need to be sure that Lsyncd logs are managed by Logrotate.

Create/update this file: /etc/logrotate.d/lsyncd

/var/log/lsyncd/*log {
    missingok
    notifempty
    sharedscripts
    postrotate
    if [ -f /var/lock/lsyncd ]; then
      /sbin/service lsyncd restart > /dev/null 2>/dev/null || true
    fi
    endscript
}

 

On CentOS7, you need to use sistemctl instead service command:

/var/log/lsyncd/*log {
missingok
notifempty
sharedscripts
postrotate
if [ -f /var/lock/lsyncd ]; then
/bin/systemctl restart lsyncd.service > /dev/null 2>/dev/null || true
fi
endscript
}

Test the logrotate config

You can test this using the command:

logrotate -d /etc/logrotate.d/lsyncd

 

For more advanced Lsyncd configuration, check this article 🙂

Linux SSH auth passwordless using key

Pretty basic, but handy for whoever start playing with Linux.

Here simple steps to follow in order to have box1 to be able to connect securely to box2 over SSH without being required to insert password.
This is very handy if you run scripts 😉

On BOX1

You can run this as any user.

ssh-keygen -b 1024 -t rsa -f id_rsa -P ""

This will generate  ~/.ssh/id_rsa (private key) and  ~/.ssh/id_rsa.pub (public key).
The .pub is the key that needs to be appended in ~/.ssh/authorized_keys on BOX2.

If the following command is available, that’s the best/safest way to setup BOX2.

ssh-copy-id user@box2

Password for user on box2 will be requested.
Once completed, you can try to ssh user@box2 and theoretically you should be able to connect without need to insert the password again!

If ssh-copy-id does not exist (e.g. Mac or other Distros), you can scp the .pub file and append it as per below:

scp ~/.ssh/id_rsa.pub user@box2:/tmp

Then connect to box2 with user and run this:

cat /tmp/id_rsa.pub >> ~/.ssh/authorized_keys
rm -f /tmp/id_rsa.pub

After those 2 commands, the key should be added to the authorised ones, so  ssh user@box2 should work.

NOTE: if you are experiencing issues, please make sure that the permissions of id_rsa file is 600 on BOX1 and that sshd_conf on BOX2 is set to allow key auth connections

vim without .vimrc

If you want to run vim without executing a customer’s .vimrc, as they’ve got crazy colours and random stuff all over the show, just do use NONE as a special value to skip any .vimrc parsing;

vim -u NONE

You might need to run :set nocp in vim itself if you’re like me and used to the non-vi compatible features.

NFS – quick win

This is a very basic step-by-step guide to create a CentOS7 NFS server that shares a folder /nfsshare over 192.168.4.0/24 network. This share will be owned by apache and mountable on a CentOS web server.

Here the instructions how to create the server and how to setup the client.

NFS Server

Add this line in IPTABLES:

-A INPUT -s 192.168.4.0/24 -m comment --comment "NFS Network" -j ACCEPT

 

Run the following to create a share folder and setup NFS:

mkdir /nfsshare
yum install nfs-utils nfs-utils-lib -y
systemctl enable nfs-server
echo "/nfsshare 192.168.4.0/24(rw,sync,no_root_squash)" >> /etc/exports
sed -i 's/#Domain = local.domain.edu/Domain = nfsdomain.loc/' /etc/idmapd.conf
systemctl start rpcbind
systemctl start nfs-server

# Create apache user/group
# (NFS clients will read/write using this user so we want to have 
# the same set also on the server for an easier ownership management)
groupadd -g 48 apache
useradd -g 48 -u 48 apache

 

NFS Client

e.g. assuming that NFS server’s IP is 192.168.4.1

Add this line in IPTABLES:

-A INPUT -s 192.168.4.0/24 -m comment --comment "NFS Network" -j ACCEPT

Then, run this:

yum install nfs-utils rpcbind

sed -i 's/#Domain = local.domain.edu/Domain = nfsdomain.loc/' /etc/idmapd.conf
echo "192.168.4.1 NFS01" >> /etc/hosts
mount -t nfs4 -o noatime,proto=tcp,actimeo=3,hard,intr,acl,_netdev NFS01:/nfsshare
tail -1 /proc/mounts >> /etc/fstab

NOTE: we are hardly mapping the NFS server’s IP in /etc/hosts to make easier to recognise the mount (in case of multiple mounts).

If you are facing the issue where you mount /nfsshare and you see the owner of the files and folders showing as nobody:nobody, it could be related to rpcidmapd and DNS. To fix this, try to update /etc/hosts on the Client with <hostname>.nfsdomain.loc

# ============= #
# Ubuntu Notes  #
# ============= #

!! Same users on Server and Client - for the exported partition !!

SERVER
apt-get install nfs-kernel-server

vim /etc/exports
/var/www/vhosts		192.168.3.*(rw,sync,no_root_squash,no_subtree_check)

service nfs-kernel-server restart
exportfs -a


CLIENT
apt-get install nfs-common
mount -t nfs4 192.168.3.1:/var/www/vhosts /var/www/vhosts/

!! CHECK the output of cat /proc/mounts and you can get the correct rsize/wsize. If the firewall/network can handle, keep this value as big as possible:

e.g.

noatime,proto=tcp,actimeo=3,hard,intr,acl,_netdev,rsize=1048576,wsize=1048576

vim /etc/fstab
192.168.3.1:/var/www/vhosts   /var/www/vhosts nfs4    noatime,actimeo=3,hard,intr,acl,_netdev,rsize=32768,wsize=32768 0 0

 

recap utility 

# Install recap
git clone https://github.com/rackerlabs/recap.git
cd recap
./recap-installer 


# If there is no MySQL or it can;t connect to it edit:
/etc/recap
                                USEMYSQL=no
                                USEMYSQLPROCESSLIST=no

Edit cron if you want to run often than 10 minutes:
                                /etc/cron.d/recap 


=================================================

LOG ANALYSIS SCRIPTS

>> Check for memory usage
grep "buffers/cache" /var/log/recap/resources_* | sed 's/^.*resources_\(.*\)\.log.*cache:\s*\([0-9]*\)\s*\([0-9]*\)$/FREE: \3| USED: \2| DATE: \1/ '

>> Running the following to check if the memory went less than 4999MB
grep "buffers/cache" /var/log/recap/resources_* | sed 's/^.*resources_\(.*\)\.log.*cache:\s*\([0-9]*\)\s*\([0-9]*\)$/FREE: \3| USED: \2| DATE: \1/ ' | egrep -v "FREE: [65][0-9]{3}\|" | wc -l