Category Archives: Linux

LVM – Add space

Add extra disk in a VG and expand existing LV

# df -Th /
Filesystem           Type   Size  Used Avail Use% Mounted on
/dev/mapper/vglocal00-root00
                     ext4    26G   23G  1.5G  95% /

# cfdisk /dev/sdc 
Disk has been changed.

# pvcreate /dev/sdc1 
  Physical volume "/dev/sdc1" successfully created

# pvs
  PV         VG        Fmt  Attr PSize  PFree  
  /dev/sda2  vglocal00 lvm2 a--u 29.75g 224.00m
  /dev/sdb1  vglocal01 lvm2 a--u  9.97g  64.00m
  /dev/sdc1            lvm2 ---- 10.00g  10.00g

# vgextend vglocal00 /dev/sdc1
  Volume group "vglocal00" successfully extended

# pvdisplay  /dev/sdc1 | grep Free
  Free PE               319

# lvextend --extents +319 -n /dev/vglocal00/root00 
  Size of logical volume vglocal00/root00 changed from 25.53 GiB (817 extents) to 35.50 GiB (1136 extents).
  Logical volume root00 successfully resized.

# resize2fs /dev/vglocal00/root00
resize2fs 1.41.12 (17-May-2010)
Filesystem at /dev/vglocal00/root00 is mounted on /; on-line resizing required
old desc_blocks = 2, new_desc_blocks = 3
Performing an on-line resize of /dev/vglocal00/root00 to 9306112 (4k) blocks.
The filesystem on /dev/vglocal00/root00 is now 9306112 blocks long.

# df -Th /
Filesystem           Type   Size  Used Avail Use% Mounted on
/dev/mapper/vglocal00-root00
                     ext4    35G   23G   11G  68% /

 

For a most updated article, check this one: http://blog.tian.it/lvm-how-to/

LVM – quick win

pvscan
fdisk -l | grep Disk | egrep -v "mapper|identifier"

# Create a new primary partition - Linux Type LVM (8E)
cfdisk /dev/sdb

fdisk -l | grep LVM

pvcreate /dev/sdb1 && vgcreate vglocal01 /dev/sdb1 && lvcreate -n data1 -l 100%VG vglocal01

mkdir -p /mnt/data1
mkfs.ext4 /dev/mapper/vglocal01-data1 && mount /dev/mapper/vglocal01-data1 /mnt/data1


tail -1 /etc/mtab

tail -1 /etc/mtab >> /etc/fstab

 

LVM for dummies

You have your disk /dev/sdc

You need to cfdisk/fdisk it to set the flag “Linux LVM”, (flag 8E in cfdisk).

After that, you need to make this partition/device a physical volume (pvcreate /dev/sdc1) to make this device “usable” in a Virtual Group (VG).

The VG si basically a huge disk that can be partitioned in Logical Volumes (LVs).

Once is done, you need to extend the VG to include this new device (pv) => vgextend vglocal00 /dev/sdc1

Now the space is available to the VG vglocal00 and can be used to create/extend Logical Volumes (LV), which are some sort of “partitions” of the VG.

The LV is your “new device to format”.

DISK --> 8E flag --> PV ---> VG ---> LV1
			      |_____ LV2
			      |_____ LV3

 

Apache MaxClients and ServerLimit on Centos 7 and Ubuntu 14.04

In Apache 2.4 (which is in Centos 7 and Ubuntu 14.04 default) the mpm_worker MaxClients has been replaced with MaxRequestWorkers.

In Ubuntu 14.04 you can see the below in /etc/apache2/mods-enabled/mpm_prefork.conf

<IfModule mpm_prefork_module>
StartServers 5
MinSpareServers 5
MaxSpareServers 10
MaxRequestWorkers 150
MaxConnectionsPerChild 500
ServerLimit 300
</IfModule>

In Centos 7 however there is NO configuration for this, which means it’s at the default value of 256 unless set by the user. This also means that StartServers is set by default to 3, MinSpareServers is set to 5, and MaxSpareServers is 10.
To configure Centos and set some limits, you need to edit this file /etc/httpd/conf.modules.d/00-mpm.conf appending the above content.

Always verify with apachectl -t or  httpd -t if all is ok before reloading/restarting Apache.

NOTE1: ServerLimit is not in the default configurations for either Centos 7 or Ubuntu 14.04 which means that if you set MaxRequestWorkers above 256, you must remember to add ServerLimit!

NOTE2MaxConnectionsPerChild set to 0 on Ubuntu 14.04, and 0 is also the default in Centos 7. This means that on both the Apache processes will not expire. This is going to be bad for users who like to set their php memory_limit to 1G!

Find files based on date/time

# ONE LINERS


> Modified in the last 12 hours (720 min)
find . -cmin -720 

> Modified in the last day
find . -mtime -1



# => ctime - for hacked/modified files 
# look for ctime instead, hacked scripts can't set that to what they want as opposed to mtime:

find -cmin -$n_minutes_ago
find -ctime -$n_days_ago
ls -lc   ## sorted by name
ls -ltc   ## sorted by time


>> File OLDER THAN xx days:
find . -type f -ctime +$n_days_ago

>> Find files RESTORED older that xx days and MOVE them
find . -type f -mtime +$n_days_ago | xargs -I '{}' mv {} /destination/path/

 

Automatic Updates on Raspberry Pi

How to configure automatic updates on your raspberry pi and make sure it reboots in the night (if required)

apt-get install unattended-upgrades apt-listchanges

sed -i 's/^\/\/      "o=Raspbian,n=jessie"/      "o=Raspbian,n=jessie"/g' /etc/apt/apt.conf.d/50unattended-upgrades
sed -i 's/^\/\/Unattended-Upgrade::Mail "root";/Unattended-Upgrade::Mail "root";/g' /etc/apt/apt.conf.d/50unattended-upgrades
sed -i 's/^\/\/Unattended-Upgrade::Automatic-Reboot "false";/Unattended-Upgrade::Automatic-Reboot "true";/g' /etc/apt/apt.conf.d/50unattended-upgrades
sed -i 's/^\/\/Unattended-Upgrade::Automatic-Reboot-Time "02:00";/Unattended-Upgrade::Automatic-Reboot-Time "02:00";/g' /etc/apt/apt.conf.d/50unattended-upgrades

tee /etc/apt/apt.conf.d/20auto-upgrades > /dev/null <<EOF
APT::Periodic::Update-Package-Lists "1";
APT::Periodic::Unattended-Upgrade "1";
EOF

Check the next day the log /var/log/unattended-upgrades/unattended-upgrades.log to see if it worked 🙂

 

Source: here

How to combine X consecutive lines in one using sed

It happens that you have a list and you’d like to combine multiple lines in one.
For example, a list like this one:

Mark
Smith
London
Sarah
Ruffle
Glasgow
Paul
Thompson
Liverpool
....

And have something like that:

Mark - Smith - London
Sarah - Ruffle - Glasgow
Paul - Thompson - Liverpool

How to achieve it?

Use this command:

sed 'N;N;s/\n/ - /g' list.txt

Use one extra “N” for every line you want to merge. It’s like (N-1). So, if you want to merge 3 lines like in this example, you need 2 N’s. If you’d like to merge 2 lines, you just need 1 N, and so.

Happy merging 🙂

Ubuntu Mac Keyboard

Select the right model of your keyboard

Keyboard Model -> (vendor) Apple / (model) Apple

Switch the Command key with Control key

Go into System -> Preferences -> Keyboard
Click on the “Layouts” tab and then click the “Layout Options” button.
Click on “Alt/Win key behavior
Select “Control is mapped to Win keys (and the usual ctrl key).”

Choose right layout

Keyboard Preferences -> English US (Macintosh) layout

Remote port forwarding via SSH

Imagine that you want to access a specific port on a remote server from your local machine. Basically, a “remote port forwarding”.

This remote server is not accessible directly from internet. It is NAT’d behind firewall.
To access the remote server you need to connect firstly to a remote bastion server (accessible from internet) and from there, you will be able to access the server.
Your current machine is also within restricted network and unable to ssh out. You can ssh into a local bastion server only. From this local bastion you can ssh out.

As long as you have access to the 2 bastions servers, you will be able to run the following script.

+-------------------------------+                  +-------------------------------+
|                               |                  |                               |
| +--------+         +--------+ |                  | +--------+         +--------+ |
| | LOCAL  |         | LOCAL  | |                  | | REMOTE |         | REMOTE | |
| | MACHINE| +-----> | BASTION| +---> INTERNET +---> | BASTION| +-----> | SERVER | |
| |        |         |        | |                  | |        |         |        | |
| +--------+         +--------+ |                  | +--------+         +--------+ |
|                               |                  |                               |
+-------------------------------+                  +-------------------------------+

The script points/links a local_port on your local machine to the ssh port of the remote bastion, via your local bastion.
After that, it will connect the remote port or the remote server to a new_local_port, ssh’ing via local_port.

Example below shows a way to have the VNC port 5900 available locally on port 5910.
I’m using port 8888 as local port.
Local Bastion ssh port is 8022.
Remote Bastion ssh port is 9022.

Example:

ssh -N -f -p 8022 -L8888:remote_bastion:9022 local_bastion_user@local_bastion
ssh -N -f -p 8888 -L5910:remote_server:5900 remote_bastion_user@localhost

 

And here a full script:

#!/bin/bash
#
# ============================================ #
# PORT FORWARD from CURRENT_HOST to DEST_HOST  #
# via LOC_BASTION and REMOTE_BASTION           #
# ============================================ #
#
# The scripts creates an SSH tunnel connecting
# the local port TUN_LOC_PORT to the REMOTE_BASTION ssh port
# via LOC_BASTION.
# After that, it forwards the remote port DEST_FW_PORT to
# DEST_FW_PORT using the ssh tunnel just created.
#
###########################################################

LOC_BASTION_HOST=""
LOC_BASTION_USER=""
LOC_BASTION_SSH_PORT=""

REMOTE_BASTION_HOST=""
REMOTE_BASTION_USER=""
REMOTE_BASTION_SSH_PORT=""

DEST_HOST=""
DEST_USER=""
DEST_FW_PORT="5900"

TUN_LOC_PORT="8888"
LISTENING_LOC_PORT=""

############################################################

CHECK_TUNS=$(ps aux | grep "[s]sh -N -f -p $LOC_BASTION_SSH_PORT -L$TUN_LOC_PORT:$REMOTE_BASTION_HOST:$REMOTE_BASTION_SSH_PORT $LOC_BASTION_USER@$LOC_BASTION_HOST" | awk '{print $2}')

N_TUNS=$(echo $CHECK_TUNS | wc -l)

create_tunnel(){
  # Create a connection between localhost:$TUN_LOC_PORT to MIDDLE_BOX:SSH_PORT
  # It will ask for MIDDLE_BOX's password
  # -N -f keep the connection open in background executing No commands
  ssh -N -f -p $LOC_BASTION_SSH_PORT -L$TUN_LOC_PORT:$REMOTE_BASTION_HOST:$REMOTE_BASTION_SSH_PORT $LOC_BASTION_USER@$LOC_BASTION_HOST
  echo "Created new tunnel"
}

check_tunnel(){
nc -w 1 -z localhost $TUN_LOC_PORT > /dev/null 2>&1
}

reset_tunnel() {
for PID in $CHECK_TUNS; do
   kill -9 $PID > /dev/null 2>&1
   echo "Found multiple tunnels. Killed all."
done
}


# Hidden function. Add 'cleanup' as argument to close all the tunnels
[ "$1" == "cleanup" ] && reset_tunnel && exit 0

if [ $N_TUNS -eq 0 ] ; then
   create_tunnel
elif [ $N_TUNS -eq 1 ] ; then
   check_tunnel
   if [ $? -eq 0 ] ; then
      echo "Tunnel already up and running"
   else
      reset_tunnel
      create_tunnel
   fi
else
   reset_tunnel
   create_tunnel
fi


CHECK_PORT_FWD=$(ps aux | grep -q "[s]sh -N -f -p $TUN_LOC_PORT -L$LISTENING_LOC_PORT:$DEST_HOST:$DEST_FW_PORT -l $REMOTE_BASTION_USER localhost")
if [ $? -eq 0 ] ; then
   echo "Port forward already created. Remote port $DEST_FW_PORT should be accessible on localhost port $LISTENING_LOC_PORT"
   exit 0
 else
   # This will create 'link' between $DEST_FW_PORT from $DEST_HOST to $TUN_LOC_PORT on localhost
   echo "Creating link between $DEST_FW_PORT to $TUN_LOC_PORT on localhost via $DEST_HOST"
   ssh -N -f -p $TUN_LOC_PORT -L$LISTENING_LOC_PORT:$DEST_HOST:$DEST_FW_PORT -l $REMOTE_BASTION_USER localhost
   echo "You can now access $DEST_FW_PORT listening on $DEST_HOST from localhost on port $LISTENING_LOC_PORT."
fi