Tag Archives: raspberry pi

Auto mount an encrypted IMG file stored on NFS share

Yes, here we are again.
Now that I have a NAS at home, it’s about time to get rid of all these single USB disks connected to my Raspberry PIs.

I have a share called nfsshare available from my NAS (IP: 192.168.1.10). The full share path is 192.168.1.10:/volume1/nfsshare. My NAS handles NFS version 4.

So, here what I’ve done, to setup my Banana Pro Pi with Armbian based on Debian 10 (buster).

Configure NFS client

First of all, we need to create the mount point where we’re going to access the nfs share (let’s use /nfs) and install the packages for NFS.

mkdir /nfs
apt-get install nfs-common

1 2	mkdir /nfs apt-get install nfs-common

Once done, a minimal tuning of idmapd.conf, if you have defined a domain/workgroup within your network. In this example I’m using mydomain.loc.

sed -i 's/#Domain = local.domain.edu/Domain = mydomain.loc/' /etc/idmapd.conf

1	sed -i 's/#Domain = local.domain.edu/Domain = mydomain.loc/' /etc/idmapd.conf

Update our /etc/fstab file, to make sure it mounts at boot, and test if all works as expected:

192.168.1.10:/volume1/nfsshare /nfs nfs4 auto,_netdev,nofail,noatime,nolock 0 0

1	192.168.1.10:/volume1/nfsshare /nfs nfs4 auto,_netdev,nofail,noatime,nolock 0 0

I have used _netdev to make sure that the system understands that this needs the network up before trying to mount, and, if something goes wrong, the boot continues (nofail). This is very handy on systems without a proper monitor where you rely on ssh connections.

Now, with a simple mount /nfs command, you should be able to get the share mounted. df -Th or mount commands are what I would you to verify.

Cool, we have now the share mounted. Issue a quick shutdown -r now to see if all works as expected. Once your device is back online, ssh into it and check with df -Th or mount commands again. Hopefully, you can see your nfs share mounted to /nfs.

Create and configure your Encrypted “space”

I have already discussed something about encrypted devices in another post. This will be a revised version of the previous post, without custom scripts, but simply using what Debian offers out of the box.

Create an empty IMG file to host our encrypted space

I have decided to create 500GB of encrypted space to store my data. To do so, I did the following:

install the required software for encryption
create a sparsefile (on my /nfs share)
encrypt it
format it (ext4)
setup the auto mount

apt-get install cryptsetup

dd of=/nfs/file_container.img bs=1 count=0 seek=500G

cryptsetup -y luksFormat /nfs/file_container.img
cryptsetup luksOpen /nfs/file_container.img cryptcontainer

mkfs.ext4 -L cryptarchive /dev/mapper/cryptcontainer

apt-get install cryptsetup

dd of=/nfs/file_container.img bs=1 count=0 seek=500G

cryptsetup -y luksFormat /nfs/file_container.img

cryptsetup luksOpen /nfs/file_container.img cryptcontainer

mkfs.ext4 -L cryptarchive /dev/mapper/cryptcontainer

During the above steps, you will be asked to set a passphrase, and use it to open the IMG file. Pretty straight forward, isn’t it?

Cool. Now we have 500GB sparsefile called file_container.img store on our share /nfs ready to be mounted somewhere and utilised.

To make sure we can mount at boot, we need a secret key that we are going to use to decrypt the IMG file without typing any passphrase.

Let’s create this key stored under /root (in this example). You can store wherever you want, as long as it’s accessible before the decryption start. Another good place is /boot.

dd if=/dev/urandom of=/root/keyfile bs=1024 count=4
chmod 0400 /root/keyfile

1 2	dd if=/dev/urandom of=/root/keyfile bs=1024 count=4 chmod 0400 /root/keyfile

Now we need to add this key within the IMG file

cryptsetup luksAddKey /nfs/file_container.img /root/keyfile

1	cryptsetup luksAddKey /nfs/file_container.img /root/keyfile

Next step, is to instruct /etc/crypttab, with the details about our encrypted file and its key.
Just add the following line at the end of /etc/crypttab file.

cryptcontainer /nfs/file_container.img /root/keyfile luks

1	cryptcontainer /nfs/file_container.img /root/keyfile luks

Now, there is a problem. Your OS needs to know that the IMG file isn’t stored locally and has a dependency on the NFS share. In fact, if the OS tries to decrypt the IMG file before mounting the NFS share, it will fail, and you might get stuck in a no-end booting, forcing sometimes to get your mini monitor for troubleshooting, a spare keyboard and anti-human positions to reach your small Pi etc etc… basically, a nightmare!

So, here a trick that seems working.
In Debian, there is a file called /etc/default/cryptdisks
Within this file, we are going to make sure that CRYPTDISKS_ENABLE is set to yes and CRYPTDISKS_MOUNT is set to our NFS mount (/nfs). In this way, the service that handles the encryption/decription will wait for /nfs mounted before starting.
IMPORTANT: this must be a mountpoint within /etc/fstab

Here the content of my /etc/default/cryptdisks file

# Run cryptdisks initscripts at startup? Default is Yes.
CRYPTDISKS_ENABLE=Yes

# Mountpoints to mount, before cryptsetup is invoked at initscripts. Takes
# mountpoins which are configured in /etc/fstab as arguments. Separate
# mountpoints by space.
# This is useful for keyfiles on removable media. Default is unset.
CRYPTDISKS_MOUNT="/nfs"

# Default check script. Takes effect, if the 'check' option is set in crypttab
# without a value.
CRYPTDISKS_CHECK=blkid

# Run cryptdisks initscripts at startup? Default is Yes.

CRYPTDISKS_ENABLE=Yes

# Mountpoints to mount, before cryptsetup is invoked at initscripts. Takes

# mountpoins which are configured in /etc/fstab as arguments. Separate

# mountpoints by space.

# This is useful for keyfiles on removable media. Default is unset.

CRYPTDISKS_MOUNT="/nfs"

# Default check script. Takes effect, if the 'check' option is set in crypttab

# without a value.

CRYPTDISKS_CHECK=blkid

Amazing! Now, just the last bit: update /etc/fstab with the reference of our device. Because now we have setup all the necessary to open the encrypted IMG file and associate it to a mountable device. But we haven’t mounted it yet!

Create the mount point:

mkdir /cryptoarchive

1	mkdir /cryptoarchive

Update /etc/fstab, appending this line:

/dev/mapper/cryptcontainer /cryptoarchive ext4 defaults,nofail 0 2

1	/dev/mapper/cryptcontainer /cryptoarchive ext4 defaults,nofail 0 2

Again, the nofail, as for the NFS share, to avoid the boot process to get stuck in case of errors, and allow you to ssh into the device and troubleshoot.

Now we’re ready to try a mount /cryptoarchive, a df -Th and mount checks, and also a shutdown -r now, to verify that the NFS share gets mounted and the IMG encrypted disk mounted and available too.

Happy playing! 😉

Fail2ban Debian 9

Scratch pad with conf files to configure Fail2ban on Debian 9

This setup will configure Fail2ban to monitor SSH and keep track of the bad guys. Every time an IP gets banned, it will be stored in /etc/fail2ban/ip.blacklist .
This files gets processed every time Fail2ban restarts.
A cron will sanitise the file daily.

HOW TO

1) Create a custom action file: /etc/fail2ban/action.d/iptables-allports-CUSTOM.conf

# Fail2Ban configuration file

[INCLUDES]

before = iptables-common.confhttps://docs.google.com/document/d/1DjP5z7tvkaMWJMZXVAnMOCgfynfQNHvRkqJyxQdEB84/edit?usp=sharing


[Definition]

# Option:  actionstart
# Notes.:  command executed once at the start of Fail2Ban.
# Values:  CMD
#
actionstart = <iptables> -N f2b-<name>
              <iptables> -A f2b-<name> -j <returntype>
              <iptables> -I <chain> -p <protocol> -j f2b-<name>
              # Persistent banning of IPs
              cat /etc/fail2ban/ip.blacklist | grep -v ^\s*#|awk '{print $1}' | while read IP; do <iptables> -I f2b-<name> 1 -s $IP -j DROP; done

# Option:  actionstop
# Notes.:  command executed once at the end of Fail2Ban
# Values:  CMD
#
actionstop = <iptables> -D <chain> -p <protocol> -j f2b-<name>
             <iptables> -F f2b-<name>
             <iptables> -X f2b-<name>

# Option:  actioncheck
# Notes.:  command executed once before each actionban command
# Values:  CMD
#
actioncheck = <iptables> -n -L <chain> | grep -q 'f2b-<name>[ \t]'

# Option:  actionban
# Notes.:  command executed when banning an IP. Take care that the
#          command is executed with Fail2Ban user rights.
# Tags:    See jail.conf(5) man page
# Values:  CMD
#
actionban = <iptables> -I f2b-<name> 1 -s <ip> -j <blocktype>
            # Persistent banning of IPs
            echo '<ip>' >> /etc/fail2ban/ip.blacklist

# Option:  actionunban
# Notes.:  command executed when unbanning an IP. Take care that the
#          command is executed with Fail2Ban user rights.
# Tags:    See jail.conf(5) man page
# Values:  CMD
#
actionunban = <iptables> -D f2b-<name> -s <ip> -j <blocktype>

[Init]

# Fail2Ban configuration file

[INCLUDES]

before = iptables-common.confhttps://docs.google.com/document/d/1DjP5z7tvkaMWJMZXVAnMOCgfynfQNHvRkqJyxQdEB84/edit?usp=sharing

[Definition]

# Option: actionstart

# Notes.: command executed once at the start of Fail2Ban.

# Values: CMD

actionstart = <iptables> -N f2b-<name>

# Persistent banning of IPs

cat /etc/fail2ban/ip.blacklist | grep -v ^\s*#|awk '{print $1}' | while read IP; do <iptables> -I f2b-<name> 1 -s $IP -j DROP; done

# Option: actionstop

# Notes.: command executed once at the end of Fail2Ban

# Values: CMD

actionstop = <iptables> -D <chain> -p <protocol> -j f2b-<name>

# Option: actioncheck

# Notes.: command executed once before each actionban command

# Values: CMD

actioncheck = <iptables> -n -L <chain> | grep -q 'f2b-<name>[ \t]'

# Option: actionban

# Notes.: command executed when banning an IP. Take care that the

# command is executed with Fail2Ban user rights.

# Tags: See jail.conf(5) man page

# Values: CMD

actionban = <iptables> -I f2b-<name> 1 -s <ip> -j <blocktype>

# Persistent banning of IPs

echo '<ip>' >> /etc/fail2ban/ip.blacklist

# Option: actionunban

# Notes.: command executed when unbanning an IP. Take care that the

# command is executed with Fail2Ban user rights.

# Tags: See jail.conf(5) man page

# Values: CMD

actionunban = <iptables> -D f2b-<name> -s <ip> -j <blocktype>

[Init]

2) Create /etc/fail2ban/jail.local

# Fail2Ban custom configuration file.


[DEFAULT]

# "ignoreip" can be an IP address, a CIDR mask or a DNS host
ignoreip = 127.0.0.1 192.168.1.0/24 192.168.2.0/24

# Ban forever => -1
#bantime=-1

# Ban 3 days => 259200
bantime = 259200 

# A host is banned if it has generated "maxretry" during the last "findtime" seconds.
findtime = 30

banaction = iptables-allports-CUSTOM

[sshd]
enabled = true
filter = sshd
logfile = /var/log/auth.log
maxretry = 3

# Fail2Ban custom configuration file.

[DEFAULT]

# "ignoreip" can be an IP address, a CIDR mask or a DNS host

ignoreip = 127.0.0.1 192.168.1.0/24 192.168.2.0/24

# Ban forever => -1

#bantime=-1

# Ban 3 days => 259200

bantime = 259200

# A host is banned if it has generated "maxretry" during the last "findtime" seconds.

findtime = 30

banaction = iptables-allports-CUSTOM

[sshd]

enabled = true

filter = sshd

logfile = /var/log/auth.log

maxretry = 3

3) Remove the default debian jail configuration (is integrated in the above custom jail.local file):

rm -f /etc/fail2ban/jail.d/defaults-debian.conf

1	rm -f /etc/fail2ban/jail.d/defaults-debian.conf

4) Set this cron:

# Daily rotate of ip.blacklist
0 20 * * * tail -100 /etc/fail2ban/ip.blacklist | sort | uniq > /tmp/ip.blacklist ; cat /tmp/ip.blacklist > /etc/fail2ban/ip.blacklist ; rm -f /tmp/ip.blacklist > /dev/null 2>&1

1 2	# Daily rotate of ip.blacklist 0 20 * * * tail -100 /etc/fail2ban/ip.blacklist \| sort \| uniq > /tmp/ip.blacklist ; cat /tmp/ip.blacklist > /etc/fail2ban/ip.blacklist ; rm -f /tmp/ip.blacklist > /dev/null 2>&1

5) Run the cron manually once, just to be sure all works AND to have an empty file

6) Restart Fail2ban … and good luck 😉

Systemd – find what’s wrong with systemctl

True: all the last changes in Linux distro didn’t make me really really happy.
I still like to use init.d to start a process (it took me a while to get used to service yourservice status syntax) and so.

Anyway, the main big ones don’t seem to look back, and we need to get used to this 🙂

I have few raspberry PIs at home, and I’ve noticed that after a restart I was experiencing different weird behaviours. The main two:

stuck and not rebooting
receiving strange logrotate email alerts (e.g. /etc/cron.daily/logrotate:
gzip: stdin: file size changed while zipping)

I tried to ignore them, but when you issue a reboot from a remote place and it doesn’t reboot, you understand that you should start to check what’s going on, instead of just unplug-replug your PI.

And here the discovery: systemctl

This magic command was able to show me the processes with issues, and slowly find out what was wrong with logrotate or my reboot. Or, better, I have realised that after fixing what was marked as failed, I didn’t experience any weird behaviors.

So, here few steps that I’d like to share – to help maybe someone else in the future, myself included – as I tend to forget things if I don’t use them 🙂

To check if your system is healthy or not:

systemctl status

1	systemctl status

Output should return “running”. If you get “degraded”, well, there is definitely something wrong.

Use the following to check what has failed:

systemctl --failed

1	systemctl --failed

Now, investigate those specific processes. Try to analyse their status and logs or literally try to restart them to see live what is the error:

systemctl status <broken_service>

journalctl _PID=<PID_of_broken_service>

tail /var/log/<broken_service>

systemctl restart <broken_service>

systemctl status <broken_service>

journalctl _PID=<PID_of_broken_service>

tail /var/log/<broken_service>

systemctl restart <broken_service>

After fixing all, I tried to reboot few times and after I was checking again the overall status to make sure it was “running”.

In my case, I had few issues with “systemd-modules-load.service”. This probably related to my dist-upgrade. Some old and no longer existing modules were still listed in /etc/modules and, of course, the service wasn’t able to load them, miserably failing.
I’ve tested each module using modprobe <module_name> and I’ve commented out the ones where failing. Restarted and voila`, status… running!

On another PI I had some issues with Apache, but I can’t remember how I fixed it. Still, the goal of this post is mostly make everyone aware that systemctl can give you some interesting info about the system and you can focus your energies on the failed services.

I admit in totally honesty that I have no much clue why after fixing these failed services, all issues disappeared. In fact, the reboot wasn’t affecting one PI with the same non-existing modules listed, but it was stopping another one during the boot. Again, I could probably troubleshoot further but I have a life to live as well 🙂

Sources:

Automatic Updates on Raspberry Pi

How to configure automatic updates on your raspberry pi and make sure it reboots in the night (if required)

apt-get install unattended-upgrades apt-listchanges

sed -i 's/^\/\/      "o=Raspbian,n=jessie"/      "o=Raspbian,n=jessie"/g' /etc/apt/apt.conf.d/50unattended-upgrades
sed -i 's/^\/\/Unattended-Upgrade::Mail "root";/Unattended-Upgrade::Mail "root";/g' /etc/apt/apt.conf.d/50unattended-upgrades
sed -i 's/^\/\/Unattended-Upgrade::Automatic-Reboot "false";/Unattended-Upgrade::Automatic-Reboot "true";/g' /etc/apt/apt.conf.d/50unattended-upgrades
sed -i 's/^\/\/Unattended-Upgrade::Automatic-Reboot-Time "02:00";/Unattended-Upgrade::Automatic-Reboot-Time "02:00";/g' /etc/apt/apt.conf.d/50unattended-upgrades

tee /etc/apt/apt.conf.d/20auto-upgrades > /dev/null <<EOF
APT::Periodic::Update-Package-Lists "1";
APT::Periodic::Unattended-Upgrade "1";
EOF

apt-get install unattended-upgrades apt-listchanges

sed -i 's/^\/\/ "o=Raspbian,n=jessie"/ "o=Raspbian,n=jessie"/g' /etc/apt/apt.conf.d/50unattended-upgrades

sed -i 's/^\/\/Unattended-Upgrade::Mail "root";/Unattended-Upgrade::Mail "root";/g' /etc/apt/apt.conf.d/50unattended-upgrades

sed -i 's/^\/\/Unattended-Upgrade::Automatic-Reboot "false";/Unattended-Upgrade::Automatic-Reboot "true";/g' /etc/apt/apt.conf.d/50unattended-upgrades

sed -i 's/^\/\/Unattended-Upgrade::Automatic-Reboot-Time "02:00";/Unattended-Upgrade::Automatic-Reboot-Time "02:00";/g' /etc/apt/apt.conf.d/50unattended-upgrades

tee /etc/apt/apt.conf.d/20auto-upgrades > /dev/null <<EOF

APT::Periodic::Update-Package-Lists "1";

APT::Periodic::Unattended-Upgrade "1";

EOF

Check the next day the log /var/log/unattended-upgrades/unattended-upgrades.log to see if it worked 🙂

Source: here

Whatsapp to command your Raspberry Pi and Nagios monitoring

Do you want to command your Raspberry Pi via Whatsapp and have this system monitored and brought up by Nagios in case it dies?

Follow this guide! 🙂

Requirements:

Spare SIM card (number will be used by your Raspberry Pi)
A phone to keep the SIM card on during the registration process only
A Raspberry Pi (Debian 8 recommended)
Nagios

Let’s do it!

Step 1: Put your SIM in the phone and make sure the SIM can receive text messages (no data is required)

Step 2: Install/configure your Raspberry Pi

Installation

Yuwsup

To make all this magic happening, we’re going to use Yowsup

Here some easy steps to install on Raspian: (you can use also pip install yowsup2):

$ sudo apt-get install git python-dev libncurses5-dev
$ git clone git://github.com/tgalal/yowsup.git
$ cd yowsup
$ sudo python setup.py install

$ sudo apt-get install git python-dev libncurses5-dev

$ git clone git://github.com/tgalal/yowsup.git

$ cd yowsup

$ sudo python setup.py install

Once installed, you need to register your phone number, extract the password and use it to configure the following scripts.

To register, create a file called mydetails and add the following (replace country code and phone number accordingly):

cc=44
phone=447711111123

1 2	cc=44 phone=447711111123

After that, run this:

python yowsup-cli registration --config mydetails --requestcode sms

1	python yowsup-cli registration --config mydetails --requestcode sms

You should receive a text on your phone with a 6 digits code (xxx-xxx format). Use the following command to get the password:

python yowsup-cli registration --config mydetails --register xxx-xxx

1	python yowsup-cli registration --config mydetails --register xxx-xxx

After a little while, you should see some output like this:

status: ok

kind: free

pw: 9BkIpOaLpCk1LxuQIK8Vrt6XwNkj=

price: 0.79

price_expiration: 1434674994

currency: GBP

cost: 0.79

expiration: 1463544490

login: 4425784541474

type: new

status: ok

kind: free

pw: 9BkIpOaLpCk1LxuQIK8Vrt6XwNkj=

price: 0.79

price_expiration: 1434674994

currency: GBP

cost: 0.79

expiration: 1463544490

type: new

Grab the pw bit and add append to your mydetails file:

cc=44
phone=447711111123
password=9BkIpOaLpCk1LxuQIK8Vrt6XwNkj=

cc=44

phone=447711111123

password=9BkIpOaLpCk1LxuQIK8Vrt6XwNkj=

Now you can test using the below bash script (demo.sh):

#!/bin/bash
echo -e "\e[31m"
echo 'Once you get the prompt just use /L to go online'
echo 'After that you can send a message in this way:'
echo '/message send 449988776655 "Hello from the Raspberry Pi!"'
echo -e "\e[0m\n"

yowsup-cli demos --yowsup --config mydetails

#!/bin/bash

echo -e "\e[31m"

echo 'Once you get the prompt just use /L to go online'

echo 'After that you can send a message in this way:'

echo '/message send 449988776655 "Hello from the Raspberry Pi!"'

echo -e "\e[0m\n"

yowsup-cli demos --yowsup --config mydetails

All should (hopefully) work! 🙂

Python scripts for yowsup

The following scripts and configurations are based on the following:

the user “piuser” is the one who will run the main scripts
scripts are stored into /home/piuser/WhatsappOnPi/scripts
the user “nagios” will need some extra privileges to run some scripts

In /home/piuser/WhatsappOnPi/scripts create the following scripts:

1) whatsapp.py

This script is the one that keeps layer.py script up and running.

from yowsup.stacks                             import YowStackBuilder
from yowsup.common                             import YowConstants
from yowsup.layers                             import YowLayerEvent
from layer                                     import EchoLayer
from yowsup.layers.auth                        import YowAuthenticationProtocolLayer
from yowsup.layers.coder                       import YowCoderLayer
from yowsup.layers.network                     import YowNetworkLayer
from yowsup.env                                import YowsupEnv
from mysettings import *

#Uncomment to log
#import logging
#logging.basicConfig(level=logging.DEBUG)

CREDENTIALS = (phone, password)

if __name__==  "__main__":
    stackBuilder = YowStackBuilder()

    stack = stackBuilder\
        .pushDefaultLayers(True)\
        .push(EchoLayer)\
        .build()

    stack.setProp(YowAuthenticationProtocolLayer.PROP_CREDENTIALS, CREDENTIALS)       #setting credentials
    stack.broadcastEvent(YowLayerEvent(YowNetworkLayer.EVENT_STATE_CONNECT))          #sending the connect signal
    stack.setProp(YowNetworkLayer.PROP_ENDPOINT, YowConstants.ENDPOINTS[0])           #whatsapp server address
    stack.setProp(YowCoderLayer.PROP_DOMAIN, YowConstants.DOMAIN)
    stack.setProp(YowCoderLayer.PROP_RESOURCE, YowsupEnv.getCurrent().getResource())  #info about us as WhatsApp client

    stack.loop( timeout = 0.5, discrete = 0.5 )

from yowsup.stacks import YowStackBuilder

from yowsup.common import YowConstants

from yowsup.layers import YowLayerEvent

from layer import EchoLayer

from yowsup.layers.auth import YowAuthenticationProtocolLayer

from yowsup.layers.coder import YowCoderLayer

from yowsup.layers.network import YowNetworkLayer

from yowsup.env import YowsupEnv

from mysettings import *

#Uncomment to log

#import logging

#logging.basicConfig(level=logging.DEBUG)

CREDENTIALS = (phone, password)

if __name__== "__main__":

stackBuilder = YowStackBuilder()

stack = stackBuilder\

.pushDefaultLayers(True)\

.push(EchoLayer)\

.build()

stack.setProp(YowAuthenticationProtocolLayer.PROP_CREDENTIALS, CREDENTIALS) #setting credentials

stack.broadcastEvent(YowLayerEvent(YowNetworkLayer.EVENT_STATE_CONNECT)) #sending the connect signal

stack.setProp(YowNetworkLayer.PROP_ENDPOINT, YowConstants.ENDPOINTS[0]) #whatsapp server address

stack.setProp(YowCoderLayer.PROP_DOMAIN, YowConstants.DOMAIN)

stack.setProp(YowCoderLayer.PROP_RESOURCE, YowsupEnv.getCurrent().getResource()) #info about us as WhatsApp client

stack.loop( timeout = 0.5, discrete = 0.5 )

2) layer.py

This script is the main one that you need to customise as you’d like:

# -*- coding: utf-8 -*-
import os, subprocess, time, re
from yowsup.layers                                     import YowLayer
from yowsup.layers.interface                           import YowInterfaceLayer, ProtocolEntityCallback
from yowsup.layers.protocol_messages.protocolentities  import TextMessageProtocolEntity
from yowsup.layers.protocol_receipts.protocolentities  import OutgoingReceiptProtocolEntity
from yowsup.layers.protocol_acks.protocolentities      import OutgoingAckProtocolEntity
from mysettings	import *

ap = set(allowedPersons)

# Message on Start Up
startcommand='yowsup-cli demos -l %s:%s -s %s "*[INFO] System Started*: `uptime`" 2>&1 > /dev/null ' % (phone, password, destphone)
subprocess.call(startcommand , shell=True)



def sendMessage(self, messageProtocolEntity, msg):
   outgoingMessageProtocolEntity = TextMessageProtocolEntity(
      ''+msg+'',
      to = messageProtocolEntity.getFrom())
   self.toLower(outgoingMessageProtocolEntity)

class EchoLayer(YowInterfaceLayer):
    @ProtocolEntityCallback("message")
    def onMessage(self, messageProtocolEntity):
        #send receipt otherwise we keep receiving the same message over and over

        if True:
            receipt = OutgoingReceiptProtocolEntity(messageProtocolEntity.getId(), messageProtocolEntity.getFrom(), 'read', messageProtocolEntity.getParticipant())
            self.toLower(receipt)

        if messageProtocolEntity.getFrom(False) in ap:
             message_body = messageProtocolEntity.getBody().lower().strip(' \t\r\n\0')
             #print (message_body)

             # Local System Control
             if 'help' in (message_body):
		msg='Commands available:\nrestart <device>\nuptime\ndf\nlast\nrouter'
                sendMessage(self, messageProtocolEntity, msg)
	     #elif 'reboot' in (message_body):
             #   result=subprocess.check_output(["sudo", "reboot"])
             #   msg='reboot: '+result+''
             #   sendMessage(self, messageProtocolEntity, msg)
             elif 'uptime' in (message_body):
                result=subprocess.check_output(["uptime"])
                msg=''+result+''
                sendMessage(self, messageProtocolEntity, msg)
             elif 'df' in (message_body):
                result=subprocess.check_output(["df", "-h"])
                msg=''+result+''
                sendMessage(self, messageProtocolEntity, msg)
             elif 'last' in (message_body):
                result=subprocess.check_output(["last"])
                msg=''+result+''
                sendMessage(self, messageProtocolEntity, msg)
             elif 'router' in (message_body):
                result=subprocess.check_output(["/usr/lib/nagios/plugins/check_router_speed"])
                msg=''+result+''
                sendMessage(self, messageProtocolEntity, msg)


             # Reboots Control
             # ---------------
             # this uses a wrapper called 'restart_device'
             # bash script with 'case' that issues specific commands over ssh
             # to restart different hosts

             elif message_body.startswith('restart'):
                cmd = message_body.split('restart', 1)[1]
                if 'skyhub' in (cmd):
		   result=subprocess.check_output(["sudo", "restart_device", "router1"])
                   msg=''+result+''
                   sendMessage(self, messageProtocolEntity, msg)
                elif 'asus8uk' in (cmd):
		   result=subprocess.check_output(["sudo", "restart_device", "router2"])
                   msg=''+result+''
                   sendMessage(self, messageProtocolEntity, msg)
		elif 'raspberrino' in (cmd):
		   result=subprocess.check_output(["sudo", "restart_device", "raspberrypi"])
                   msg=''+result+''
                   sendMessage(self, messageProtocolEntity, msg)
		elif 'raspbxino' in (cmd):
                   result=subprocess.check_output(["sudo", "restart_device", "pbx"])
                   msg=''+result+''
                   sendMessage(self, messageProtocolEntity, msg)
                else:
                   msg='Usage: restart (router1|router2|raspberrypi|pbx)'
                   sendMessage(self, messageProtocolEntity, msg)

             else:
                msg='Command '+messageProtocolEntity.getBody()+' unknown.\nUse: help'
                sendMessage(self, messageProtocolEntity, msg)

        else:
	     # Report
             msg='** Alert**  \nSender: '+messageProtocolEntity.getFrom()+' '+messageProtocolEntity.getBody()+''
             outgoingMessageProtocolEntity = TextMessageProtocolEntity(
                ''+msg+'',
                to = '%s@s.whatsapp.net' % destphone )
             self.toLower(outgoingMessageProtocolEntity)

             # Reply
             msg='No'
             sendMessage(self, messageProtocolEntity, msg)

    @ProtocolEntityCallback("receipt")
    def onReceipt(self, entity):
        ack = OutgoingAckProtocolEntity(entity.getId(), "receipt", entity.getType(), entity.getFrom())
        self.toLower(ack)

100

101

102

103

104

105

106

107

108

109

110

# -*- coding: utf-8 -*-

import os, subprocess, time, re

from yowsup.layers import YowLayer

from yowsup.layers.interface import YowInterfaceLayer, ProtocolEntityCallback

from yowsup.layers.protocol_messages.protocolentities import TextMessageProtocolEntity

from yowsup.layers.protocol_receipts.protocolentities import OutgoingReceiptProtocolEntity

from yowsup.layers.protocol_acks.protocolentities import OutgoingAckProtocolEntity

from mysettings import *

ap = set(allowedPersons)

# Message on Start Up

startcommand='yowsup-cli demos -l %s:%s -s %s "*[INFO] System Started*: `uptime`" 2>&1 > /dev/null ' % (phone, password, destphone)

subprocess.call(startcommand , shell=True)

def sendMessage(self, messageProtocolEntity, msg):

outgoingMessageProtocolEntity = TextMessageProtocolEntity(

''+msg+'',

to = messageProtocolEntity.getFrom())

self.toLower(outgoingMessageProtocolEntity)

class EchoLayer(YowInterfaceLayer):

@ProtocolEntityCallback("message")

def onMessage(self, messageProtocolEntity):

#send receipt otherwise we keep receiving the same message over and over

if True:

receipt = OutgoingReceiptProtocolEntity(messageProtocolEntity.getId(), messageProtocolEntity.getFrom(), 'read', messageProtocolEntity.getParticipant())

self.toLower(receipt)

if messageProtocolEntity.getFrom(False) in ap:

message_body = messageProtocolEntity.getBody().lower().strip(' \t\r\n\0')

#print (message_body)

# Local System Control

if 'help' in (message_body):

msg='Commands available:\nrestart <device>\nuptime\ndf\nlast\nrouter'

sendMessage(self, messageProtocolEntity, msg)

#elif 'reboot' in (message_body):

# result=subprocess.check_output(["sudo", "reboot"])

# msg='reboot: '+result+''

# sendMessage(self, messageProtocolEntity, msg)

elif 'uptime' in (message_body):

result=subprocess.check_output(["uptime"])

msg=''+result+''

sendMessage(self, messageProtocolEntity, msg)

elif 'df' in (message_body):

result=subprocess.check_output(["df", "-h"])

msg=''+result+''

sendMessage(self, messageProtocolEntity, msg)

elif 'last' in (message_body):

result=subprocess.check_output(["last"])

msg=''+result+''

sendMessage(self, messageProtocolEntity, msg)

elif 'router' in (message_body):

result=subprocess.check_output(["/usr/lib/nagios/plugins/check_router_speed"])

msg=''+result+''

sendMessage(self, messageProtocolEntity, msg)

# Reboots Control

# ---------------

# this uses a wrapper called 'restart_device'

# bash script with 'case' that issues specific commands over ssh

# to restart different hosts

elif message_body.startswith('restart'):

cmd = message_body.split('restart', 1)[1]

if 'skyhub' in (cmd):

result=subprocess.check_output(["sudo", "restart_device", "router1"])

msg=''+result+''

sendMessage(self, messageProtocolEntity, msg)

elif 'asus8uk' in (cmd):

result=subprocess.check_output(["sudo", "restart_device", "router2"])

msg=''+result+''

sendMessage(self, messageProtocolEntity, msg)

elif 'raspberrino' in (cmd):

result=subprocess.check_output(["sudo", "restart_device", "raspberrypi"])

msg=''+result+''

sendMessage(self, messageProtocolEntity, msg)

elif 'raspbxino' in (cmd):

result=subprocess.check_output(["sudo", "restart_device", "pbx"])

msg=''+result+''

sendMessage(self, messageProtocolEntity, msg)

else:

msg='Usage: restart (router1|router2|raspberrypi|pbx)'

sendMessage(self, messageProtocolEntity, msg)

else:

msg='Command '+messageProtocolEntity.getBody()+' unknown.\nUse: help'

sendMessage(self, messageProtocolEntity, msg)

else:

# Report

msg='** Alert** \nSender: '+messageProtocolEntity.getFrom()+' '+messageProtocolEntity.getBody()+''

outgoingMessageProtocolEntity = TextMessageProtocolEntity(

''+msg+'',

to = '%[email protected]' % destphone )

self.toLower(outgoingMessageProtocolEntity)

# Reply

msg='No'

sendMessage(self, messageProtocolEntity, msg)

@ProtocolEntityCallback("receipt")

def onReceipt(self, entity):

ack = OutgoingAckProtocolEntity(entity.getId(), "receipt", entity.getType(), entity.getFrom())

self.toLower(ack)

3) mysettings.py

This is included in both scripts and it needs to be updated accordingly:

#################################################################################################
# TO EDIT - add your phone number and password
#
phone="447711111123"
password="9BkIpOaLpCk1LxuQIK8Vrt6XwNkj="
destphone="<your_personal_number>"
#allowedPersons=['<your_personal_number>','<another_number_allowed_to_talk_with_your_pi>']
allowedPersons=['<your_personal_number>']
#
#################################################################################################

#################################################################################################

# TO EDIT - add your phone number and password

phone="447711111123"

password="9BkIpOaLpCk1LxuQIK8Vrt6XwNkj="

destphone="<your_personal_number>"

#allowedPersons=['<your_personal_number>','<another_number_allowed_to_talk_with_your_pi>']

allowedPersons=['<your_personal_number>']

#################################################################################################

Now let’s create a wrapper to start the script: /usr/local/bin/whatsapp_start

#!/bin/bash

# kill process if running
for PID in $(ps aux | grep -i "[p]ython.*whatsapp" | awk '{print $2}') ; do
   kill -9 $PID > /dev/null 2>&1
done

# restart process
python /home/piuser/WhatsappOnPi/scripts/whatsapp.py &

#!/bin/bash

# kill process if running

for PID in $(ps aux | grep -i "[p]ython.*whatsapp" | awk '{print $2}') ; do

kill -9 $PID > /dev/null 2>&1

done

# restart process

python /home/piuser/WhatsappOnPi/scripts/whatsapp.py &

And now let’s append this into /etc/rc.local:

echo "Starting whatsapp service"
su - piuser -c /usr/local/bin/whatsapp_start

1 2	echo "Starting whatsapp service" su - piuser -c /usr/local/bin/whatsapp_start

Done!
Every time we reboot the server, the script will start!

But… what happens if the script dies or something goes wrong?

Answer: Nagios!

Create custom plugin script for Nagios and save it in /usr/lib/nagios/plugins/check_whatsapp

NOTE: Make sure to follow the notes in this script to proper setup visudo

#!/bin/bash

####################################################################################
# user 'nagios' needs to be allowed to run 'netstat' using sudo
# and /usr/local/bin/whatsapp_start to run as 'piuser'
#
# Required lines in visudo:
#
#    Cmnd_Alias WHATSAPP_CMD = /bin/su - piuser -c /usr/local/bin/whatsapp_start
#    nagios ALL=(ALL) NOPASSWD:/bin/netstat,WHATSAPP_CMD
#
####################################################################################

sudo netstat -pant | grep -q "ESTABLISHED.*python"

if [ $? -eq 0 ] ; then
   echo "OK- Whatsapp is up and running"
   exit 0
else
   echo "CRITICAL- Whatsapp service stopped or not connected. Attempting restart."
   sudo su - piuser -c /usr/local/bin/whatsapp_start
   exit 2
fi

#!/bin/bash

####################################################################################

# user 'nagios' needs to be allowed to run 'netstat' using sudo

# and /usr/local/bin/whatsapp_start to run as 'piuser'

# Required lines in visudo:

# Cmnd_Alias WHATSAPP_CMD = /bin/su - piuser -c /usr/local/bin/whatsapp_start

# nagios ALL=(ALL) NOPASSWD:/bin/netstat,WHATSAPP_CMD

####################################################################################

sudo netstat -pant | grep -q "ESTABLISHED.*python"

if [ $? -eq 0 ] ; then

echo "OK- Whatsapp is up and running"

exit 0

else

echo "CRITICAL- Whatsapp service stopped or not connected. Attempting restart."

sudo su - piuser -c /usr/local/bin/whatsapp_start

exit 2

Now let’s enable this script in /etc/nagios/nrpe_local.cfg:

command[check_whatsapp]=/usr/lib/nagios/plugins/check_whatsapp

1	command[check_whatsapp]=/usr/lib/nagios/plugins/check_whatsapp

On the Nagios SERVER, let’s add the new service.
Following my current setup mentioned here, I’m going to add the following in /etc/nagios3/conf.d/hostgroups_services.cfg

#######################################################################################
# Check Whatsapp Service
define hostgroup {
        hostgroup_name  whatsapp-servers
                alias           whatsapp servers
        }
define service {
        hostgroup_name                  whatsapp-servers
        service_description             whatsapp service
	normal_check_interval           5
        retry_check_interval            2
        check_command                   check_nrpe_1arg!check_whatsapp
        use                             generic-service
}

#######################################################################################

#######################################################################################

# Check Whatsapp Service

define hostgroup {

hostgroup_name whatsapp-servers

alias whatsapp servers

}

define service {

hostgroup_name whatsapp-servers

service_description whatsapp service

normal_check_interval 5

retry_check_interval 2

check_command check_nrpe_1arg!check_whatsapp

use generic-service

}

#######################################################################################

When the service is configured, we need to append this service on the host where we want the check to be executed and verified (config in /etc/nagios3/conf.d/hosts.cfg – eg:)

        hostgroups      allservers,ssh-servers,http-servers,whatsapp-servers

1	hostgroups allservers,ssh-servers,http-servers,whatsapp-servers

A couple of restarts/reloads (nagios client and nagios server), and the check should be now visible in the web interface! 🙂

NOTE: You might see “Waiting for this message. This may take a while.” on your Whatsapp while trying to talk with your Pi. And you can wait as much as you like, but it won’t get fixed by itself.

So… how make things working again?
What I’ve done to fix it is:

stopping nagios3 (setup to try to restart Whatsapp script if down)
kill the whatsapp python script running
use the above demo.sh script to send/receive some manual messages
if you can chat (send/receive correctly), you can now stop demo.sh script and start again your whatsapp python script

This always fixed this issue for me 🙂

Apologies for the typos and mistakes. This has been published more as a note for me than a proper how-to

Source: http://www.techradar.com/how-to/computing/how-to-control-a-raspberry-pi-using-whatsapp-1315610/2

Many thanks to Paul for the initial python scripts templates 🙂

Physically restart Sky router via Raspberry Pi

I have a Sky Hub router, the SR102 (black). Previously I had the white version as well.
Nice routers, pretty stable, but badly locked. You can change the SID of your wifi, change the password… not either sure if you can do a proper port forwarding. So… perfect for my mum, but a pain for whoever wants a little bit of extra control.

I had already an ASUS RT-N16 with DD-WRT firmware so I used the DMZ function on the Sky router to have some sort of “link” of the public IP of my broadband directly on the WAN interface of the router. In this way it’s like that is my ASUS router that does the connection and I can play as freely as I want, without caring much about the Sky router.

However, it happens that sometimes you need to give a full reboot to the main Sky router. And maybe do this automatically or via command line/script. And here it’s when things are getting more complicated.

The Sky Hub router allows you to reboot it via HTTP. Using the DMZ anyway will bypass the router itself and forward all the requests to the ASUS router. Also, I have never liked the idea to expose my router management page to the Internet, but I rather prefer to connect via SSH on a Raspberry Pi and issue commands from the terminal (telnet/ssh).

So, beside my multiple attempts to find a way to curl the button on the page, I had to find an alternative way to makes this happen. Of course, it didn’t help either to call the Sky Helpline asking if there was a remote possibility to have telnet enabled.

After a bit of talks on Facebook with some friends, here the solution: Remote Controlled Sockets with Pi-mote

Yes. If I can’t reboot from inside, let’s to that from outside!

The process was pretty straight forward.

First of all, I had to turn off my Raspberry Pi, and plug the “little green piece of board” as mentioned in here

After that, I’ve turned the pi on again, and installed the required packages. Happily I found that there is now the python library available for energenie, so I have installed them as well, making my life easier 🙂

apt-get install python-rpi.gpio python-pip
pip install energenie

1 2	apt-get install python-rpi.gpio python-pip pip install energenie

Once done, I have created these two basic script and I have run one a time, to configure the socket plugs.

Make sure to plug the ONE SOCKET PLUG A TIME and run the relative script.

You can find more information in the previous PDF, but these sockets learn who they are based on which commands they are receiving during the learning mode (enabling keeping the green button pressed for about 5 seconds when switched off). So if you run the first script with both plugs connected and in learning mode, they will do exactly the same, and unless you want to control two sockets at the same time, better to follow the instructions 🙂

Script to configure the first socket:

from energenie import switch_on, switch_off
from time import sleep

# turn the first plug socket ON and then OFF
switch_on(1)
sleep(5)
switch_off(1)

from energenie import switch_on, switch_off

from time import sleep

# turn the first plug socket ON and then OFF

switch_on(1)

sleep(5)

switch_off(1)

Script to configure the second socket:

from energenie import switch_on, switch_off
from time import sleep

# turn the second plug socket ON and then OFF
switch_on(2)
sleep(5)
switch_off(2)

from energenie import switch_on, switch_off

from time import sleep

# turn the second plug socket ON and then OFF

switch_on(2)

sleep(5)

switch_off(2)

And now, my simple script to make… “the magic”: plugs.py

from energenie import switch_on, switch_off
from time import sleep
import sys

if len(sys.argv) == 1:
    print "Use:\n# python plug.py <plug_ID> <ON/OFF>\ne.g. # python plug.py 1 ON"
    exit(1)

else:
    plug = sys.argv[1]
    status = sys.argv[2]

if status.lower() == 'on':
   switch_on(int(plug))
else:
   switch_off(int(plug))

from energenie import switch_on, switch_off

from time import sleep

import sys

if len(sys.argv) == 1:

print "Use:\n# python plug.py <plug_ID> <ON/OFF>\ne.g. # python plug.py 1 ON"

exit(1)

else:

plug = sys.argv[1]

status = sys.argv[2]

if status.lower() == 'on':

switch_on(int(plug))

else:

switch_off(int(plug))

You can use this script to control any sockets (up to 4 – hardware limitation).

And here a bash wrapper (I’m not really good in python sorry) that calls plugs.py and restart the router: restart_sky_router

#!/bin/bash

# This script requires plug.py script

if [ "$EUID" -ne 0 ]
  then echo "Please run as root or use 'sudo'"
  exit
fi

echo "Switching OFF and then ON the physical socket"


# Uses ENERGENIE Radio controlled Sockets
python plug.py 1 off
sleep 10
python plug.py 1 on

#!/bin/bash

# This script requires plug.py script

if [ "$EUID" -ne 0 ]

then echo "Please run as root or use 'sudo'"

exit

echo "Switching OFF and then ON the physical socket"

# Uses ENERGENIE Radio controlled Sockets

python plug.py 1 off

sleep 10

python plug.py 1 on

Now, I can have my Nagios system to check for the speed as documented here and eventually issue restart_sky_router script to see if it fixes the issue. Or simply be able to have a command to integrate in your scripts!

Lighttpd and virtualhosts

Here a quick how to, about how to configure Lighttpd to run with Virtualhosts.
This has been installed and tested on a Raspberry Pi.

apt-get install lighttpd php5 php5-cgi

1	apt-get install lighttpd php5 php5-cgi

Enable modules:

lighttpd-enable-mod auth cgi fastcgi fastcgi-php nagios3 simple-vhost ssl status

1	lighttpd-enable-mod auth cgi fastcgi fastcgi-php nagios3 simple-vhost ssl status

Content of /etc/lighttpd/lighttpd.conf

server.modules = (
        "mod_access",
        "mod_alias",
        "mod_compress",
        "mod_redirect",
#       "mod_rewrite",
)

server.document-root        = "/var/www"
server.upload-dirs          = ( "/var/cache/lighttpd/uploads" )
server.errorlog             = "/var/log/lighttpd/error.log"
server.pid-file             = "/var/run/lighttpd.pid"
server.username             = "www-data"
server.groupname            = "www-data"
server.port                 = 80


index-file.names            = ( "index.php", "index.html", "index.lighttpd.html" )
url.access-deny             = ( "~", ".inc" )
static-file.exclude-extensions = ( ".php", ".pl", ".fcgi" )

compress.cache-dir          = "/var/cache/lighttpd/compress/"
compress.filetype           = ( "application/javascript", "text/css", "text/html", "text/plain" )

# default listening port for IPv6 falls back to the IPv4 port
include_shell "/usr/share/lighttpd/use-ipv6.pl " + server.port
include_shell "/usr/share/lighttpd/create-mime.assign.pl"
include_shell "/usr/share/lighttpd/include-conf-enabled.pl"

server.modules = (

"mod_access",

"mod_alias",

"mod_compress",

"mod_redirect",

# "mod_rewrite",

)

server.document-root = "/var/www"

server.upload-dirs = ( "/var/cache/lighttpd/uploads" )

server.errorlog = "/var/log/lighttpd/error.log"

server.pid-file = "/var/run/lighttpd.pid"

server.username = "www-data"

server.groupname = "www-data"

server.port = 80

index-file.names = ( "index.php", "index.html", "index.lighttpd.html" )

url.access-deny = ( "~", ".inc" )

static-file.exclude-extensions = ( ".php", ".pl", ".fcgi" )

compress.cache-dir = "/var/cache/lighttpd/compress/"

compress.filetype = ( "application/javascript", "text/css", "text/html", "text/plain" )

# default listening port for IPv6 falls back to the IPv4 port

include_shell "/usr/share/lighttpd/use-ipv6.pl " + server.port

include_shell "/usr/share/lighttpd/create-mime.assign.pl"

include_shell "/usr/share/lighttpd/include-conf-enabled.pl"

To easily manage virtual hosts, edit /etc/lighttpd/conf-available/10-simple-vhost.conf

server.modules += ( "mod_simple_vhost" )
simple-vhost.server-root = "/var/www/vhost"
simple-vhost.default-host = "error.default.loc"
simple-vhost.document-root = "/"

server.modules += ( "mod_simple_vhost" )

simple-vhost.server-root = "/var/www/vhost"

simple-vhost.default-host = "error.default.loc"

simple-vhost.document-root = "/"

This configuration above will allow you to manage your virutalhosts simply storing them in a folder under /var/www/vhost
No extra configuration is needed from the server side.
Simply go into /var/www/vhost and create a folder named as the virtualhost you would like to manage.
In this particular case, please make sure to have a folder called error.default.loc with a page inside which will be displayed in case of ANY error.
For example, if you want to manage mysite.example.com, simply do the following:

cd /var/www/vhost
mkdir mysite.example.com
chown www-data:www-data mysite.example.com

cd /var/www/vhost

mkdir mysite.example.com

chown www-data:www-data mysite.example.com

…and put the html/php files inside that new folder! 🙂

To test if our webserver works, you can always use curl command as explained here.

Dynamic DNS update script

Below a script that I’ve created to update your Dynamic DNS service.

It has been tested on Raspian (Raspberry Pi), Ubuntu 18.04 and Debian 9.

It works with Internet.bs, No-ip.com and CloudFlare

https://bitbucket.org/thtieig/dynip_update/src/master/

Also, for who as a router running DD-WRT, here a quick article about how to set it up.

Enjoy! 😉

SSL PASSIVE FTP with virtual users on Raspberry Pi

I found this handy plugin to backup my blog: BackWPup
It has also an interesting feature which is the ability to backup remotely, for example on a FTP server.

So… here we go! 🙂

Few notes:

This uses vsftpd software
It will work ONLY over SSL
Due to SSL encryption, the FTP will also work ONLY in PASSIVE mode (ACTIVE mode is disabled)
This configuration has been made based of the fact that this raspberry pi is behind a router
This will use ONLY virtual users, chroot’ed, to increase the security (vsftpd will use a custom PAM auth file, which won’t lookup in /etc/passwd files – for this reason, any local user attempts to login will fail)
Virtual users usernames and credentials will be stored in a file
There is a workaround in place to avoid some common issues like “500 OOPS: Vsftpd: Refusing to Run With Writable Root Inside Chroot ()” – FYI, allow_writeable_chroot=yes does NOT work on vsftpd version 2.3.5.

Install required packets:

apt-get install vsftpd apache2-utils libpam-pwdfile

1	apt-get install vsftpd apache2-utils libpam-pwdfile

Create SSL certificate:

openssl req -x509 -nodes -days 7300 -newkey rsa:2048 -keyout /etc/ssl/certs/vsftpd.pem -out /etc/ssl/certs/vsftpd.pem
chmod 600 /etc/ssl/certs/vsftpd.pem

1 2	openssl req -x509 -nodes -days 7300 -newkey rsa:2048 -keyout /etc/ssl/certs/vsftpd.pem -out /etc/ssl/certs/vsftpd.pem chmod 600 /etc/ssl/certs/vsftpd.pem

Add a local user with limited access (like no console) that vsfpd will use to run virtual users:

useradd --home /home/vsftpd --gid nogroup -m --shell /bin/false vsftpd

1	useradd --home /home/vsftpd --gid nogroup -m --shell /bin/false vsftpd

Create directory structures for the virtual users:

mkdir -p /space/ftpusers/
chmod a-w /space/ftpusers/
mkdir -p /space/ftpusers/ftp01/rw
chmod a-w /space/ftpusers/ftp01
chown -R vsftpd:nogroup /space/ftpusers/ftp01

mkdir -p /space/ftpusers/

chmod a-w /space/ftpusers/

mkdir -p /space/ftpusers/ftp01/rw

chmod a-w /space/ftpusers/ftp01

chown -R vsftpd:nogroup /space/ftpusers/ftp01

Please note that all new virtual users added need its home directory manually created as per above. Also, due to the chroot option and the current limitation on vsftpd, if you want a user to be able to write in its home directory, you need to create an extra folder. Its root home folder has to be -w. This is a workaround that works 🙂

Setup PAM authentication

Create a new file /etc/pam.d/vsftpd.virtual and add the following:

auth required pam_pwdfile.so pwdfile /etc/vsftpd/vsftpd.users
account required pam_permit.so

1 2	auth required pam_pwdfile.so pwdfile /etc/vsftpd/vsftpd.users account required pam_permit.so

Now, let’s reorder a bit vsftp files in a directory:

mkdir -p /etc/vsftpd
cd /etc/
mv vsftpd.conf vsftpd
ln -s /etc/vsftpd/vsftpd.conf .

mkdir -p /etc/vsftpd

cd /etc/

mv vsftpd.conf vsftpd

ln -s /etc/vsftpd/vsftpd.conf .

Add new users (password max 8 characters):

htpasswd -c -d -b /etc/vsftpd/vsftpd.users ftp01 ftp01password

1	htpasswd -c -d -b /etc/vsftpd/vsftpd.users ftp01 ftp01password

Use the flag -c only the first time to create the file. If you re-use it, the file will be overwritten!
Also the -d flag is required because vsftpd is unable to read MD5 hashed password (default if -d is not used). The downside of this is a password limited to 8 characters.
Openssl could be used to produce a MD5 based BSD password with algorithm 1 using # openssl passwd -1 (not tested)

Let’s configure vsftpd

vi /etc/vsftpd.conf

# Main Settings
listen=YES
listen_port=21
connect_from_port_20=NO
ftpd_banner=Welcome to my FTP :-)
use_localtime=YES
force_dot_files=YES

# FTP Passive settings
pasv_enable=YES
pasv_min_port=1100
pasv_max_port=1150
pasv_addr_resolve=YES
pasv_enable=YES
pasv_addr_resolve=YES
pasv_address=<span style="color: #ff0000;">&lt;EXTERNAL IP or DYN DNS&gt;</span>

# Virtual user settings
local_enable=YES
chroot_local_user=YES
secure_chroot_dir=/var/run/vsftpd/empty
virtual_use_local_privs=YES
guest_enable=YES
guest_username=vsftpd
pam_service_name=vsftpd.virtual
user_sub_token=$USER
local_root=/space/ftpusers/$USER
hide_ids=YES

# Anonymous settings
anonymous_enable=NO
anon_upload_enable=NO
no_anon_password=NO
anon_other_write_enable=NO
anon_mkdir_write_enable=NO

# Write permissions
write_enable=YES
local_umask=022
async_abor_enable=YES

# SSL
ssl_enable=YES
force_local_data_ssl=YES
force_local_logins_ssl=YES
ssl_tlsv1=YES
ssl_sslv2=NO
ssl_sslv3=NO
require_ssl_reuse=NO
ssl_ciphers=HIGH
rsa_cert_file=/etc/ssl/certs/vsftpd.pem

# Logging
xferlog_enable=YES
log_ftp_protocol=NO
syslog_enable=NO
vsftpd_log_file=/var/log/vsftpd.log

vi /etc/vsftpd.conf

# Main Settings

listen=YES

listen_port=21

connect_from_port_20=NO

ftpd_banner=Welcome to my FTP :-)

use_localtime=YES

force_dot_files=YES

# FTP Passive settings

pasv_enable=YES

pasv_min_port=1100

pasv_max_port=1150

pasv_addr_resolve=YES

pasv_enable=YES

pasv_addr_resolve=YES

pasv_address=<span style="color: #ff0000;"><EXTERNAL IP or DYN DNS></span>

# Virtual user settings

local_enable=YES

chroot_local_user=YES

secure_chroot_dir=/var/run/vsftpd/empty

virtual_use_local_privs=YES

guest_enable=YES

guest_username=vsftpd

pam_service_name=vsftpd.virtual

user_sub_token=$USER

local_root=/space/ftpusers/$USER

hide_ids=YES

# Anonymous settings

anonymous_enable=NO

anon_upload_enable=NO

no_anon_password=NO

anon_other_write_enable=NO

anon_mkdir_write_enable=NO

# Write permissions

write_enable=YES

local_umask=022

async_abor_enable=YES

# SSL

ssl_enable=YES

force_local_data_ssl=YES

force_local_logins_ssl=YES

ssl_tlsv1=YES

ssl_sslv2=NO

ssl_sslv3=NO

require_ssl_reuse=NO

ssl_ciphers=HIGH

rsa_cert_file=/etc/ssl/certs/vsftpd.pem

# Logging

xferlog_enable=YES

log_ftp_protocol=NO

syslog_enable=NO

vsftpd_log_file=/var/log/vsftpd.log

Now, on your router, make sure that the module ip_conntrack_ftp is loaded using lsmod command.
This is required for FTP PASSIVE mode to work.
I’ve realised that this can be called also nf_conntrack_ftp.
A good way to check all the alias associated to that netfilter module is using the following command:

# modinfo nf_conntrack_ftp
filename: /lib/modules/3.3.7/kernel/net/netfilter/nf_conntrack_ftp.ko
alias: nfct-helper-ftp
alias: <span style="color: #ff0000;">ip_conntrack_ftp</span>
description: ftp connection tracking helper
author: Rusty Russell &lt;rusty@rustcorp.com.au&gt;
license: GPL
depends: nf_conntrack
intree: Y
vermagic: 3.3.7 mod_unload MIPS32_R1 32BIT
parm: ports:array of ushort
parm: loose:bool

# modinfo nf_conntrack_ftp

filename: /lib/modules/3.3.7/kernel/net/netfilter/nf_conntrack_ftp.ko

alias: nfct-helper-ftp

alias: <span style="color: #ff0000;">ip_conntrack_ftp</span>

description: ftp connection tracking helper

author: Rusty Russell <rusty@rustcorp.com.au>

license: GPL

depends: nf_conntrack

intree: Y

vermagic: 3.3.7 mod_unload MIPS32_R1 32BIT

parm: ports:array of ushort

parm: loose:bool

Also, make sure to setup a port forwarding like as below:

$IPT -t nat -A PREROUTING -p tcp -i $EXTIF -d $EXTIP --dport 21 -j DNAT --to $FTPIP:21 # FTP connection port
$IPT -t nat -A PREROUTING -d $EXTIP -p tcp -m tcp --dport 1100:1150 -j DNAT --to-destination $FTPI # FTP PASS ports
$IPT -A FORWARD -i $EXTIF -d $FTPI -p tcp --dport 21 -j ACCEPT
$IPT -A FORWARD -i $EXTIF -d $FTPI -p tcp --dport 1100:1150 -j ACCEPT

$IPT -t nat -A PREROUTING -p tcp -i $EXTIF -d $EXTIP --dport 21 -j DNAT --to $FTPIP:21 # FTP connection port

$IPT -t nat -A PREROUTING -d $EXTIP -p tcp -m tcp --dport 1100:1150 -j DNAT --to-destination $FTPI # FTP PASS ports

$IPT -A FORWARD -i $EXTIF -d $FTPI -p tcp --dport 21 -j ACCEPT

$IPT -A FORWARD -i $EXTIF -d $FTPI -p tcp --dport 1100:1150 -j ACCEPT

Backup Raspberry Pi SD on your Mac… and restore.

Plug the SD in your Mac.

In the Terminal, as root, use diskutil to identify your SD.
Generally it’s the last in the list, if you’ve just plugged in.

root:~ # diskutil list

1	root:~ # diskutil list

You will see something like this:

In my case, the SD is /dev/disk4. For this reason, I run the following to unmount the whole disk.

root:~ # diskutil umountDisk /dev/disk4
Unmount of all volumes on disk4 was successful

1 2	root:~ # diskutil umountDisk /dev/disk4 Unmount of all volumes on disk4 was successful

Once done, you can create the backup using dd utility, but make sure to change the device from /dev/diskX to /dev/rdiskX, adding the “r“.

root:~ # dd if=/dev/<span style="color: #ff0000;">rdisk4</span> of=/path/to/mypibackup.img bs=1m

1	root:~ # dd if=/dev/<span style="color: #ff0000;">rdisk4</span> of=/path/to/mypibackup.img bs=1m

To restore, of course… invert if (input file) with of (output file)… 🙂

root:~ # dd if=/path/to/mypibackup.img of=/dev/rdisk4 bs=1m

1	root:~ # dd if=/path/to/mypibackup.img of=/dev/rdisk4 bs=1m

My notepad

Chris' IT notes…

Tag Archives: raspberry pi

Auto mount an encrypted IMG file stored on NFS share

Configure NFS client

Create and configure your Encrypted “space”

Create an empty IMG file to host our encrypted space

Fail2ban Debian 9

HOW TO

Systemd – find what’s wrong with systemctl

Automatic Updates on Raspberry Pi

Whatsapp to command your Raspberry Pi and Nagios monitoring

Requirements:

Let’s do it!

Installation

Yuwsup

Python scripts for yowsup

1) whatsapp.py

2) layer.py

3) mysettings.py

Answer: Nagios!

Physically restart Sky router via Raspberry Pi

Lighttpd and virtualhosts

Dynamic DNS update script

SSL PASSIVE FTP with virtual users on Raspberry Pi

Backup Raspberry Pi SD on your Mac… and restore.