Tag Archives: raspberry pi

Fail2ban Debian 9

Scratch pad with conf files to configure Fail2ban on Debian 9

This setup will configure Fail2ban to monitor SSH and keep track of the bad guys. Every time an IP gets banned, it will be stored in /etc/fail2ban/ip.blacklist .
This files gets processed every time Fail2ban restarts.
A cron will sanitise the file daily.

HOW TO

1) Create a custom action file: /etc/fail2ban/action.d/iptables-allports-CUSTOM.conf

2) Create /etc/fail2ban/jail.local

3) Remove the default debian jail configuration (is integrated in the above custom jail.local file):

4) Set this cron:

5) Run the cron manually once, just to be sure all works AND to have an empty file

6) Restart Fail2ban … and good luck 😉

 

 

Systemd – find what’s wrong with systemctl

True: all the last changes in Linux distro didn’t make me really really happy.
I still like to use init.d to start a process (it took me a while to get used to service yourservice status syntax) and so.

Anyway, the main big ones don’t seem to look back, and we need to get used to this 🙂

I have few raspberry PIs at home, and I’ve noticed that after a restart I was experiencing different weird behaviours. The main two:

  • stuck and not rebooting
  • receiving strange logrotate email alerts (e.g. /etc/cron.daily/logrotate:
    gzip: stdin: file size changed while zipping)

I tried to ignore them, but when you issue a reboot from a remote place and it doesn’t reboot, you understand that you should start to check what’s going on, instead of just unplug-replug your PI.

 

And here the discovery: systemctl

This magic command was able to show me the processes with issues, and slowly find out what was wrong with logrotate or my reboot. Or, better, I have realised that after fixing what was marked as failed, I didn’t experience any weird behaviors.

So, here few steps that I’d like to share – to help maybe someone else in the future, myself included – as I tend to forget things if I don’t use them 🙂

To check if your system is healthy or not:

Output should return “running”. If you get “degraded”, well, there is definitely something wrong.

Use the following to check what has failed:

Now, investigate those specific processes. Try to analyse their status and logs or literally try to restart them to see live what is the error:

 

After fixing all, I tried to reboot few times and after I was checking again the overall status to make sure it was “running”.

In my case, I had few issues with “systemd-modules-load.service”. This probably related to my dist-upgrade. Some old and no longer existing modules were still listed in /etc/modules and, of course, the service wasn’t able to load them, miserably failing.
I’ve tested each module using modprobe <module_name> and I’ve commented out the ones where failing. Restarted and voila`, status… running!

On another PI I had some issues with Apache, but I can’t remember how I fixed it. Still, the goal of this post is mostly make everyone aware that systemctl can give you some interesting info about the system and you can focus your energies on the failed services.

I admit in totally honesty that I have no much clue why after fixing these failed services, all issues disappeared. In fact, the reboot wasn’t affecting one PI with the same non-existing modules listed, but it was stopping another one during the boot. Again, I could probably troubleshoot further but I have a life to live as well 🙂

 

Sources:

Automatic Updates on Raspberry Pi

How to configure automatic updates on your raspberry pi and make sure it reboots in the night (if required)

Check the next day the log /var/log/unattended-upgrades/unattended-upgrades.log to see if it worked 🙂

 

Source: here

Whatsapp to command your Raspberry Pi and Nagios monitoring

Do you want to command your Raspberry Pi via Whatsapp and have this system monitored and brought up by Nagios in case it dies?

Follow this guide! 🙂

Requirements:

  • Spare SIM card (number will be used by your Raspberry Pi)
  • A phone to keep the SIM card on during the registration process only
  • A Raspberry Pi (Debian 8 recommended)
  • Nagios

Let’s do it!

Step 1: Put your SIM in the phone and make sure the SIM can receive text messages (no data is required)

Step 2: Install/configure your Raspberry Pi

 

Installation

Yuwsup

To make all this magic happening, we’re going to use Yowsup

Here some easy steps to install on Raspian: (you can use also pip install yowsup2):

Once installed, you need to register your phone number, extract the password and use it to configure the following scripts.

To register, create a file called mydetails and add the following (replace country code and phone number accordingly):

After that, run this:

You should receive a text on your phone with a 6 digits code (xxx-xxx format). Use the following command to get the password:

After a little while, you should see some output like this:

Grab the pw bit and add append to your mydetails file:

Now you can test using the below bash script (demo.sh):

All should (hopefully) work! 🙂

Python scripts for yowsup

The following scripts and configurations are based on the following:

  • the user “piuser” is the one who will run the main scripts
  • scripts are stored into /home/piuser/WhatsappOnPi/scripts
  • the user “nagios” will need some extra privileges to run some scripts

 

In /home/piuser/WhatsappOnPi/scripts create the following scripts:

1) whatsapp.py

This script is the one that keeps layer.py script up and running.

2) layer.py

This script is the main one that you need to customise as you’d like:

3) mysettings.py

This is included in both scripts and it needs to be updated accordingly:

 

Now let’s create a wrapper to start the script:  /usr/local/bin/whatsapp_start

 

And now let’s append this into /etc/rc.local:

Done!
Every time we reboot the server, the script will start!

 

But… what happens if the script dies or something goes wrong?

Answer: Nagios!

Create custom plugin script for Nagios and save it in /usr/lib/nagios/plugins/check_whatsapp

NOTE: Make sure to follow the notes in this script to proper setup visudo

 

Now let’s enable this script in /etc/nagios/nrpe_local.cfg:

 

On the Nagios SERVER, let’s add the new service.
Following my current setup mentioned here, I’m going to add the following in /etc/nagios3/conf.d/hostgroups_services.cfg

When the service is configured, we need to append this service on the host where we want the check to be executed and verified (config in /etc/nagios3/conf.d/hosts.cfg – eg:)

 

A couple of restarts/reloads (nagios client and nagios server), and the check should be now visible in the web interface! 🙂


NOTE: You might see Waiting for this message. This may take a while.” on your Whatsapp while trying to talk with your Pi. And you can wait as much as you like, but it won’t get fixed by itself.

So… how make things working again?
What I’ve done to fix it is:

  • stopping nagios3 (setup to try to restart Whatsapp script if down)
  • kill the whatsapp python script running
  • use the above demo.sh script to send/receive some manual messages
  • if you can chat (send/receive correctly), you can now stop demo.sh script and start again your whatsapp python script

This always fixed this issue for me 🙂


Apologies for the typos and mistakes. This has been published more as a note for me than a proper how-to

Source: http://www.techradar.com/how-to/computing/how-to-control-a-raspberry-pi-using-whatsapp-1315610/2

Many thanks to Paul for the initial python scripts templates 🙂

Physically restart Sky router via Raspberry Pi

I have a Sky Hub router, the SR102 (black). Previously I had the white version as well.
Nice routers, pretty stable, but badly locked. You can change the SID of your wifi, change the password… not either sure if you can do a proper port forwarding. So… perfect for my mum, but a pain for whoever wants a little bit of extra control.

I had already an ASUS RT-N16 with DD-WRT firmware so I used the DMZ function on the Sky router to have some sort of “link” of the public IP of my broadband directly on the WAN interface of the router. In this way it’s like that is my ASUS router that does the connection and I can play as freely as I want, without caring much about the Sky router.

However, it happens that sometimes you need to give a full reboot to the main Sky router. And maybe do this automatically or via command line/script. And here it’s when things are getting more complicated.

The Sky Hub router allows you to reboot it via HTTP. Using the DMZ anyway will bypass the router itself and forward all the requests to the ASUS router. Also, I have never liked the idea to expose my router management page to the Internet, but I rather prefer to connect via SSH on a Raspberry Pi and issue commands from the terminal (telnet/ssh).

So, beside my multiple attempts to find a way to curl the button on the page, I had to find an alternative way to makes this happen. Of course, it didn’t help either to call the Sky Helpline asking if there was a remote possibility to have telnet enabled.

After a bit of talks on Facebook with some friends, here the solution: Remote Controlled Sockets with Pi-mote

Yes. If I can’t reboot from inside, let’s to that from outside!

The process was pretty straight forward.

First of all, I had to turn off my Raspberry Pi, and plug the “little green piece of board” as mentioned in here

After that, I’ve turned the pi on again, and installed the required packages. Happily I found that there is now the python library available for energenie, so I have installed them as well, making my life easier 🙂

Once done, I have created these two basic script and I have run one a time, to configure the socket plugs.

Make sure to plug the ONE SOCKET PLUG A TIME and run the relative script.

You can find more information in the previous PDF, but these sockets learn who they are based on which commands they are receiving during the learning mode (enabling keeping the green button pressed for about 5 seconds when switched off). So if you run the first script with both plugs connected and in learning mode, they will do exactly the same, and unless you want to control two sockets at the same time, better to follow the instructions 🙂

Script to configure the first socket:

 

Script to configure the second socket:

 

And now, my simple script to make… “the magic”: plugs.py

You can use this script to control any sockets (up to 4 – hardware limitation).

And here a bash wrapper (I’m not really good in python sorry) that calls plugs.py and restart the router: restart_sky_router

 

Now, I can have my Nagios system to check for the speed as documented here and eventually issue restart_sky_router script to see if it fixes the issue. Or simply be able to have a command to integrate in your scripts!

 

Lighttpd and virtualhosts

Here a quick how to, about how to configure Lighttpd to run with Virtualhosts.
This has been installed and tested on a Raspberry Pi.

Enable modules:

Content of /etc/lighttpd/lighttpd.conf

To easily manage virtual hosts, edit /etc/lighttpd/conf-available/10-simple-vhost.conf

This configuration above will allow you to manage your virutalhosts simply storing them in a folder under /var/www/vhost
No extra configuration is needed from the server side.
Simply go into /var/www/vhost and create a folder named as the virtualhost you would like to manage.
In this particular case, please make sure to have a folder called error.default.loc with a page inside which will be displayed in case of ANY error.
For example, if you want to manage mysite.example.com, simply do the following:

…and put the html/php files inside that new folder! 🙂

To test if our webserver works, you can always use curl command as explained here.

Dynamic DNS update script

Here a script that I’ve created to update your Dynamic DNS service.
You can run it manually or put in cron to run every few minutes.
It sends the update ONLY if the IP has changed. So you will avoid any “abuse” error, in case of too many attempts to update the IP.

This script currently works with Internet.bs and NO-IP.com services.

It requires curl package.
Tested on Raspberry Pi and Debian stable distros.

This is composed by 2 files:
Config file: /etc/dynip_update.conf

Script file: /usr/local/bin/dynip_update

Also, for who as a router running DD-WRT, here a quick article about how to set it up.

Enjoy! 😉

SSL PASSIVE FTP with virtual users on Raspberry Pi

I found this handy plugin to backup my blog: BackWPup
It has also an interesting feature which is the ability to backup remotely, for example on a FTP server.

So… here we go! 🙂

Few notes:

  • This uses vsftpd software
  • It will work ONLY over SSL
  • Due to SSL encryption, the FTP will also work ONLY in PASSIVE mode (ACTIVE mode is disabled)
  • This configuration has been made based of the fact that this raspberry pi is behind a router
  • This will use ONLY virtual users, chroot’ed, to increase the security (vsftpd will use a custom PAM auth file, which won’t lookup in /etc/passwd files – for this reason, any local user attempts to login will fail)
  • Virtual users usernames and credentials will be stored in a file
  • There is a workaround in place to avoid some common issues like “500 OOPS: Vsftpd: Refusing to Run With Writable Root Inside Chroot ()” – FYI, allow_writeable_chroot=yes does NOT work on vsftpd version 2.3.5.

Install required packets:

Create SSL certificate:

Add a local user with limited access (like no console) that vsfpd will use to run virtual users:

Create directory structures for the virtual users:

Please note that all new virtual users added need its home directory manually created as per above. Also, due to the chroot option and the current limitation on vsftpd, if you want a user to be able to write in its home directory, you need to create an extra folder. Its root home folder has to be -w. This is a workaround that works 🙂

Setup PAM authentication

Create a new file /etc/pam.d/vsftpd.virtual and add the following:

Now, let’s reorder a bit vsftp files in a directory:

Add new users (password max 8 characters):

Use the flag -c only the first time to create the file. If you re-use it, the file will be overwritten!
Also the -d flag is required because vsftpd is unable to read MD5 hashed password (default if -d is not used). The downside of this is a password limited to 8 characters.
Openssl could be used to produce a MD5 based BSD password with algorithm 1 using # openssl passwd -1 (not tested)

Let’s configure vsftpd

Now, on your router, make sure that the module ip_conntrack_ftp is loaded using lsmod command.
This is required for FTP PASSIVE mode to work.
I’ve realised that this can be called also nf_conntrack_ftp.
A good way to check all the alias associated to that netfilter module is using the following command:

Also, make sure to setup a port forwarding like as below:

Backup Raspberry Pi SD on your Mac… and restore.

Plug the SD in your Mac.

In the Terminal, as root, use diskutil to identify your SD.
Generally it’s the last in the list, if you’ve just plugged in.

You will see something like this:
diskutil_list_pi_sd

In my case, the SD is /dev/disk4. For this reason, I run the following to unmount the whole disk.

Once done, you can create the backup using dd utility, but make sure to change the device from /dev/diskX to /dev/rdiskX, adding the “r“.

To restore, of course… invert if (input file) with of (output file)… 🙂

Mac – Time Machine on a Raspberry Pi

Requirements

First of all, you need the following:

  • a raspberry pi installed with 1 USB port available,
  • 1 external USB drive (I’ve used a 3TB external USB drive), pre-formatted under your Mac in HSF+, labelled as TimeMachine
  • internet connection 🙂
  • …and some time to perform all the steps and the initial backup.

Installation of your pi

As root, run the following to upgrade the system and install the required packages:

Now all the required debian packages are installed. We will install netatalk from the source code, because the debian packages are very old.

Setup the hard drive

Create a mount point as root:

Connect your external USB drive to your pi and run as root the following:

The output should be something like this:

Please take note of the UUID where you can see your hard drive installed and add the following line in your /etc/fstab file:

Now your drive will be automatically mounted at each reboot of the machine.

Install and configure Netatalk demon

Here you have 3 options:

  1. You can follow this guide if you want to create a .deb package which is easier manageable, especially on Debian;
  2. (For the lazy ones) download my precompiled deb file of version 3.1.1 from here, or 3.1.7 version from here, unzip and install it using dpkg -i
  3. Otherwise… just follow the next steps 🙂
Compile and install Netatalk

Download the latest stable code from here.
I’ve personally used version 3.0.4 for a while, and now I’ve upgraded to version 3.1.7.

Decompress the file, enter the folder and run the following:

If no errors:

As root:

Configure afp.conf

Based on your version, you could have the configuration files in different locations.
Use the command afpd -v | grep "afp.conf"  to locate your afp.conf file.

Edit afp.conf based on your network configuration:

I’ve setup also an extra bit called [Homes] which shares the user /home directory using the same protocol. Quite handy to move around files 🙂 But this last part is NOT required for the scope of this post.

Now we can mount the drive and restart the services:

Now the disk should be visible from your Mac and Time Machine application 🙂

First Time Machine backup

For the first Time Machine backup, I would really recommend to connect the external drive directly to the USB port of your Mac and follow these steps.

Before disconnect the drive from your pi, make sure to stop the services and unmount properly the disk:

Make sure the output does NOT show /TimeMachine mounted.

Create a sparsebundle manually

Connect the USB drive to your Mac and from the Terminal do the following:

NAME is the hostname
XXXXXXXXXX is your MAC Address without “:”
200g means 1.5 TB (1500 GB). Please make sure it’s big enough to contain the first backup. After it will expand automatically. But if you set it less than the space required for the first backup, Time Machine might ignore it and create a normal folder structure, typical of USB hard drives.
Best practise is set about 80% of the capacity of your hard drive 😉

Start the first backup

Then, add this disk to Time Machine and let’s the backup start and finish.
For the fist backup, if it’s around 1Tb, be ready to leave the PC on all day.

Make sure that NO directories are created in the USB disk.
Just check the size of the sparsebundle file during the initial backup process. If it increases, it’s working! 🙂

I have to admit that I was doing something else when I’ve launched the backup and I’ve realised (I guess) that the sparsebundle file got renamed by Time Machine in a format like <hostname>.sparsebundle. Not sure why but… well, all worked so… from now on, I will use this new name for that file. :-p

If the backup doesn’t start automatically using the sparsebundle file, you can try to force it following these steps:

1) double click on your file sparsebunle to mount it
2) From Terminal, as superuser, run

3) Now force to run a new backup

Once completed, just take notes of the current permission of that file. You can check via terminal running ls -l /Volumes/TimeMachine/.
Now unmount the hard drive and reconnect it to your Raspberry Pi.

Final setup on the Raspberry Pi

Before mounting, do a quick check running the following (/dev/sdb2 is my Time Machine drive – please review the output of blkid if unsure):

If all is fine, you can mount the drive:

And now, do a little trick to make sure to avoid permissions issues, and allow Time Machine to be able to write into the sparsebundle “file” (you will realise on Linux that it’s in fact a directory tree with files).

We’re going to create the same user and group that we have on the Mac on our Raspberry Pi.
Run a handy ls -l /TimeMachine and take note of the UID and GID.
If the group shows as numeric value, as root, you can run groupadd -g GID MACGROUP .

NOTE:GID is the value that you’ve seen with the previous command and MACGROUP is the name of the group that you saw running ls -l /Volumes/TimeMachine/ on you Mac Terminal.

Now, as root, create a new user that matches with the Mac user.

NOTE: Same as before, UID is the value that you’ve seen with the previous command and MACUSER is the name of the user that you saw running ls -l /Volumes/TimeMachine/ on you Mac Terminal.

Now check again with ls -l /TimeMachine and you should see something like this: drwxr-xr-x 1 macuser staff 11 Apr 2 10:22 HOSTNAME.sparsebundle

NOTE: The group might be displayed as ‘dialout’ (or something else) instead of ‘staff’ as on your Mac. This is not a problem, as long as the numeric GID matches. If the group still shows as numeric value, it groupadd -g GID ‘staff’

All good? Then let’s restart netatalk:

And now, we’re ready to:

  1. Add the remote disk “TimeMachine” to Time Machine;
  2. Initialise a second backup, just to make sure that all works fine (this will be probably less than 1GB);
  3. Have a drink as reward after all this work 🙂

 

Sources:
http://garmoncheg.blogspot.com.au/2012/11/time-capsule-for-25.html
http://administeria.com/?s=time+machine
http://buffalo.nas-central.org/wiki/Time_Machine_&_Time_Capsule_support_on_your_LinkStation