Networking – IP Address classes explained

IP address classes

IP addresses can be broken down into classes. These classes are A, B, C, D, E and their possible ranges can be seen below.

Class Start address Finish address
A 0.0.0.0 126.255.255.255
B 128.0.0.0 191.255.255.255
C 192.0.0.0 223.255.255.255
D 224.0.0.0 239.255.255.255
E 240.0.0.0 255.255.255.255

If you look at the table you may notice something strange. The range of IP address from Class A to Class B skips the 127.0.0.0-127.255.255.255 range. That is because this range is reserved for the special addresses called Loopback addresses that have already been discussed above.

The rest of classes are allocated to companies and organizations based upon the amount of IP addresses that they may need. Listed below are descriptions of the IP classes and the organizations that will typically receive that type of allocation.

Default Network: The special network 0.0.0.0 is generally used for routing.

Class A: From the table above you see that there are 126 class A networks. These networks consist of 16,777,214 possible IP addresses that can be assigned to devices and computers. This type of allocation is generally given to very large networks such as multi-national companies.

Loopback: This is the special 127.0.0.0 network that is reserved as a loopback to your own computer. These addresses are used for testing and debugging of your programs or hardware.

Class B: This class consists of 16,384 individual networks, each allocation consisting of 65,534 possible IP addresses. These blocks are generally allocated to Internet Service Providers and large networks, like a college or major hospital.

Class C: There is a total of 2,097,152 Class C networks available, with each network consisting of 255 individual IP addresses. This type of class is generally given to small to mid-sized companies.

Class D: The IP addresses in this class are reserved for a service called Multicast.

Class E: The IP addresses in this class are reserved for experimental use.

Broadcast: This is the special network of 255.255.255.255, and is used for broadcasting messages to the entire network that your computer resides on.

Private Addresses

There are also blocks of IP addresses that are set aside for internal private use for computers not directly connected to the Internet. These IP addresses are not supposed to be routed through the Internet, and most service providers will block the attempt to do so. These IP addresses are used for internal use by company or home networks that need to use TCP/IP but do not want to be directly visible on the Internet. These IP ranges are:

Class
Private Start Address
Private End Address
A
10.0.0.0
10.255.255.255
B
172.16.0.0
172.31.255.255
C
192.168.0.0
192.168.255.255

If you are on a home/office private network and want to use TCP/IP, you should assign your computers/devices IP addresses from one of these three ranges. That way your router/firewall would be the only device with a true IP address which makes your network more secure.

Full source (and loads of thanks):  http://www.bleepingcomputer.com/tutorials/ip-addresses-explained/

 

… some extra summary table from here:

Class 1st Octet Decimal Range 1st Octet High Order Bits Network/Host ID (N=Network, H=Host) Default Subnet Mask Number of Networks Hosts per Network (Usable Addresses)
A 1 – 126* 0 N.H.H.H 255.0.0.0 126 (27 – 2) 16,777,214 (224 – 2)
B 128 – 191 10 N.N.H.H 255.255.0.0 16,382 (214 – 2) 65,534 (216 – 2)
C 192 – 223 110 N.N.N.H 255.255.255.0 2,097,150 (221– 2) 254 (28 – 2)
D 224 – 239 1110 Reserved for Multicasting
E 240 – 254 1111 Experimental; used for research

Note: Class A addresses 127.0.0.0 to 127.255.255.255 cannot be used and is reserved for loopback and diagnostic functions.

Private IP Addresses

Class Private Networks Subnet Mask Address Range
A 10.0.0.0 255.0.0.0 10.0.0.0 – 10.255.255.255
B 172.16.0.0 – 172.31.0.0 255.240.0.0 172.16.0.0 – 172.31.255.255
C 192.168.0.0 255.255.0.0 192.168.0.0 – 192.168.255.255

Regex – Grep reference

 

Regex operator

Meaning

.

Matches any single character.

?

The preceding item is optional and will be matched, at most, once.

*

The preceding item will be matched zero or more times.

+

The preceding item will be matched one or more times.

{N}

The preceding item is matched exactly N times.

{N,}

The preceding item is matched N or more times.

{N,M}

The preceding item is matched at least N times, but not more than M times.

Represents the range if it’s not first or last in a list or the ending point of a range in a list.

^

Matches the empty string at the beginning of a line; also represents the characters not in the range of a list.

$

Matches the empty string at the end of a line.

\b

Matches the empty string at the edge of a word.

\B

Matches the empty string provided it’s not at the edge of a word.

\<

Match the empty string at the beginning of word.

\>

Match the empty string at the end of word.

(a|b)

Match a or b

[abc]

Range (a or b or c)

[^abc]

NOT a or b or c

[a-z]

Any letter (lowercase) from a to z

[A-Z]

Any letter (uppercase) from a to z

[a-zA-Z]

Any letter (upper and lowercase) from a to z

[0-9]

Any digit from 0 to 9

A most exhaustive cheat sheet is available here

Netcat – such a powerful ‘cat’!

I was just looking around info about netcat and telnet, trying to understand a bit more. Well… in few words: no point to install telnet if you have netcat! 🙂 Netcat is perfect for scripting, ’cause it’s non-interactive, UDP/TCP capable, can be a listener as well… very powerful tool. Here some example.

How to check if your httpd is up and running:

~ $ nc -zv localhost 80
Connection to localhost 80 port [tcp/http] succeeded!

…and it closes gracefully 😉

How to check port-range ports:

~ $ nc -zv localhost 20-25
nc: connect to localhost port 20 (tcp) failed: Connection refused
Connection to localhost 21 port [tcp/ftp] succeeded!
Connection to localhost 22 port [tcp/ssh] succeeded!
nc: connect to localhost port 23 (tcp) failed: Connection refused
nc: connect to localhost port 24 (tcp) failed: Connection refused
nc: connect to localhost port 25 (tcp) failed: Connection refused

… or a list of ports:

$ nc -zv localhost 20 22 80 443
nc: connect to localhost port 20 (tcp) failed: Connection refused
Connection to localhost 22 port [tcp/ssh] succeeded!
Connection to localhost 80 port [tcp/http] succeeded!
Connection to localhost 443 port [tcp/https] succeeded!

NOTE: If you want to grep or play with the “output” of the command, you need to use 2>&1
For example:

nc -zv localhost 1-1024 <strong>2>&1</strong> | grep succeeded

How to check the service that’s running on that port:

(From man) Alternatively, it might be useful to know which server software is running, and which versions. This information is often contained within the greeting banners. In order to retrieve these, it is necessary to first make a connection, and then break the connection when the banner has been retrieved. This can be accomplished by specifying a small timeout with the -w flag, or perhaps by issuing a “QUIT” command to the server:

$ echo "QUIT" | nc host.example.com 20-30
SSH-1.99-OpenSSH_3.6.1p2
Protocol mismatch.
220 host.example.com IMS SMTP Receiver Version 0.84 Ready

In some cases, it’s handy to add -q 1 at the end, if nc hangs (I’ve noticed this in some cases) Like this:

$ echo "QUIT" | nc host.example.com 20-30 <strong>-q 1</strong>

Or how to send/receive a file:

On the receiver side:

$ nc -l 1234 > /tmp/file_to_receive

On the sender side:

$ cat file_to_send | nc receiver_ip_or_fqdn 1234

or

$ nc receiver_ip_or_fqdn 1234 < file_to_send

There are plenty of things that you can do. These are just simple examples… enjoy! 🙂

Lighttpd and virtualhosts

Here a quick how to, about how to configure Lighttpd to run with Virtualhosts.
This has been installed and tested on a Raspberry Pi.

apt-get install lighttpd php5 php5-cgi

Enable modules:

lighttpd-enable-mod auth cgi fastcgi fastcgi-php nagios3 simple-vhost ssl status

Content of /etc/lighttpd/lighttpd.conf

server.modules = (
        "mod_access",
        "mod_alias",
        "mod_compress",
        "mod_redirect",
#       "mod_rewrite",
)

server.document-root        = "/var/www"
server.upload-dirs          = ( "/var/cache/lighttpd/uploads" )
server.errorlog             = "/var/log/lighttpd/error.log"
server.pid-file             = "/var/run/lighttpd.pid"
server.username             = "www-data"
server.groupname            = "www-data"
server.port                 = 80


index-file.names            = ( "index.php", "index.html", "index.lighttpd.html" )
url.access-deny             = ( "~", ".inc" )
static-file.exclude-extensions = ( ".php", ".pl", ".fcgi" )

compress.cache-dir          = "/var/cache/lighttpd/compress/"
compress.filetype           = ( "application/javascript", "text/css", "text/html", "text/plain" )

# default listening port for IPv6 falls back to the IPv4 port
include_shell "/usr/share/lighttpd/use-ipv6.pl " + server.port
include_shell "/usr/share/lighttpd/create-mime.assign.pl"
include_shell "/usr/share/lighttpd/include-conf-enabled.pl"

To easily manage virtual hosts, edit /etc/lighttpd/conf-available/10-simple-vhost.conf

server.modules += ( "mod_simple_vhost" )
simple-vhost.server-root = "/var/www/vhost"
simple-vhost.default-host = "error.default.loc"
simple-vhost.document-root = "/"

This configuration above will allow you to manage your virutalhosts simply storing them in a folder under /var/www/vhost
No extra configuration is needed from the server side.
Simply go into /var/www/vhost and create a folder named as the virtualhost you would like to manage.
In this particular case, please make sure to have a folder called error.default.loc with a page inside which will be displayed in case of ANY error.
For example, if you want to manage mysite.example.com, simply do the following:

cd /var/www/vhost
mkdir mysite.example.com
chown www-data:www-data mysite.example.com

…and put the html/php files inside that new folder! 🙂

To test if our webserver works, you can always use curl command as explained here.

Puppet – Let’s start

Puppet is a quite powerful configuration manager tool which allows you to configure automatically hosts and keep configurations consistence.

I did some tests using 3 VMs:

  • puppetmaster (server)
  • puppetagent01 (client)
  • puppetagent02 (client)

Of course, most of the work is done on puppetmaster server. On the last two machines you will simply see the outcome of the configurations that you’re going do set on puppetmaster.

Important: all the machines have to be able to communicate between each others. Please make sure DNS is working or set local names/IPs in /etc/hosts file, and do some ping tests before proceeding.

Client setup

On each puppetagent machine, just install the package puppet

apt-get install puppet

By default, the client will look for a host called “puppet” on the network.
If your DNS/hosts file doesn’t have this entry, and it can’t be resolved, you can manually set the name of the puppetmaster in /etc/puppet/puppet.conf file, adding this line under [main] section:

server=puppetmaster.yournet.loc

Now, no more configuration is required from the client side. Just edit /etc/default/puppet to start at boot time and start the service.

# Defaults for puppet - sourced by /etc/init.d/puppet

# Start puppet on boot?
START=yes

# Startup options
DAEMON_OPTS=""

 

service puppet start

Starting the service, will make automatically a request to the server to be added under his control.

If you want to do some tests, you can eventually use the following command to run puppet only once. This will also force the polling updates, which by default runs every 30 minutes.

puppet agent --no-daemonize --onetime --verbose

You can repeat all these steps on the second client machine.

Server setup

apt-get install puppetmaster

Check if the service is running, otherwise, start it up.

Sign clients’ certificates on the server side

Puppet uses this client/server certificate sign system to add/remove hosts from being managed by the server.

To see who has requested to be “controlled” use this command:

puppet cert --list

This will show all the hosts waiting to be added under puppetmaster server.

puppet cert --sign

This command will add the host.

Puppetmaster configuration files

The main configuration file is /etc/puppet/manifests/site.pp

Inside manifests folder, I’ve created a subfolder called classes with extra definitions (content of these files is showed later in this post).

/etc/puppet/manifests# tree
.
|___ classes
|   |___ apache.pp
|   |___ mysite.pp
|   |___ ntpd.pp
|   |___ packages.pp
|___ site.pp

/etc/puppet/manifests/site.pp

import 'classes/*.pp'
# This add all the custom .pp files into classes folder
class puppettools {
# Creates a file, setting permissions and content
        file { '/usr/local/sbin/puppet_once.sh':
                owner => root, group => root, mode => 755,
                content => "#!/bin/sh\npuppet agent --no-daemonize --onetime --verbose $1\n",
        }
# Install (if not present) some puppet modules required for 'vimconf' class
        exec { "install_puppet_module":
        command => "puppet module install puppetlabs-stdlib",
        path => [ "/bin", "/sbin", "/usr/bin", "/usr/sbin",
              "/usr/local/bin", "/usr/local/sbin" ],
        onlyif  => "test `puppet module list | grep puppetlabs-stdlib | wc -l` -eq 0"
        }
}

class vimconf {
# Modify vimrc conf file, enabling syntax on
        file_line { 'vim_syntax_on':
        path  => '/etc/vim/vimrc',
        match => '^.*syntax on.*$',
        line  => 'syntax on',
        }
}

node  default {
# this will be applied to all nodes without specific node definitions
        include packages
        include vimconf
        include ntp
        include puppettools
}

node  'puppetagent01' inherits default {
# this specific node, gets all the default classes PLUS some extras
        include mysite
}

Here the content of the single files .pp in classes folder:

class apache {
	package { 'apache2-mpm-prefork':
		ensure => installed
	}

	service { 'apache2':
		ensure => running,
		hasstatus => true,
		hasrestart => true,
	}
}

 

class mysite {

	include apache

	file { '/etc/apache2/sites-available/mysite':
		owner => root, group => root, mode => 0644,
		source => "puppet:///files/mysite/mysite_apache.conf",
	}

	file {'/var/www/mysite.localdomain':
		ensure => directory,
	}

	file {'/var/www/mysite.localdomain/index.html':
                owner => root, group => www-data, mode => 0755,
                source => "puppet:///files/mysite/index.html",
	}

	 exec {'/usr/sbin/a2dissite * ; /usr/sbin/a2ensite mysite':
            	onlyif => '/usr/bin/test -e /etc/apache2/sites-available/mysite',
		notify => Service['apache2'],
	}
}

 

class ntp {
		package { ntp: ensure => present }
		file { "/etc/ntp.conf":
			owner	 => root,
			group	 => root,
			mode	=> 444,
			backup => false,
			source	=> "puppet:///files/etc/ntp.conf",
			require => Package["ntp"],
                        notify  => Service["ntp"],
		}
		service { "ntp":
			enable => true ,
			ensure => running,
			subscribe => [Package[ntp], File["/etc/ntp.conf"],],
		}
	}

 

class packages  {
        Package { ensure => "installed" }

        package { "screen": }
        package { "dselect": }
        package { "vim": }
        package { "curl": }
}

 

It’s important to remember to NOT duplicate entries.
For example, in this case, we have a specific file where we have setup ntp service, including the required package. This means that we do NOT have to add this package in the list into packages.pp, otherwise you will get an error and configs won’t get pushed.

As I’m sure you’ve noted, there are references to some “files”.
Yes, we need some extra configuration, to tell puppet to run as file server as well and where files are located.

In our example we are storing our files in here:

mkdir -p /etc/puppet/files

Now we need to add the following in /etc/puppet/fileserver.conf

[files]
  path /etc/puppet/files
allow *

Last bit, is creating the subfolders and place the files required for our configuration:

mkdir -p /etc/puppet/files 
cd /etc/puppet/files 
mkdir mysite mkdir etc

Inside mysite create mysite_apache.conf and index.html files.

Example mysite_apache.conf

<VirtualHost *:80> 
  ServerName mysite.localdomain 
  DocumentRoot /var/www/mysite.localdomain 
</VirtualHost>

For index.html, you can simply have some text, just for testing purposes.

In this example, we have also setup ntp to be installed and to have a custom ntp.conf file pushed.
For this reason, we need to make sure to have this file present into /etc/puppet/files/etc as declared into our .pp file.

After doing all these changes, you should restart your puppetmaster service on the server.

If all went well, you should have the following:

  • puppetagent02 host with screen, dselect, vim (installed and with syntax on), ntp (installed, running with custom ntp.conf file)
  • puppetagent01: with the same as puppetagent02 PLUS apache with a running website

Of course this is just a raw example and you can use template and other super features.
But I think it’s a good start 😉

 

Sources:


https://forge.puppetlabs.com/puppetlabs/stdlib
http://finninday.net/wiki/index.php/Zero_to_puppet_in_one_day
http://www.puppetcookbook.com/
http://foaa.de/old-blog/2010/07/playing-with-puppets-on-debian/trackback/index.html
http://www.harker.com/puppet/BayLISA100715.html
http://docs.puppetlabs.com/puppet/latest/reference/lang_relationships.html

Dynamic DNS update script

Below a script that I’ve created to update your Dynamic DNS service.

It has been tested on Raspian (Raspberry Pi), Ubuntu 18.04 and Debian 9.

It works with Internet.bs, No-ip.com and CloudFlare

https://bitbucket.org/thtieig/dynip_update/src/master/

Also, for who as a router running DD-WRT, here a quick article about how to set it up.

Enjoy! 😉

Mac – Enable syntax highlight VIM

Instert these lines below the “set backspace=2” in your /usr/share/vim file:

set ai                  " auto indenting
set history=100         " keep 100 lines of history
set ruler               " show the cursor position
syntax on               " syntax highlighting
set hlsearch            " highlight the last searched term
filetype plugin on      " use the file type plugins

" When editing a file, always jump to the last cursor position
autocmd BufReadPost *
\ if ! exists("g:leave_my_cursor_position_alone") |
\ if line("'\"") > 0 && line ("'\"") <= line("$") |
\ exe "normal g'\"" |
\ endif |
\ endif

Source

SSL PASSIVE FTP with virtual users on Raspberry Pi

I found this handy plugin to backup my blog: BackWPup
It has also an interesting feature which is the ability to backup remotely, for example on a FTP server.

So… here we go! 🙂

Few notes:

  • This uses vsftpd software
  • It will work ONLY over SSL
  • Due to SSL encryption, the FTP will also work ONLY in PASSIVE mode (ACTIVE mode is disabled)
  • This configuration has been made based of the fact that this raspberry pi is behind a router
  • This will use ONLY virtual users, chroot’ed, to increase the security (vsftpd will use a custom PAM auth file, which won’t lookup in /etc/passwd files – for this reason, any local user attempts to login will fail)
  • Virtual users usernames and credentials will be stored in a file
  • There is a workaround in place to avoid some common issues like “500 OOPS: Vsftpd: Refusing to Run With Writable Root Inside Chroot ()” – FYI, allow_writeable_chroot=yes does NOT work on vsftpd version 2.3.5.

Install required packets:

apt-get install vsftpd apache2-utils libpam-pwdfile

Create SSL certificate:

openssl req -x509 -nodes -days 7300 -newkey rsa:2048 -keyout /etc/ssl/certs/vsftpd.pem -out /etc/ssl/certs/vsftpd.pem
chmod 600 /etc/ssl/certs/vsftpd.pem

Add a local user with limited access (like no console) that vsfpd will use to run virtual users:

useradd --home /home/vsftpd --gid nogroup -m --shell /bin/false vsftpd

Create directory structures for the virtual users:

mkdir -p /space/ftpusers/
chmod a-w /space/ftpusers/
mkdir -p /space/ftpusers/ftp01/rw
chmod a-w /space/ftpusers/ftp01
chown -R vsftpd:nogroup /space/ftpusers/ftp01

Please note that all new virtual users added need its home directory manually created as per above. Also, due to the chroot option and the current limitation on vsftpd, if you want a user to be able to write in its home directory, you need to create an extra folder. Its root home folder has to be -w. This is a workaround that works 🙂

Setup PAM authentication

Create a new file /etc/pam.d/vsftpd.virtual and add the following:

auth required pam_pwdfile.so pwdfile /etc/vsftpd/vsftpd.users
account required pam_permit.so

Now, let’s reorder a bit vsftp files in a directory:

mkdir -p /etc/vsftpd
cd /etc/
mv vsftpd.conf vsftpd
ln -s /etc/vsftpd/vsftpd.conf .

Add new users (password max 8 characters):

htpasswd -c -d -b /etc/vsftpd/vsftpd.users ftp01 ftp01password

Use the flag -c only the first time to create the file. If you re-use it, the file will be overwritten!
Also the -d flag is required because vsftpd is unable to read MD5 hashed password (default if -d is not used). The downside of this is a password limited to 8 characters.
Openssl could be used to produce a MD5 based BSD password with algorithm 1 using # openssl passwd -1 (not tested)

Let’s configure vsftpd

vi /etc/vsftpd.conf

# Main Settings
listen=YES
listen_port=21
connect_from_port_20=NO
ftpd_banner=Welcome to my FTP :-)
use_localtime=YES
force_dot_files=YES

# FTP Passive settings
pasv_enable=YES
pasv_min_port=1100
pasv_max_port=1150
pasv_addr_resolve=YES
pasv_enable=YES
pasv_addr_resolve=YES
pasv_address=<span style="color: #ff0000;"><EXTERNAL IP or DYN DNS></span>

# Virtual user settings
local_enable=YES
chroot_local_user=YES
secure_chroot_dir=/var/run/vsftpd/empty
virtual_use_local_privs=YES
guest_enable=YES
guest_username=vsftpd
pam_service_name=vsftpd.virtual
user_sub_token=$USER
local_root=/space/ftpusers/$USER
hide_ids=YES

# Anonymous settings
anonymous_enable=NO
anon_upload_enable=NO
no_anon_password=NO
anon_other_write_enable=NO
anon_mkdir_write_enable=NO

# Write permissions
write_enable=YES
local_umask=022
async_abor_enable=YES

# SSL
ssl_enable=YES
force_local_data_ssl=YES
force_local_logins_ssl=YES
ssl_tlsv1=YES
ssl_sslv2=NO
ssl_sslv3=NO
require_ssl_reuse=NO
ssl_ciphers=HIGH
rsa_cert_file=/etc/ssl/certs/vsftpd.pem

# Logging
xferlog_enable=YES
log_ftp_protocol=NO
syslog_enable=NO
vsftpd_log_file=/var/log/vsftpd.log

Now, on your router, make sure that the module ip_conntrack_ftp is loaded using lsmod command.
This is required for FTP PASSIVE mode to work.
I’ve realised that this can be called also nf_conntrack_ftp.
A good way to check all the alias associated to that netfilter module is using the following command:

# modinfo nf_conntrack_ftp
filename: /lib/modules/3.3.7/kernel/net/netfilter/nf_conntrack_ftp.ko
alias: nfct-helper-ftp
alias: <span style="color: #ff0000;">ip_conntrack_ftp</span>
description: ftp connection tracking helper
author: Rusty Russell <[email protected]>
license: GPL
depends: nf_conntrack
intree: Y
vermagic: 3.3.7 mod_unload MIPS32_R1 32BIT
parm: ports:array of ushort
parm: loose:bool

Also, make sure to setup a port forwarding like as below:

$IPT -t nat -A PREROUTING -p tcp -i $EXTIF -d $EXTIP --dport 21 -j DNAT --to $FTPIP:21 # FTP connection port
$IPT -t nat -A PREROUTING -d $EXTIP -p tcp -m tcp --dport 1100:1150 -j DNAT --to-destination $FTPI # FTP PASS ports
$IPT -A FORWARD -i $EXTIF -d $FTPI -p tcp --dport 21 -j ACCEPT
$IPT -A FORWARD -i $EXTIF -d $FTPI -p tcp --dport 1100:1150 -j ACCEPT