Tag Archives: command line

TOP – memory explanation

(just few notes – to avoid to forget)

  • VIRT: not really relevant nowadays. It’s the memory that the process could use. But the OS loads only what needed, so rarely really used. On 32bit OS, it could be the only time when you need to keep an eye as the OS can allocate up to 2-3GB only.
  • RES: Resident Set Size memory – this is the actual memory in RAM. On low used machines, it might still show high usage even if not utilised as the process to free-up the memory costs more than leaving it. In fact, Linux OS tends to use as much memory available (“unused memory is wasted memory“).
  • SH: this is the shared memory which generally contains libraries etc

LVM – How to

Intro

LVM is a very powerful technology, and can really help the Sysadmin’s life.
However, this is something that we generally setup at the beginning (most of the time now it’s automatically setup during the installation process), and it’s well know… when we stop using something, we tend to forget how to use it.

This is why I’m writing this how to, mostly to keep track of the major features and commands, in case I will need them again in the future ūüėČ

Before proceeding, please digest the following journey of this poor physical device that gets abstracted up to usable pieces.

 

 

Prepare partions

First of all, we need to find which device(s) we want to setup for LVM

fdisk -l

We can see 3 md devices, probably RAID devices. These are the ones that we are going to use for our LVM exercise.

Now, let’s create an LVM partition.

fdisk <device> => n , p , 1 , (enter) , (enter) , t , 8e , w

Do the same for all the devices that you want to use for LVM. In my example, I’ve done this for /dev/md1, /dev/md2 and /dev/md3.

Shortcut (risky but quicker) ūüôā

All seems now good to go: we have Linux LVM partitions!

Time to start to configure LVM

Configure LVM

First of all, we need to make these Linux LVM partition able to be part of a group (vg). I always find tricky to remember the logic behind. Let’s imagine that the device itself now is just labelled “Linux LVM” but we need to¬†initiate¬†it in somehow.

pvcreate <dev>

Now these guys are ready to be part of a group. In this case a Virtual Group (vg).
Let’s check that it’s actually true:
pvs

Time to create a group with these devices (this could be done also with just a single device):

vgcreate <lvmgroupname> <dev> <dev> …

Now, let’s check again with pvs and vgs

Now pvs shows the VG group no longer empty but with mylvmvg. And vgs tells us that the VG is about 14GB in size, fully free with no LV in it.

Good! Now, let’s make some LVs (logical volumes). These will be the new¬†“partitions/disks” that we will be actually able to format, mount and use! ūüôā

lvcreate -n <name> -L xGB <vg_group_name>

Some checks to verify:

A new LV appears in vgs and lvs shows the 2GB volume that we have created.

Let’s create another one, but this time, using the full remaining space (using -l 100%VG¬†option instead of -L xGB)

Magic!

Now, we have two devices, both ‘a’ -> active and ready to be formatted:
mkfs.ext4 <device>

I’ve done this for /dev/mylvmvg/part1 and /dev/mylvmvg/part2.

Let’s create the mount points and mount them:

As you can see, the devices are appearing now as /dev/mapper/mylvmvg-partX. You can use either /dev/mylvmvg/partX or /dev/mapper/mylvmvg-partX. Theoretically, the mapper one is recommended (my bad!).

Now the 2 devices are ready to be used as a typical disk/partition formatted with ext4 filesystem.


Resize Logical Volume

Now, imagine that¬†part1 is too small, and you need more space. And luckily, your¬†part2 volume has plenty. Is there any way to “steal” some space from¬†part2¬†and give it to¬†part1?
Ooohh yesss! ūüôā

How?

  1. shrink part2 logical volume AND its filesystem
  2. expand part1 logical volume AND its filesystem

Here the comments inline:

 

Move logical volume onto a new RAID array

Now, let’s imagine that one of the 3 initial¬†md devices are having problems, or simply we want to move on a faster/bigger raid array.
The magic of LVM is that we can actually do this with NO DOWNTIME!

How?

In this example we assume that a new /dev/md10 device is attached to our server and we want to remove /dev/md2 device.

  1. We need to take the new device and go through all the previous steps:
    1. fdisk
    2. pvcreate
  2. After that, we need to add this initialised device in the existing volume group (vg)
  3. Move whatever is stored on the physical device
  4. Shrink the volume group
  5. Remove the device

Now where the new bits are starting:
pvmove, vgreduce, pvremove

 

In this example we have left LVM to decide where to put the data that was stored on /dev/md2 device.
Just for reference, we could have specified the destination physical device (e.g. if we were thinking to remove more devices and make sure that the data was ending up on the new RAID and not sprat across the other disks):

Or, in case we just wanted to move a specific logical volume, let’s say¬†part1

 

…happy LVM’ing! ūüėČ

Varnish – basic notes

 

Sophos antivirus notes

Generic checks

Scan

Example for multiple folders with final report:

(suggested to run in a screen session)

  1. Create a temporary folder:
  2. list all directories that you want to scan (full path) into a file called list_folder.txt within the temp folder;
  3. Run the following:
  4. Check report.txt 

 

Plesk notes

 

 

Atop – notes

 

ftp/sftp – vsftpd

 

One liners to automatic creation of username and passwords

Automatic creation of users/passwords (FTP)

Manually create list.txt with user:doc_root
e.g.:

Get commands to create FTP users

 

Get commands to set FTP permissions (if doc_root exists already)

 

Generate and Assign random passwords to the users.

 


Create a list of vhosts’ paths: vhosts.txt

Example with only .com domains:
/var/www/domain1.com
/var/www/domain2.com
/var/www/domain3.com

Use a regex for sed to extract the vhost name, removing dots (example based on the example above)
This will return a list of PATH and VHOSTNAME. We will use VHOSTNAME as USER for that path

 

Print out the commands to run to add FTP users (no SSH)
Once checked the output, run these lines

(for sftp only):

 

This will print out commands to run to assign user:apache_group to the vhosts’ paths

(for sftp only):

 

Set g+s on vhosts to preserve directory owner
[TO CHECK]

[THIS EXECUTE]

 

Create list of random passwords using pwgen

 

Create list of random passwords using openssl

 

Apply these passwords automatically

 

Print output for reference

cat: create/write file

 

Create file without replacing variables:

If you check /path/file you will see exactly the content above.
Create file REPLACING the variables while creating:

In this example, the variable ${0} will be replaced during the creation of the file, hence the content will display your username.