Tag Archives: advance

How to chown and setuid

Something to watch out for.

If you change the owner of a file that has setuid permission, the setuid bit gets removed:

# ls -l /usr/bin/gpasswd
-rwsr-xr-x. 1 root root 78144 Mar 19 2013 /usr/bin/gpasswd 
# chown nobody /usr/bin/gpasswd 
# ls -l /usr/bin/gpasswd 
-rwxr-xr-x. 1 nobody root 78144 Mar 19 2013 /usr/bin/gpasswd

# ls -l /usr/bin/gpasswd

-rwsr-xr-x. 1 root root 78144 Mar 19 2013 /usr/bin/gpasswd

# chown nobody /usr/bin/gpasswd

# ls -l /usr/bin/gpasswd

-rwxr-xr-x. 1 nobody root 78144 Mar 19 2013 /usr/bin/gpasswd

Therefore if you reset the owner of such a file (for instance, after an accidental recursive chown), then you need to reset the permissions afterwards.
If you reset both the owner AND the permissions, it has to be done in the correct order – ownership first, then permissions.

So what will this do?

# rpm --setperms --setugids

1	# rpm --setperms --setugids

It turns out (on RHEL5 at any rate) that it changes the permissions first. So the binaries that ought to have the setuid bit were left without it.

The moral of this story is that you should do the work in two passes:

# rpm --setugids

# rpm --setperms

# rpm --setugids

# rpm --setperms

Space utilised one liners

# Current  folder space
du -sh <path>

# 10 biggest folders
du -m <path> | sort -nr | head -n 10

# Check high directories usage.
du -hcx --max-depth=5 | grep [0-9]G | sort -nr

# Exclude a path from the final calculation
cd /path
du -sh --exclude=./relative/path/to/uploads

# Check APPARENT size
du -h --apparent-size /path/file



# Check how much space is "wasted":
lsof | grep deleted | sed 's/^.* \(REG.*deleted.*$\)/\1/' | awk '{print $5, $3}' | sort | uniq | awk '{sum += $2 } END { print sum }'

# >> *if* the number is like "1.5e+10", you might need to use this to see that converted in MB or GB
lsof | grep deleted | sed 's/^.* \(REG.*deleted.*$\)/\1/' | awk '{print $5, $3}' | sort | uniq | awk '{sum += $2 } END { print sum " bytes - " sum/1024**2 " MB - " sum/1024**3 " G" }'

# Check the biggest files:
lsof | grep deleted | sed 's/^.* \(REG.*deleted.*$\)/\1/' | awk '{print $5, $3}' | sort | uniq | awk '{print $2, $1}' | sort -nr

>> than you can grep the file name from the output of "lsof | grep deleted" and check for the PID that holds that file (second column)
>> and issue the following command:
kill -HUP <PID>
>> And check again. This should release the used file.

# Current folder space

du -sh <path>

# 10 biggest folders

du -m <path> | sort -nr | head -n 10

# Check high directories usage.

du -hcx --max-depth=5 | grep [0-9]G | sort -nr

# Exclude a path from the final calculation

cd /path

du -sh --exclude=./relative/path/to/uploads

# Check APPARENT size

du -h --apparent-size /path/file

# Check how much space is "wasted":

lsof | grep deleted | sed 's/^.* $REG.*deleted.*$$/\1/' | awk '{print $5, $3}' | sort | uniq | awk '{sum += $2 } END { print sum }'

# >> *if* the number is like "1.5e+10", you might need to use this to see that converted in MB or GB

lsof | grep deleted | sed 's/^.* $REG.*deleted.*$$/\1/' | awk '{print $5, $3}' | sort | uniq | awk '{sum += $2 } END { print sum " bytes - " sum/1024**2 " MB - " sum/1024**3 " G" }'

# Check the biggest files:

lsof | grep deleted | sed 's/^.* $REG.*deleted.*$$/\1/' | awk '{print $5, $3}' | sort | uniq | awk '{print $2, $1}' | sort -nr

>> than you can grep the file name from the output of "lsof | grep deleted" and check for the PID that holds that file (second column)

>> and issue the following command:

kill -HUP <PID>

>> And check again. This should release the used file.

Apparent size is the number of bytes your applications think are in the file. It’s the amount of data that would be transferred over the network (not counting protocol headers) if you decided to send the file over FTP or HTTP. It’s also the result of cat theFile | wc -c, and the amount of address space that the file would take up if you loaded the whole thing using mmap.

Disk usage is the amount of space that can’t be used for something else because your file is occupying that space.

In most cases, the apparent size is smaller than the disk usage because the disk usage counts the full size of the last (partial) block of the file, and apparent size only counts the data that’s in that last block. However, apparent size is larger when you have a sparse file (sparse files are created when you seek somewhere past the end of the file, and then write something there — the OS doesn’t bother to create lots of blocks filled with zeros — it only creates a block for the part of the file you decided to write to).

Source (clarification): http://stackoverflow.com/questions/5694741/why-is-the-output-of-du-often-so-different-from-du-b

MySQL – How much memory for InnoDB buffer?

Use this to find out how much storage each engine uses:

mysql -e "select engine, SUM(DATA_LENGTH)/1024 as data_size, SUM(INDEX_LENGTH)/1024 as index_size, ((sum(DATA_LENGTH)+sum(INDEX_LENGTH))/1024) as total_size from information_schema.tables group by engine;"
+--------------------+-----------+------------+------------+
| engine | data_size | index_size | total_size |
+--------------------+-----------+------------+------------+
| CSV | 0.0000 | 0.0000 | 0.0000 |
| InnoDB | 480.0000 | 0.0000 | 480.0000 |
| MEMORY | 0.0000 | 0.0000 | 0.0000 |
| MyISAM | 0.0000 | 1.0000 | 1220.5859 |
| PERFORMANCE_SCHEMA | 0.0000 | 0.0000 | 0.0000 |
+--------------------+-----------+------------+------------+

mysql -e "select engine, SUM(DATA_LENGTH)/1024 as data_size, SUM(INDEX_LENGTH)/1024 as index_size, ((sum(DATA_LENGTH)+sum(INDEX_LENGTH))/1024) as total_size from information_schema.tables group by engine;"

+--------------------+-----------+------------+------------+

+--------------------+-----------+------------+------------+

| CSV | 0.0000 | 0.0000 | 0.0000 |

| InnoDB | 480.0000 | 0.0000 | 480.0000 |

| MEMORY | 0.0000 | 0.0000 | 0.0000 |

| MyISAM | 0.0000 | 1.0000 | 1220.5859 |

| PERFORMANCE_SCHEMA | 0.0000 | 0.0000 | 0.0000 |

+--------------------+-----------+------------+------------+

From the data above, it may seem like a good idea to keep innodb_buffer_pool low, because we have so much MyISAM data.
BUT if I get one query a minute against that MyISAM data, and 500 queries per second against the considerably smaller InnoDB data-set, that doesn’t make sense anymore.
Then we’d want all of the InnoDB data in memory, hence a buffer pool of 512M+

It’s down to data usage and where the hotspots are!

Percona Toolkit to ‘get the hotspots’

wget http://www.percona.com/downloads/percona-toolkit/2.2.12/tarball/percona-toolkit-2.2.12.tar.gz
tar -zxvf percona-toolkit-2.2.12.tar.gz -C /opt/ 
cd /opt/percona-toolkit-2.2.12/ 
perl Makefile.PL 
make

wget http://www.percona.com/downloads/percona-toolkit/2.2.12/tarball/percona-toolkit-2.2.12.tar.gz

tar -zxvf percona-toolkit-2.2.12.tar.gz -C /opt/

cd /opt/percona-toolkit-2.2.12/

perl Makefile.PL

make

Get the hotspots:
you can set NUMPACKETS or keep the tcpdump running for 20 minutes. JUST keep an eye on the file size!

NUMPACKETS=1000
tcpdump -s 65535 -x -nn -q -tttt -i any -c $NUMPACKETS port 3306 > /tmp/mysql.tcp.txt 

bin/pt-query-digest --type tcpdump /tmp/mysql.tcp.txt

NUMPACKETS=1000

tcpdump -s 65535 -x -nn -q -tttt -i any -c $NUMPACKETS port 3306 > /tmp/mysql.tcp.txt

bin/pt-query-digest --type tcpdump /tmp/mysql.tcp.txt

FYI: https://bugs.launchpad.net/percona-toolkit/+bug/1402776 official bug for that database being wrong thing

My notepad

Chris' IT notes…

Tag Archives: advance

How to chown and setuid

Space utilised one liners

MySQL – How much memory for InnoDB buffer?

Percona Toolkit to ‘get the hotspots’