Category Archives: Linux

Rackspace Cloud – Remove old System IDs via command line

Rough script/instructions 🙂

>> set your variables:
TOKEN=""
REGION="lon"
DDI=""  < this is the account number

>> Generate a list of backup agents
curl -sH  "X-Auth-Token: $TOKEN" -H "Content-type: application/json" -X GET https://$REGION.backup.api.rackspacecloud.com/v1.0/$DDI/user/agents | python -m json.tool | egrep "MachineName|MachineAgentId" | awk -F":" '{print $2}' | sed 's/ //g' | sed '{N;s/\n//}' > list.txt

>> Manually remove WANTED backup agents (leave only the ones you want to remove):
vim list.txt 

>> Generate remove list
awk -F, '{print $1}' list.txt > remove.txt


>> generate the exec file to review
for AGENTID in `cat remove.txt`; do echo curl -sH \"X-Auth-Token: $TOKEN\" -H \"Content-type: application/json\" -X POST https://$REGION.backup.api.rackspacecloud.com/v1.0/$DDI/agent/delete -d \'{\"MachineAgentId\": $AGENTID}\' ; done >> exec_me

>> exec the API calls
/bin/bash exec_me

 

Lsync monitoring on Rackspace Cloud 

mkdir -p /usr/lib/rackspace-monitoring-agent/plugins/
cd /usr/lib/rackspace-monitoring-agent/plugins/
wget https://raw.githubusercontent.com/racker/rackspace-monitoring-agent-plugins-contrib/master/lsyncd-status.sh
chmod 755 lsyncd-status.sh

You can test the above script by calling it directly to see if it is working and reporting stats:

/usr/lib/rackspace-monitoring-agent/plugins/lsyncd-status.sh

 

Now, we need to create the alert itself.
[To get the token, you can use this]

curl -i -X POST \
-H 'X-Auth-Token: [AUTH_TOKEN]' \
-H 'Content-Type: application/json; charset=UTF-8' \
-H 'Accept: application/json' \
--data-binary \
'{"label": "Lsyncd", "type": "agent.plugin", "details": {"file": "lsyncd-status.sh","args": ["arg1","arg2"]}}' \
'https://monitoring.api.rackspacecloud.com/v1.0/[ACCOUNT_ID]/entities/[ENTITY_ID]/checks'

NOTE: ENTITY_ID is the Monitoring ID, NOT the server ID!!

Once the alert has been created, you can add the alarm manually via the Control Panel:

if (metric['lsyncd_status'] != 'running') {
return new AlarmStatus(CRITICAL, 'Lsyncd Service is NOT running.');
}
if (metric['lsyncd_status'] == 'running' && metric['percent_used_watches'] >= 80) {
return new AlarmStatus(WARNING, 'Lsyncd is running but the number of directories has reached 80% of notify watches.');
}
if (metric['lsyncd_status'] == 'running' && metric['percent_used_watches'] >= 95) {
return new AlarmStatus(CRITICAL, 'Lsyncd is running but the number of directories has reached 95% of notify watches.');
}
return new AlarmStatus(OK, 'Lsyncd Service is running.');

Make sure to test and save the alert.

Rackspace Cloud Driveclient not working

First of all, checks the logs: /var/log/driveclient.log

You might find 403 errors and lines that are showing that the agent can’t connect properly.

In this case, the first step is trying to re-register the backup agent:
3) Maybe the customer has changed the API key so try re-register the backup agent:

# /usr/local/bin/driveclient --configure
WARNING: Agent already configured. Overwrite? [Y/n]: Y
Username: My_Username
Password: My_APIKey

Desired Output:

Registration successful!
Bootstrap created at: /etc/driveclient/bootstrap.json

In case you get something like “ERROR: Registration failed: Could not authenticate user. Identity returned 401“, this means that you probably need to force a bit the registration, using the following command:

# driveclient -u USER_NAME -k API_KEY -t LON -l raxcloudserver -a lon.backup.api.rackspacecloud.com -c

 

Force reset/repush network configuration Rackspace Cloud server

Run the following command on the Cloud server (this works only on Linux servers):

UUID=`uuidgen`; xenstore-write data/host/$UUID '{"name":"resetnetwork","value":""}'; sleep 10; xenstore-read data/guest/$UUID; unset UUID

If completed successfully it will return something like this:

{"message": "", "returncode": "0"}

 

Apache ProxyPass for WordPress master-slave setup

Simple way

Ensure certain traffic goes to a certain server (master), you can use this:

<LocationMatch "^/wordpress/wp-admin/?.*>
ProxyPreserveHost On
ProxyPass http://ip.of.master.server/
</LocationMatch>

 

For a better setup with Variables, just follow the… following steps 🙂

Step One: Configure Environment

We need to setup some environment variables to get this to work correctly.
Add the following to your environment on the slave server(s):

RHEL/CentOS: /etc/sysconfig/httpdi

OPTIONS="-DSLAVE"
export MASTER_SERVER="SERVERIP HERE"

Ubuntu: /etc/apache2/envvars

export APACHE_ARGUMENTS="-DSLAVE"
export MASTER_SERVER="SERVERIP HERE"

Step Two: Configure your VirtualHost

In your VirtualHost configuration do something like the following.

<IfDefine SLAVE>
RewriteEngine On
ProxyPreserveHost On
ProxyPass /wp-admin/http://${MASTER_SERVER}/wp-admin/
ProxyPassReverse /wp-admin/http://${MASTER_SERVER}/wp-admin/

RewriteCond %{REQUEST_METHOD} =POST
RewriteRule . http://${MASTER_SERVER}%{REQUEST_URI} [P]
</IfDefine>

 

Block UserAgent libwww-perl and test

If you want to apply this globally, edit /etc/httpd/conf/httpd.conf (Centos). Otherwise, just modify the related vhost config file, adding the following:

ServerSignature Off
SetEnvIfNoCase User-Agent "^libwww-perl" bad_bot
<Location />
Order allow,deny
Allow from all
Deny from env=bad_bot
</Location>

How to test:

Check if the site responds:

$ curl -I http://www.mywebsite.com
HTTP/1.1 200 OK
Date: Thu, 13 Nov 2014 17:12:16 GMT
Server: Apache
Last-Modified: Thu, 13 Nov 2014 16:00:35 GMT
ETag: "1fa5dc-9dd5-507bf9cd66ec0"
Accept-Ranges: bytes
Content-Length: 40405
Vary: Accept-Encoding,Cookie
Content-Type: text/html; charset=UTF-8

 

Check if the agent libwww-perl is allowed or forbidden:

$ curl -A "libwww-perl" -I http://www.mywebsite.com
HTTP/1.1 403 Forbidden
Date: Thu, 13 Nov 2014 17:12:51 GMT
Server: Apache
Vary: Accept-Encoding
Content-Type: text/html; charset=iso-8859-1

 

Check if using Mozilla as agent works or not:

$ curl -A "Mozilla/5.0 (iPhone; U; CPU PerformanceOptimized iPhone OS 4_3_3 like Mac OS X; en-us) AppleWebKit/533.17.9 (KHTML, like Gecko) Version/5.0.2 Mobile/8J2 Safari/6533.18.5" -I http://www.mywebsite.com
HTTP/1.1 200 OK
Date: Thu, 13 Nov 2014 17:13:27 GMT
Server: Apache
Last-Modified: Thu, 13 Nov 2014 16:00:35 GMT
ETag: "1fa5dc-9dd5-507bf9cd66ec0"
Accept-Ranges: bytes
Content-Length: 40405
Vary: Accept-Encoding,Cookie
Content-Type: text/html; charset=UTF-8

 

Source IP in Apache logs

If your server is under a load balancer, you might see HTTP requests coming from the Load Balancer’s IP instead of the actual visitor. Generally, load balancers are “recording” the original source IP in the X-Forwarded-For header. This means that this is the header that we need to log in our Apache logs to get the information that we want.

Here how to make this happen:

Default rule:

LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined

Modified rule that includes the X-Forwarded-For definition:

LogFormat "%{X-Forwarded-For}i %h %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"" combined


==============================================

>> Best way:


On httpd.conf:
=====================
SetEnvIfNoCase X-Forwarded-For "." from_proxy=1

LogFormat "%{CF-Connecting-IP}i %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" forwarded
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
=======================

On vhost.conf
============================
CustomLog /var/log/httpd/vhost-access.log forwarded env=from_proxy
CustomLog /var/log/httpd/vhost-access.log combined env=!from_proxy
============================

 

Source: https://community.rackspace.com/products/f/25/t/211

Apache Rewrite rules

Rewrite rules examples:

This can be added in vhost configuration OR in .htaccess file

How to rewrite all web request on my site without www to www.domain.com

RewriteCond %{HTTP_HOST} !^www\.example\.com$ [NC]
RewriteRule (.*) http://www.example.com$1 [R,L]

 

How to redirect all web requests on port 80 (or HTTP) to port 443 (HTTPS)

RewriteCond %{SERVER_PORT} !^443$
RewriteRule ^/(.*) https://%{SERVER_NAME}/$1 [L,R]

 

How to disable TRACE and TRACK methods on Apache

RewriteCond %{REQUEST_METHOD} ^(TRACE|TRACK)
RewriteRule .* - [F]

 

How to exclude mod_status from being rewritten by existing rules (placed before the problem rule)

RewriteCond %{REQUEST_URI} !=/server-status

 

How do I redirect all web requests on www.mysite.net/web to www.mysite.net/sect1/web

RewriteCond %{http_host} ^[www\.]*example\.com
RewriteRule ^web(/)?$ /sect1/web [R=301,L]

 

Rewrite all non-www to www

RewriteEngine On
RewriteCond %{HTTP_HOST} !^www\. [NC]
RewriteRule ^(.*)$ http://www.%{HTTP_HOST}%{REQUEST_URI} [R=301,L]

 

Force all URLs to be lowercase

RewriteEngine On
RewriteMap lc int:tolower
RewriteCond %{REQUEST_URI} [A-Z]
RewriteRule (.*) ${lc:$1} [R=301,L]