Tag Archives: nginx

Set X-Forward in Nginx logs

http://wp.sgrosshome.com/2014/03/03/how-to-configure-nginx-logs-to-show-real-ip-when-behind-a-reverse-proxy/

log_format LB_log '$remote_addr forwarded for $http_x_realip - $remote_user [$time_local] '
'"$request" $status $body_bytes_sent '
'"$http_referer" "$http_user_agent"';
access_log /path/to/you/log/directory/your-log-file-name.log LB_log;

 

After that, Nginx needs to be restarted/reloaded, of course šŸ˜‰

Which PHP script is eating my memory?

For mod_php, you can just add this to your Apache LogFormat:

%{mod_php_memory_usage}n

Might be best to add it right at the end, so as not to break any log parsing.
Credit: http://tech.superhappykittymeow.com/?p=220

For PHP-FPM (i.e. anyone with NginX or anyone with one of our optimised Magento setups.), put the following into your FPM pool config file, probably here:
– /etc/php-fpm.d/website.conf (RHEL/CentOS)
– /etc/php5-fpm/pools.d/website.conf (Ubuntu)

access.log = /var/log/php-fpm/domain.com-access.log
access.format = "%p %{HTTP_X_FORWARDED_FOR}e - %u %t \"%m %{REQUEST_URI}e\" %s %f %{mili}d %{kilo}M %C%% \"%{HTTP_USER_AGENT}e\""

 
Notes:
ā€¢ Permissions matter. Check that the User and/or Group (of THIS fpm pool) can write to /var/log/php-fpm (or /php5-fpm, whatever)
ā€¢ %{HTTP_X_FORWARDED_FOR}e is there because I was behind a Load Balancer, Varnish and/or other reverse proxy.
ā€¢ %{REQUEST_URI}e\ is there because this CMS (like most, now), rewrite everything to index.php. I want to know the original request, not just the script name.
ā€¢ %{kilo}M %C – These are the kickers: Memory usage and CPU PerformanceOptimized usage per request. Ker-pow. Pick your favourite awk/sort one-liner to weed out the heavy hitters.

You can log pretty much anything: any arbitrary header, (like REQUEST_URI), and you should find all the options in the comments under the default “www.conf” packaged config file.
Given that this starts with the PID, it should also help track down any segfaults you see in /var/log/messages / kern.log.
Oh, and while youā€™re at it, PHP-FPM has a slow log you can enable.
Oh, and donā€™t forget your old friend logrotate.

Counting the average PHP-FPM process memory usage

Looking at the ā€˜psā€™ output isnā€™t always accurate because of shared memory. Hereā€™s a one-liner to count up my FPM pool, where ā€œwwwā€ is in name of the FPM pool:

for pid in $(ps aux | grep fpm | grep "pool www" | awk '{print $2}'); do pmap -d $pid | tail -1 ; done | sed 's/K//' | awk '{sum+=$4} END {print sum/NR/1024}ā€™

Thanks to Dan Farmer for his original Apache memory script, which made me think to do this.

Divide and conquer

Do make sure that multiple websites on the same server are using their own FPM pools, that way it should be much easier to see whatā€™s what.

But you can also separate by URL. Iā€™ve done this with the M-word, but the theory should stand for any CMS where the tasks performed under the admin section will be more intensive than normal front-end user traffic.
The following should work with WordPress, assuming the path is /wp-admin . Magento users can (and should) change their admin path from the default, so watch out for that (usually configured in local.xml)

Or anything really – maybe that ā€œimportvideoā€ script is especially memory-intensive and you donā€™t want to allow it to bloat up all your processes.

1. Set up a separate PHP-FPM pool.
– give it a different name, different log file names, and listen on a different socket/port.
– Perhaps with a much higher memory_limit
– Perhaps with many fewer pm.max_children (ask customer how many humans actually use the “backendā€). You might only need 5 or so.
– Perhaps it needs a longer max_execution_time ā€¦. you get the idea.

2. Send your ā€œadminā€ traffic there.
Hereā€™s how you might go about it. These are excerpts and ideas, rather than solid copy-paste config.

Nginx: something like this:

upstream backend {
server unix:/var/run/php5-domain.com.sock;
}
upstream backend-admin {
server unix:/var/run/php5-domain.com-admin.sock;
}

map "URL:$request_uri." $fcgi_pass {
default backend;
~URL:.*admin.* backend-admin;
}

… then in your ‘server{}’ bit, use the variable for fastcgi_pass:

fastcgi_pass $fcgi_pass;

Apache: something like this…

# Normal backend alias, corresponds with FastCGIExgternalServer 
Alias /php.fcgi /dev/shm/domain-php.fcgi
<Location ~ admin>
# Override Action for ā€œadminā€ URLs
Action application/x-httpd-php /domain-admin.fcgi
</Location>
Alias /domain-admin.fcgi /dev/shm/domain-admin-php.fcgi

 

3. You can probably now LOWER the memory_limit for your ā€œmainā€ pool
– if the rest of the website doesnā€™t use much memory. Now bask in the memory you just saved for the whole application server.

4. Bonus: NewRelic app separation
Donā€™t let all that heavy Admin work interfere with your nice / renice appdex statistics – we know (and expect) your backend dashboard to be slower.
Put this in the FPM pool config:

php_value[newrelic.appname] = "www.domain.com Admin"

 

Credits:Ā https://willparsons.tech/