For mod_php, you can just add this to your Apache LogFormat:
%{mod_php_memory_usage}n
Might be best to add it right at the end, so as not to break any log parsing.
Credit: http://tech.superhappykittymeow.com/?p=220
For PHP-FPM (i.e. anyone with NginX or anyone with one of our optimised Magento setups.), put the following into your FPM pool config file, probably here:
– /etc/php-fpm.d/website.conf (RHEL/CentOS)
– /etc/php5-fpm/pools.d/website.conf (Ubuntu)
access.log = /var/log/php-fpm/domain.com-access.log
access.format = "%p %{HTTP_X_FORWARDED_FOR}e - %u %t \"%m %{REQUEST_URI}e\" %s %f %{mili}d %{kilo}M %C%% \"%{HTTP_USER_AGENT}e\""
Notes:
• Permissions matter. Check that the User and/or Group (of THIS fpm pool) can write to /var/log/php-fpm (or /php5-fpm, whatever)
• %{HTTP_X_FORWARDED_FOR}e is there because I was behind a Load Balancer, Varnish and/or other reverse proxy.
• %{REQUEST_URI}e\ is there because this CMS (like most, now), rewrite everything to index.php. I want to know the original request, not just the script name.
• %{kilo}M %C – These are the kickers: Memory usage and CPU PerformanceOptimized usage per request. Ker-pow. Pick your favourite awk/sort one-liner to weed out the heavy hitters.
You can log pretty much anything: any arbitrary header, (like REQUEST_URI), and you should find all the options in the comments under the default “www.conf” packaged config file.
Given that this starts with the PID, it should also help track down any segfaults you see in /var/log/messages / kern.log.
Oh, and while you’re at it, PHP-FPM has a slow log you can enable.
Oh, and don’t forget your old friend logrotate.
Counting the average PHP-FPM process memory usage
Looking at the ‘ps’ output isn’t always accurate because of shared memory. Here’s a one-liner to count up my FPM pool, where “www” is in name of the FPM pool:
for pid in $(ps aux | grep fpm | grep "pool www" | awk '{print $2}'); do pmap -d $pid | tail -1 ; done | sed 's/K//' | awk '{sum+=$4} END {print sum/NR/1024}’
Thanks to Dan Farmer for his original Apache memory script, which made me think to do this.
Divide and conquer
Do make sure that multiple websites on the same server are using their own FPM pools, that way it should be much easier to see what’s what.
But you can also separate by URL. I’ve done this with the M-word, but the theory should stand for any CMS where the tasks performed under the admin section will be more intensive than normal front-end user traffic.
The following should work with WordPress, assuming the path is /wp-admin . Magento users can (and should) change their admin path from the default, so watch out for that (usually configured in local.xml)
Or anything really – maybe that “importvideo” script is especially memory-intensive and you don’t want to allow it to bloat up all your processes.
1. Set up a separate PHP-FPM pool.
– give it a different name, different log file names, and listen on a different socket/port.
– Perhaps with a much higher memory_limit
– Perhaps with many fewer pm.max_children (ask customer how many humans actually use the “backend”). You might only need 5 or so.
– Perhaps it needs a longer max_execution_time …. you get the idea.
2. Send your “admin” traffic there.
Here’s how you might go about it. These are excerpts and ideas, rather than solid copy-paste config.
Nginx: something like this:
upstream backend {
server unix:/var/run/php5-domain.com.sock;
}
upstream backend-admin {
server unix:/var/run/php5-domain.com-admin.sock;
}
map "URL:$request_uri." $fcgi_pass {
default backend;
~URL:.*admin.* backend-admin;
}
… then in your ‘server{}’ bit, use the variable for fastcgi_pass:
fastcgi_pass $fcgi_pass;
Apache: something like this…
# Normal backend alias, corresponds with FastCGIExgternalServer Alias /php.fcgi /dev/shm/domain-php.fcgi <Location ~ admin> # Override Action for “admin” URLs Action application/x-httpd-php /domain-admin.fcgi </Location> Alias /domain-admin.fcgi /dev/shm/domain-admin-php.fcgi
3. You can probably now LOWER the memory_limit for your “main” pool
– if the rest of the website doesn’t use much memory. Now bask in the memory you just saved for the whole application server.
4. Bonus: NewRelic app separation
Don’t let all that heavy Admin work interfere with your nice / renice appdex statistics – we know (and expect) your backend dashboard to be slower.
Put this in the FPM pool config:
php_value[newrelic.appname] = "www.domain.com Admin"
Credits: https://willparsons.tech/