Compromised Email troubleshooting notes

Here some notes about how to troubleshoot a server that got compromised by a php script.

Check email queue

  • Qmail -> qmHandle
  • Postfix -> pmHandle / postqueue
# qmHandle -s
Total messages: 7357
Messages with local recipients: 0
Messages with remote recipients: 7357
Messages with bounces: 0
Messages in preprocess: 0

Get some email IDs

# qmHandle -l | head
1348989 (16, 16/1348989)
Return-path: #@[]
From: [email protected]
To: [email protected]
Subject: failure notice
Date: 30 Jun 2015 07:42:59 +0100
Size: 5093 bytes
less
42240113 (15, 15/42240113)
Return-path: [email protected]

Check for X-PHP header in the mail message
Look for the UID and script that sent the message

# qmHandle -m1348989 | grep X-PHP
X-PHP-Originating-Script: 48:wp-content.php(1) : eval()'d code

Find the script and UID

# grep 48 /etc/passwd => this was Apache ==> this means that the code was injected via Apache

=> permissions issue??

# locate wp-content.php
/var/www/vhosts/example.com/wp-content.php

Move away the file(s) and chown 000
!! if the file starts with – , you need to user chown — 000 filename

Disable execution php following this how to

Delete all the messages containing that header

# qmHandle -h'X-PHP-Originating-Script: 48:wp-content.php'
Calling system script to terminate qmail...
Stopping : Looking for messages with headers matching X-PHP-Originating-Script: 48:wp-content.php
Message 1345933 slotted for deletion.
Message 42240608 slotted for deletion.
Message 1346796 slotted for deletion.
Message 42240391 slotted for deletion.
Message 42241954 slotted for deletion.
[...]
Deleted 113 messages from queue
Restarting qmail... Starting qmail: [ OK ]
done (hopefully).

Extra notes:

Check the queue:

postqueue -p

See the content of a message:

postcat -q <ID from postqueue output>

Check for “X-PHP-Originating-Script” header, which generally gives you the name of the script that generate the email

If they are sent to a specific domain, you can block some domains in Postfix following this guide