Currently we are being inundated with scans and exploits looking for new machines to take over, most of these are using insecure phpmyadmin installs.

If you are unsure if your box has been hacked then the things to look for are similar to this…

Perl has a high CPU load in ‘top’

top - 20:43:09 up 174 days, 20:47,  1 user,  load average: 1.00, 1.00, 1.00
Tasks: 279 total,   2 running, 277 sleeping,   0 stopped,   0 zombie
Cpu(s):  7.1%us,  0.3%sy,  0.0%ni, 92.4%id,  0.1%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:    921948k total,   897940k used,    24008k free,   130788k buffers
Swap:   131064k total,    79704k used,    51360k free,   210824k cached

PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
2239 www-data  20   0  4896 2872 1224 R 99.9  0.3   3853:33 perl   <<---
21482 root      20   0  2344 1112  744 R  3.8  0.1   0:00.04 top
28751 root      20   0 43108  13m 1604 S  1.9  1.4   6:05.22 apache2
1 root      20   0  1824  404  364 S  0.0  0.0   1:43.69 init

Odd things running as the apache user

~# ps aux |grep www-data
www-data  9442  0.0  2.1  23060  8596 ?        S    Aug18   0:00 /usr/sbin/apache2 -k start -DSSL
www-data  9448  0.0  0.0      0     0 ?        Z    Aug18   0:00 [sh] <defunct> <- sure sign here
www-data  9458 99.2  0.7   4916  2888 ?        R    Aug18 1297:10 /usr/sbin/apache/loggs
www-data  9701  0.0  0.4   4916  1892 ?        S    Aug18   0:00 /usr/sbin/apache/loggs  <- this doesnt exist
www-data  9704  0.0  0.7   5860  3020 ?        R    Aug18   0:17 inetd   <- this should never run as apache user
www-data 31545  0.0  1.1  21164  4628 ?        S    08:20   0:00 /usr/sbin/apache2 -k start -DSSL
www-data 31546  0.0  1.1  21164  4620 ?        S    08:20   0:00 /usr/sbin/apache2 -k start -DSSL
root     31672  0.0  0.2   2732   800 pts/1    D+   08:32   0:00 grep www-data

The best recovery is a reinstall or revert to a non-hacked backup. You can do this in your control panel easily yourself. Once you have restored to a non-hacked backup you need to secure it. Shutdown all daemons like apache, postfix, mysql etc. Change any user passwords (especially root!)

Find insecure instances of web software such as phpmadmin, or other things.

This will find most cases of phpmyadmin

find / -iname "*phpmyadmin*"

Then you can move that somewhere outside a documentroot or use htaccess to deny anyone usage.

Now you probably want to mount the hacked/exploited image . First you attach it as /dev/xvda3 in your rimuhosting control panel, this will reboot your vps. When it comes back up re-shutdown all daemons until you have finished.

Run:

mkdir /oldfs
mount /dev/xvda3 /oldfs

Now you can view all your old VPS files. You can copy over email from /oldfs/var/mail to /var/mail or mysql from /oldfs/var/lib/mysql to /var/lib/mysql etc. Be very careful about any files you copy over, they may be exploited.

Once you have done that, and gone over anything else with a fine tooth comb, and updated any other packages, you can restart your daemons again. Keep an eye on logs regularly for more exploits and hacks.

Image credit flickr:Clive Darr