I have a 512 MB VPS and hosting 2 WordPress websites on this CentOS server. I have installed apache + mysql + PHP + fast cgi on this server. Everything was working great from last 6 months. I have 500 users per day on both websites combined. So no huge load.
But from last night (it's been 12 hours) I have 100% CPU usage and high memory usage. The websites as well as server is not accessible. I tried to reboot server thought it could just be some error but nothing worked.
It's the output of top, but I could not understand what is the problem and how could I fix it. Seems like there are tons of php-cgi & httpd processed.
top - 09:11:43 up 2 min, 1 user, load average: 26.91, 10.07, 3.67
Tasks: 137 total, 28 running, 109 sleeping, 0 stopped, 0 zombie
Cpu(s): 36.4%us, 57.5%sy, 0.0%ni, 4.1%id, 1.4%wa, 0.0%hi, 0.1%si, 0.6%st
Mem: 511036k total, 505416k used, 5620k free, 3280k buffers
Swap: 0k total, 0k used, 0k free, 12240k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
804 root 20 0 36160 540 4 S 11.7 0.1 0:03.69 rsyslogd
1548 robert 20 0 51656 25m 812 R 5.8 5.1 0:00.76 php-cgi
1549 robert 20 0 50412 23m 528 R 5.8 4.8 0:00.70 php-cgi
1552 robert 20 0 50704 24m 764 R 5.8 4.9 0:00.64 php-cgi
1568 robert 20 0 44940 18m 760 R 5.8 3.7 0:00.42 php-cgi
1573 robert 20 0 38680 12m 792 R 5.8 2.6 0:00.32 php-cgi
1584 robert 20 0 31964 6300 704 R 5.8 1.2 0:00.19 php-cgi
1553 robert 20 0 49544 23m 1184 R 4.4 4.7 0:00.61 php-cgi
1554 robert 20 0 49544 23m 972 R 4.4 4.7 0:00.60 php-cgi
1557 robert 20 0 46288 19m 816 R 4.4 4.0 0:00.57 php-cgi
1558 robert 20 0 46288 19m 836 R 4.4 4.0 0:00.52 php-cgi
1563 robert 20 0 45452 19m 1104 R 4.4 3.9 0:00.49 php-cgi
1564 robert 20 0 45452 19m 1136 R 4.4 3.9 0:00.46 php-cgi
1565 robert 20 0 44948 18m 764 R 4.4 3.7 0:00.43 php-cgi
1569 robert 20 0 35492 9872 768 R 4.4 1.9 0:00.39 php-cgi
1572 robert 20 0 38680 12m 816 R 4.4 2.6 0:00.34 php-cgi
1574 robert 20 0 38376 12m 784 R 4.4 2.5 0:00.30 php-cgi
1576 robert 20 0 38388 12m 800 R 4.4 2.5 0:00.26 php-cgi
1583 robert 20 0 32736 7688 1332 R 4.4 1.5 0:00.20 php-cgi
1585 robert 20 0 31312 5832 1032 R 4.4 1.1 0:00.17 php-cgi
1586 robert 20 0 31312 5856 1012 R 4.4 1.1 0:00.14 php-cgi
1589 robert 20 0 30008 5320 1728 R 4.4 1.0 0:00.12 php-cgi
1593 robert 20 0 30012 5208 1620 R 4.4 1.0 0:00.07 php-cgi
1594 robert 20 0 30016 5156 1616 R 4.4 1.0 0:00.07 php-cgi
1595 robert 20 0 30008 5320 1728 D 4.4 1.0 0:00.07 php-cgi
1597 robert 20 0 12072 464 276 R 4.4 0.1 0:00.03 php-cgi
1579 robert 20 0 32736 7844 1444 R 2.9 1.5 0:00.24 php-cgi
28 root 20 0 0 0 0 S 1.5 0.0 0:02.65 kswapd0
991 mysql 20 0 139m 14m 692 S 1.5 2.8 0:04.41 mysqld
1186 robert 20 0 35172 6184 984 R 1.5 1.2 0:00.07 httpd
1546 robert 20 0 53412 28m 1632 S 1.5 5.6 0:00.75 php-cgi
1596 robert 20 0 2696 476 228 R 1.5 0.1 0:00.01 top
1 root 20 0 2900 200 4 S 0.0 0.0 0:00.77 init
2 root 20 0 0 0 0 S 0.0 0.0 0:00.00 kthreadd
3 root RT 0 0 0 0 S 0.0 0.0 0:00.00 migration/0
4 root 20 0 0 0 0 S 0.0 0.0 0:00.00 ksoftirqd/0
5 root RT 0 0 0 0 S 0.0 0.0 0:00.00 migration/0
6 root RT 0 0 0 0 S 0.0 0.0 0:00.13 watchdog/0
7 root 20 0 0 0 0 S 0.0 0.0 0:00.33 events/0
8 root 20 0 0 0 0 S 0.0 0.0 0:00.00 cgroup
9 root 20 0 0 0 0 S 0.0 0.0 0:00.00 khelper
10 root 20 0 0 0 0 S 0.0 0.0 0:00.00 netns
11 root 20 0 0 0 0 S 0.0 0.0 0:00.00 async/mgr
12 root 20 0 0 0 0 S 0.0 0.0 0:00.00 pm
13 root 20 0 0 0 0 S 0.0 0.0 0:00.00 sync_supers
14 root 20 0 0 0 0 S 0.0 0.0 0:00.00 bdi-default
15 root 20 0 0 0 0 S 0.0 0.0 0:00.00 kintegrityd/0
16 root 20 0 0 0 0 R 0.0 0.0 0:03.46 kblockd/0
17 root 20 0 0 0 0 S 0.0 0.0 0:00.00 kacpid
18 root 20 0 0 0 0 S 0.0 0.0 0:00.00 kacpi_notify
19 root 20 0 0 0 0 S 0.0 0.0 0:00.00 kacpi_hotplug
20 root 20 0 0 0 0 S 0.0 0.0 0:00.00 ata/0
21 root 20 0 0 0 0 S 0.0 0.0 0:00.00 ata_aux
22 root 20 0 0 0 0 S 0.0 0.0 0:00.00 ksuspend_usbd
23 root 20 0 0 0 0 S 0.0 0.0 0:00.00 khubd
24 root 20 0 0 0 0 S 0.0 0.0 0:00.03 kseriod
25 root 20 0 0 0 0 S 0.0 0.0 0:00.00 md/0
26 root 20 0 0 0 0 S 0.0 0.0 0:00.00 md_misc/0
27 root 20 0 0 0 0 S 0.0 0.0 0:00.00 khungtaskd
29 root 25 5 0 0 0 S 0.0 0.0 0:00.00 ksmd
30 root 20 0 0 0 0 S 0.0 0.0 0:00.00 aio/0
31 root 20 0 0 0 0 S 0.0 0.0 0:00.00 crypto/0
36 root 20 0 0 0 0 S 0.0 0.0 0:00.00 kthrotld/0
38 root 20 0 0 0 0 S 0.0 0.0 0:00.00 kpsmoused
39 root 20 0 0 0 0 S 0.0 0.0 0:00.00 usbhid_resumer
189 root 20 0 0 0 0 S 0.0 0.0 0:00.00 scsi_eh_0
190 root 20 0 0 0 0 S 0.0 0.0 0:00.00 scsi_eh_1
208 root 20 0 0 0 0 S 0.0 0.0 0:00.00 virtio-blk
263 root 20 0 0 0 0 S 0.0 0.0 0:00.03 jbd2/vda-8
264 root 20 0 0 0 0 S 0.0 0.0 0:00.00 ext4-dio-unwrit
333 root 16 -4 2512 400 4 S 0.0 0.1 0:00.15 udevd
361 root 20 0 0 0 0 S 0.0 0.0 0:00.00 virtio-net
364 root 20 0 0 0 0 S 0.0 0.0 0:00.00 vballoon
543 root 18 -2 2508 396 4 S 0.0 0.1 0:00.00 udevd
546 root 20 0 0 0 0 S 0.0 0.0 0:00.00 kstriped
600 root 20 0 0 0 0 S 0.0 0.0 0:00.00 kauditd
816 root 20 0 2020 88 4 S 0.0 0.0 0:00.00 acpid
833 root 20 0 8940 512 4 S 0.0 0.1 0:00.00 sshd
868 root 20 0 3044 184 4 S 0.0 0.0 0:00.00 mysqld_safe
992 root 20 0 0 0 0 S 0.0 0.0 0:00.02 flush-253:0
1084 root 20 0 12960 636 4 S 0.0 0.1 0:00.03 master
1091 postfix 20 0 13036 620 4 S 0.0 0.1 0:00.00 pickup
1092 postfix 20 0 13108 672 4 S 0.0 0.1 0:00.00 qmgr
1094 root 20 0 34900 5036 88 S 0.0 1.0 0:00.52 httpd
1097 robert 20 0 20568 2320 4 S 0.0 0.5 0:00.00 httpd
1103 root 20 0 3956 560 4 S 0.0 0.1 0:00.01 crond
1106 robert 20 0 35172 5216 16 S 0.0 1.0 0:00.05 httpd
1107 robert 20 0 35172 5216 16 S 0.0 1.0 0:00.02 httpd
1119 postfix 20 0 14204 848 4 S 0.0 0.2 0:00.00 smtpd
1134 postfix 20 0 13180 656 4 S 0.0 0.1 0:00.00 cleanup
1138 root 20 0 2008 60 4 S 0.0 0.0 0:00.00 mingetty
1140 root 20 0 2008 56 4 S 0.0 0.0 0:00.00 mingetty
1142 root 20 0 2008 60 4 S 0.0 0.0 0:00.00 mingetty
1144 root 20 0 2008 64 4 S 0.0 0.0 0:00.00 mingetty
1146 root 20 0 2008 64 4 S 0.0 0.0 0:00.00 mingetty
1148 root 20 0 2008 64 4 S 0.0 0.0 0:00.00 mingetty
1150 postfix 20 0 13232 680 4 S 0.0 0.1 0:00.04 smtp
1151 postfix 20 0 13232 672 4 S 0.0 0.1 0:00.00 smtp
1159 root 20 0 11884 740 8 S 0.0 0.1 0:00.01 sshd
1160 robert 20 0 35172 5216 16 S 0.0 1.0 0:00.09 httpd
1164 robert 20 0 35172 5216 16 S 0.0 1.0 0:00.03 httpd
1165 robert 20 0 35172 5216 16 S 0.0 1.0 0:00.03 httpd
1172 robert 20 0 35172 5216 16 S 0.0 1.0 0:00.05 httpd
1174 robert 20 0 35172 5216 16 S 0.0 1.0 0:00.07 httpd
1175 robert 20 0 35172 5216 16 S 0.0 1.0 0:00.07 httpd
1184 robert 20 0 35172 5216 16 S 0.0 1.0 0:00.04 httpd
1185 robert 20 0 35172 5216 16 S 0.0 1.0 0:00.09 httpd
1187 robert 20 0 35172 5216 16 S 0.0 1.0 0:00.05 httpd
1188 robert 20 0 35600 5412 16 S 0.0 1.1 0:00.10 httpd
1189 robert 20 0 35172 5264 16 S 0.0 1.0 0:00.03 httpd
1190 robert 20 0 35172 5216 16 S 0.0 1.0 0:00.02 httpd
1191 robert 20 0 35172 5216 16 S 0.0 1.0 0:00.11 httpd
1196 robert 20 0 11884 748 4 S 0.0 0.1 0:00.00 sshd
1201 robert 20 0 8220 448 4 S 0.0 0.1 0:00.00 sftp-server
1208 robert 20 0 35172 5268 16 S 0.0 1.0 0:00.12 httpd
1214 robert 20 0 35172 5216 16 S 0.0 1.0 0:00.04 httpd
1220 robert 20 0 35172 5216 16 S 0.0 1.0 0:00.02 httpd
1221 robert 20 0 35172 5216 16 S 0.0 1.0 0:00.04 httpd
1222 robert 20 0 35172 5216 16 S 0.0 1.0 0:00.05 httpd
1223 robert 20 0 35172 5216 16 S 0.0 1.0 0:00.02 httpd
1229 robert 20 0 35172 5216 16 S 0.0 1.0 0:00.07 httpd
1238 root 20 0 11884 736 8 S 0.0 0.1 0:00.01 sshd
1260 robert 20 0 12020 752 4 S 0.0 0.1 0:00.59 sshd
1265 robert 20 0 3180 308 4 S 0.0 0.1 0:00.14 bash
1266 robert 20 0 35172 5216 16 S 0.0 1.0 0:00.07 httpd
1286 robert 20 0 35172 5216 16 S 0.0 1.0 0:00.10 httpd
1287 robert 20 0 35172 5216 16 S 0.0 1.0 0:00.02 httpd
1294 robert 20 0 35172 5216 16 S 0.0 1.0 0:00.01 httpd
1295 robert 20 0 35172 5224 16 S 0.0 1.0 0:00.06 httpd
1296 robert 20 0 35172 5216 16 S 0.0 1.0 0:00.02 httpd
1332 robert 20 0 35172 5216 16 S 0.0 1.0 0:00.09 httpd
1509 root 20 0 4324 668 4 S 0.0 0.1 0:00.00 crond
1510 root 20 0 4324 668 4 S 0.0 0.1 0:00.00 crond
1512 robert 20 0 6572 380 4 S 0.0 0.1 0:00.02 wget
1513 robert 20 0 6572 376 4 S 0.0 0.1 0:00.02 wget
1545 robert 20 0 53412 28m 1636 S 0.0 5.6 0:00.76 php-cgi
1547 robert 20 0 53412 28m 1632 S 0.0 5.6 0:00.74 php-cgi
Can you guys look at it. Thanks
EDIT:
I have ton of these entries in access_logs
104.245.97.218 - - [14/Apr/2015:07:54:18 +0000] "POST /xmlrpc.php HTTP/1.0" 200 370 "-" "Mozilla/4.0 (compatible: MSIE 7.0; Windows NT 6.0)"
104.245.97.218 - - [14/Apr/2015:07:54:18 +0000] "POST /xmlrpc.php HTTP/1.0" 200 370 "-" "Mozilla/4.0 (compatible: MSIE 7.0; Windows NT 6.0)"
104.245.97.218 - - [14/Apr/2015:07:54:19 +0000] "POST /xmlrpc.php HTTP/1.0" 200 370 "-" "Mozilla/4.0 (compatible: MSIE 7.0; Windows NT 6.0)"
104.245.97.218 - - [14/Apr/2015:07:54:19 +0000] "POST /xmlrpc.php HTTP/1.0" 200 370 "-" "Mozilla/4.0 (compatible: MSIE 7.0; Windows NT 6.0)"
104.245.97.218 - - [14/Apr/2015:07:54:20 +0000] "POST /xmlrpc.php HTTP/1.0" 200 370 "-" "Mozilla/4.0 (compatible: MSIE 7.0; Windows NT 6.0)"
104.245.97.218 - - [14/Apr/2015:07:54:20 +0000] "POST /xmlrpc.php HTTP/1.0" 200 370 "-" "Mozilla/4.0 (compatible: MSIE 7.0; Windows NT 6.0)"
104.245.97.218 - - [14/Apr/2015:07:54:21 +0000] "POST /xmlrpc.php HTTP/1.0" 200 370 "-" "Mozilla/4.0 (compatible: MSIE 7.0; Windows NT 6.0)"
104.245.97.218 - - [14/Apr/2015:07:54:21 +0000] "POST /xmlrpc.php HTTP/1.0" 200 370 "-" "Mozilla/4.0 (compatible: MSIE 7.0; Windows NT 6.0)"
104.245.97.218 - - [14/Apr/2015:07:54:21 +0000] "POST /xmlrpc.php HTTP/1.0" 200 370 "-" "Mozilla/4.0 (compatible: MSIE 7.0; Windows NT 6.0)"
104.245.97.218 - - [14/Apr/2015:07:54:22 +0000] "POST /xmlrpc.php HTTP/1.0" 200 370 "-" "Mozilla/4.0 (compatible: MSIE 7.0; Windows NT 6.0)"
104.245.97.218 - - [14/Apr/2015:07:54:22 +0000] "POST /xmlrpc.php HTTP/1.0" 200 370 "-" "Mozilla/4.0 (compatible: MSIE 7.0; Windows NT 6.0)"
104.245.97.218 - - [14/Apr/2015:07:54:23 +0000] "POST /xmlrpc.php HTTP/1.0" 200 370 "-" "Mozilla/4.0 (compatible: MSIE 7.0; Windows NT 6.0)"
104.245.97.218 - - [14/Apr/2015:07:54:23 +0000] "POST /xmlrpc.php HTTP/1.0" 200 370 "-" "Mozilla/4.0 (compatible: MSIE 7.0; Windows NT 6.0)"
104.245.97.218 - - [14/Apr/2015:07:54:24 +0000] "POST /xmlrpc.php HTTP/1.0" 200 370 "-" "Mozilla/4.0 (compatible: MSIE 7.0; Windows NT 6.0)"
104.245.97.218 - - [14/Apr/2015:07:54:24 +0000] "POST /xmlrpc.php HTTP/1.0" 200 370 "-" "Mozilla/4.0 (compatible: MSIE 7.0; Windows NT 6.0)"
104.245.97.218 - - [14/Apr/2015:07:54:24 +0000] "POST /xmlrpc.php HTTP/1.0" 200 370 "-" "Mozilla/4.0 (compatible: MSIE 7.0; Windows NT 6.0)"
104.245.97.218 - - [14/Apr/2015:07:54:25 +0000] "POST /xmlrpc.php HTTP/1.0" 200 370 "-" "Mozilla/4.0 (compatible: MSIE 7.0; Windows NT 6.0)"
104.245.97.218 - - [14/Apr/2015:07:54:25 +0000] "POST /xmlrpc.php HTTP/1.0" 200 370 "-" "Mozilla/4.0 (compatible: MSIE 7.0; Windows NT 6.0)"
104.245.97.218 - - [14/Apr/2015:07:54:26 +0000] "POST /xmlrpc.php HTTP/1.0" 200 370 "-" "Mozilla/4.0 (compatible: MSIE 7.0; Windows NT 6.0)"
104.245.97.218 - - [14/Apr/2015:07:54:26 +0000] "POST /xmlrpc.php HTTP/1.0" 200 370 "-" "Mozilla/4.0 (compatible: MSIE 7.0; Windows NT 6.0)"
104.245.97.218 - - [14/Apr/2015:07:54:26 +0000] "POST /xmlrpc.php HTTP/1.0" 200 370 "-" "Mozilla/4.0 (compatible: MSIE 7.0; Windows NT 6.0)"
104.245.97.218 - - [14/Apr/2015:07:54:27 +0000] "POST /xmlrpc.php HTTP/1.0" 200 370 "-" "Mozilla/4.0 (compatible: MSIE 7.0; Windows NT 6.0)"
104.245.97.218 - - [14/Apr/2015:07:54:27 +0000] "POST /xmlrpc.php HTTP/1.0" 200 370 "-" "Mozilla/4.0 (compatible: MSIE 7.0; Windows NT 6.0)"
104.245.97.218 - - [14/Apr/2015:07:54:28 +0000] "POST /xmlrpc.php HTTP/1.0" 200 370 "-" "Mozilla/4.0 (compatible: MSIE 7.0; Windows NT 6.0)"
104.245.97.218 - - [14/Apr/2015:07:54:28 +0000] "POST /xmlrpc.php HTTP/1.0" 200 370 "-" "Mozilla/4.0 (compatible: MSIE 7.0; Windows NT 6.0)"
104.245.97.218 - - [14/Apr/2015:07:54:29 +0000] "POST /xmlrpc.php HTTP/1.0" 200 370 "-" "Mozilla/4.0 (compatible: MSIE 7.0; Windows NT 6.0)"
104.245.97.218 - - [14/Apr/2015:07:54:29 +0000] "POST /xmlrpc.php HTTP/1.0" 200 370 "-" "Mozilla/4.0 (compatible: MSIE 7.0; Windows NT 6.0)"
104.245.97.218 - - [14/Apr/2015:07:54:29 +0000] "POST /xmlrpc.php HTTP/1.0" 200 370 "-" "Mozilla/4.0 (compatible: MSIE 7.0; Windows NT 6.0)"
104.245.97.218 - - [14/Apr/2015:07:54:30 +0000] "POST /xmlrpc.php HTTP/1.0" 200 370 "-" "Mozilla/4.0 (compatible: MSIE 7.0; Windows NT 6.0)"
104.245.97.218 - - [14/Apr/2015:07:54:30 +0000] "POST /xmlrpc.php HTTP/1.0" 200 370 "-" "Mozilla/4.0 (compatible: MSIE 7.0; Windows NT 6.0)"
104.245.97.218 - - [14/Apr/2015:07:54:31 +0000] "POST /xmlrpc.php HTTP/1.0" 200 370 "-" "Mozilla/4.0 (compatible: MSIE 7.0; Windows NT 6.0)"
access_logs
.