apache running slow without using all ram

Question

I have a centos server running apache with 8GB ram. It is running very slowly to load simple php pages. I have set the following on my config file. I cannot see any httpd process more than 100m. It usually slows down after 5mins from a restart.

<IfModule prefork.c>
StartServers            12
MinSpareServers         12
MaxSpareServers         12
ServerLimit             150
MaxClients              150
MaxRequestsPerChild     1000
</IfModule>

 $ ps -ylC httpd | awk '{x += $8;y += 1} END {print "Apache Memory Usage (MB): "x/1024; print "Average Proccess Size (MB): "x/((y-1)*1024)}'
Apache Memory Usage (MB): 1896.09
Average Proccess Size (MB): 36.4633

What else can I do to make the pages load faster.

$ free -m
             total       used       free     shared    buffers     cached
Mem:          7872        847       7024          0         29        328
-/+ buffers/cache:        489       7382
Swap:         7999        934       7065



top - 15:42:17 up 545 days, 16:46,  2 users,  load average: 0.05, 0.06, 0.
Tasks: 251 total,   1 running, 250 sleeping,   0 stopped,   0 zombie
Cpu(s):  0.0%us,  2.3%sy,  0.0%ni, 97.7%id,  0.0%wa,  0.0%hi,  0.0%si,  0.
Mem:   8060928k total,   909112k used,  7151816k free,    30216k buffers
Swap:  8191992k total,   956880k used,  7235112k free,   336612k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
16544 apache    20   0  734m  47m  10m S  0.0  0.6   0:00.21 httpd
16334 apache    20   0  731m  45m  10m S  0.0  0.6   0:00.41 httpd
16212 apache    20   0  723m  37m  10m S  0.0  0.5   0:00.72 httpd
16555 apache    20   0  724m  37m  10m S  0.0  0.5   0:00.25 httpd
16347 apache    20   0  724m  36m  10m S  0.0  0.5   0:00.42 httpd
16608 apache    20   0  721m  34m  10m S  0.0  0.4   0:00.16 httpd
16088 apache    20   0  717m  31m  10m S  0.0  0.4   0:00.35 httpd
16012 apache    20   0  717m  30m  10m S  0.0  0.4   0:00.78 httpd
16338 apache    20   0  716m  30m  10m S  0.0  0.4   0:00.36 httpd
16336 apache    20   0  715m  29m  10m S  0.0  0.4   0:00.42 httpd
16560 apache    20   0  716m  29m 9.9m S  0.0  0.4   0:00.06 httpd
16346 apache    20   0  715m  28m  10m S  0.0  0.4   0:00.28 httpd
16016 apache    20   0  714m  28m  10m S  0.0  0.4   0:00.74 httpd
16497 apache    20   0  715m  28m  10m S  0.0  0.4   0:00.18 httpd
16607 apache    20   0  714m  27m   9m S  0.0  0.4   0:00.17 httpd
16007 root      20   0  597m  27m  15m S  0.0  0.3   0:00.13 httpd
16694 apache    20   0  713m  26m  10m S  0.0  0.3   0:00.10 httpd
16695 apache    20   0  712m  25m 9.9m S  0.0  0.3   0:00.04 httpd
16554 apache    20   0  712m  25m  10m S  0.0  0.3   0:00.15 httpd
16691 apache    20   0  598m  14m 2752 S  0.0  0.2   0:00.00 httpd
22613 root      20   0  884m  12m 6664 S  0.0  0.2 132:10.11 agtrep
16700 apache    20   0  597m  12m  712 S  0.0  0.2   0:00.00 httpd
16750 apache    20   0  597m  12m  712 S  0.0  0.2   0:00.00 httpd
16751 apache    20   0  597m  12m  712 S  0.0  0.2   0:00.00 httpd
 2374 root      20   0 2616m 8032 1024 S  0.0  0.1 171:31.74 python
 9699 root       0 -20 50304 6488 1168 S  0.0  0.1   1467:01 scopeux
 9535 root      20   0  644m 6304 2700 S  0.0  0.1  21:01.24 coda
14976 root      20   0  246m 5800 2452 S  0.0  0.1  42:44.70 sssd_be
22563 root      20   0  825m 4704 2636 S  0.0  0.1  44:07.68 opcmona
22496 root      20   0  880m 4540 3304 S  0.0  0.1  13:54.78 opcmsga
22469 root      20   0  856m 4428 2804 S  0.0  0.1   1:18.45 ovconfd
22433 root      20   0  654m 4144 2752 S  0.0  0.1  10:45.71 ovbbccb
22552 root      20   0  253m 2936 1168 S  0.0  0.0  50:35.27 opcle
22521 root      20   0  152m 1820 1044 S  0.0  0.0   0:53.57 opcmsgi
14977 root      20   0  215m 1736 1020 S  0.0  0.0  15:53.13 sssd_nss
16255 root      20   0  254m 1704 1152 S  0.0  0.0  92:07.63 vmtoolsd
24180 root     -51 -20 14788 1668 1080 S  0.0  0.0   9:48.57 midaemon

I do not have access to root

I have updated it to the following which seems to be better but i see occasional 7GB httpd process

<IfModule prefork.c>
StartServers            12
MinSpareServers         12
MaxSpareServers         12
ServerLimit             150
MaxClients              150
MaxRequestsPerChild     0
</IfModule>

top - 09:13:42 up 546 days, 10:18,  2 users,  load average: 1.86, 1.51, 0.78
Tasks: 246 total,   2 running, 244 sleeping,   0 stopped,   0 zombie
Cpu(s): 28.6%us,  9.5%sy,  0.0%ni, 45.2%id, 16.7%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:   8060928k total,  7903004k used,   157924k free,     2540k buffers
Swap:  8191992k total,  8023596k used,   168396k free,    31348k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
 2466 apache    20   0 14.4g 7.1g  240 R 100.0 92.1   4:58.95 httpd
 2285 apache    20   0  730m  31m 7644 S  0.0  0.4   0:02.37 httpd
 2524 apache    20   0  723m  23m 7488 S  0.0  0.3   0:01.75 httpd
 3770 apache    20   0  716m  21m  10m S  0.0  0.3   0:00.29 httpd
 3435 apache    20   0  716m  20m 9496 S  0.0  0.3   0:00.60 httpd
 3715 apache    20   0  713m  19m  10m S  0.0  0.2   0:00.35 httpd
 3780 apache    20   0  713m  19m  10m S  0.0  0.2   0:00.22 httpd
 3778 apache    20   0  713m  19m  10m S  0.0  0.2   0:00.28 httpd
 3720 apache    20   0  712m  18m  10m S  0.0  0.2   0:00.21 httpd
 3767 apache    20   0  712m  18m  10m S  0.0  0.2   0:00.21 httpd
 3925 apache    20   0  712m  17m  10m S  0.0  0.2   0:00.11 httpd
 2727 apache    20   0  716m  17m 7576 S  0.0  0.2   0:01.66 httpd
 2374 root      20   0 2680m  14m 2344 S  0.0  0.2 173:44.40 python
 9699 root       0 -20 50140 5556  624 S  0.0  0.1   1475:46 scopeux
 3924 apache    20   0  598m 5016 2872 S  0.0  0.1   0:00.00 httpd
 3926 apache    20   0  598m 5000 2872 S  0.0  0.1   0:00.00 httpd
14976 root      20   0  246m 2400 1280 S  0.0  0.0  42:51.54 sssd_be
 9535 root      20   0  644m 2392  752 S  0.0  0.0  21:07.36 coda
22563 root      20   0  825m 2000  952 S  0.0  0.0  44:16.37 opcmona
22552 root      20   0  254m 1820  868 S  0.0  0.0  50:48.12 opcle
16255 root      20   0  254m 1688 1144 S  0.0  0.0  92:53.74 vmtoolsd
22536 root      20   0  282m 1268  892 S  0.0  0.0  24:21.73 opcacta
16784 root      20   0  597m 1236  180 S  0.0  0.0   0:02.16 httpd
14977 root      20   0  215m 1092  864 S  0.0  0.0  15:57.32 sssd_nss
22496 root      20   0  880m 1076  864 S  0.0  0.0  13:57.86 opcmsga
22425 root      20   0 1834m  944  460 S  0.0  0.0  74:12.96 ovcd
22433 root      20   0  654m  896  524 S  0.0  0.0  10:48.00 ovbbccb
 2634 oiadmin   20   0 15172  876  516 R  9.1  0.0   0:14.78 top
 2888 root      20   0  103m  808  776 S  0.0  0.0   0:00.19 sshd
16397 root      20   0  207m  748  420 S  0.0  0.0  32:52.23 ManagementAgent
 2898 oiadmin   20   0  103m  696  556 S  0.0  0.0   0:00.08 sshd
22613 root      20   0  884m  580  300 S  0.0  0.0 132:34.94 agtrep
20886 root      20   0  245m  552  332 S  0.0  0.0  79:09.05 rsyslogd
 2899 oiadmin   20   0  105m  496  496 S  0.0  0.0   0:00.03 bash
24180 root     -51 -20 14788  456  408 S  0.0  0.0   9:50.43 midaemon
14978 root      20   0  203m  440  308 S  0.0  0.0   9:28.87 sssd_pam
14975 root      20   0  203m  432  288 S  0.0  0.0  21:45.01 sssd
 8215 root      20   0 88840  420  256 S  0.0  0.0   3:28.13 sendmail
18909 oiadmin   20   0  103m  408  256 S  0.0  0.0   0:02.83 sshd
 1896 root      20   0  9140  332  232 S  0.0  0.0  50:39.87 irqbalance
 2990 oiadmin   20   0 98.6m  320  276 S  0.0  0.0   0:00.04 tail
 4427 root      20   0  114m  288  196 S  0.0  0.0   8:58.77 crond
25628 root      20   0  4516  280  176 S  0.0  0.0  11:15.24 ndtask
 4382 ntp       20   0 28456  276  176 S  0.0  0.0   0:28.61 ntpd
 8227 smmsp     20   0 78220  232  232 S  0.0  0.0   0:05.09 sendmail
25634 root      20   0  6564  200   68 S  0.0  0.0   4:50.30 mgsusageag
 4926 root      20   0  110m  188  124 S  0.0  0.0   3:23.79 abrt-dump-oops
 9744 root      20   0  197m  180  136 S  0.0  0.0   1:46.59 perfalarm
22469 root      20   0  856m  128  128 S  0.0  0.0   1:18.65 ovconfd
 4506 rpc       20   0 19036   84   40 S  0.0  0.0   1:44.05 rpcbind
32193 root      20   0 66216   68   60 S  0.0  0.0   4:54.51 sshd
18910 oiadmin   20   0  105m   52   52 S  0.0  0.0   0:00.11 bash
22521 root      20   0  152m   44   44 S  0.0  0.0   0:53.71 opcmsgi
18903 root      20   0  103m   12   12 S  0.0  0.0   0:00.22 sshd
    1 root      20   0 19356    4    4 S  0.0  0.0   3:57.84 init
 1731 root      20   0  105m    4    4 S  0.0  0.0   0:01.91 rhsmcertd
 1983 dbus      20   0 97304    4    4 S  0.0  0.0   0:16.92 dbus-daemon
 2225 root      20   0  4056    4    4 S  0.0  0.0   0:00.01 mingetty

Answer 1

Your server is slow because you are condeming it to a non-threaded, ever-reclaiming childs scenario.

That is you use more than 12 processes but your maxspareservers is 12 so HTTPD is constantly spawning and despawning processes and precisely that's the biggest weakness of a non-threaded mpm. And also such a low MaxRequestsPerChild won't help either if you have a decent amount of requests per second, although understable since you are using mod_php, that value will increase the constant re-spawning.

In any OS, spawning processes uses much more cpu than spawning threads inside a process.

So either you set MaxSpareServers to a very high number so your server have lots of them ready to serve your requests, or you STOP using mod_php+prefork and probably .htaccess (like everyone in here who seems to believe that's needed for apache httpd to work). to a more reliable: HTTPD with mpm_event + mod_proxy_fcgi + php-fpm, where you can configure dozens of hundreds of threads and apache will spawn and use them in less than a blink and you leave all your php load under php's own daemon, php-fpm.

So, it's not apache, it is your ever-respawning processes setup in a non-threaded mpm what's giving you troubles.