How Do I Find Cpanel Memory RAM Problems

Question

I'm having a severe problem find out where I'm running out of memory on my CentOS cPanel Server. I'll Try to provide all the details that I can. I previously had the same server without cPanel and had no issues, so I'm thinking it's a cPanel problem.

I am currently running a cPanel server with CentOS 6.6 Installed, with 8GB of RAM and 1.5TB of Storage (Keep in mind my previous server without cPanel only has 4GB of RAM too).

Here's all my memory statistics.

$ free -m
            total       used       free     shared    buffers     cached
Mem:          8192       3647       4544       2560          0       1164
-/+ buffers/cache:       2483       5708
Swap:            0          0          0          0          0          0



$ cat /proc/user_beancounters
uid     resource                     held              maxheld              barrier                limit              failcnt
16167:  kmemsize                642249817            642637824  9223372036854775807  9223372036854775807                    0
        lockedpages                  4610                 4610              2097152              2097152                    0
        privvmpages               1971819              1972978              2097152              2097152                11579
        shmpages                   655390               655390  9223372036854775807  9223372036854775807                    0
        numproc                       493                  504                32567                32567                    0
        physpages                  932503               933409              2097152              2097152                    0
        vmguarpages                     0                    0              2097152              2097152                    0
        oomguarpages               478382               478413              2097152              2097152                    0




$ smem -u -t -k
User     Count     Swap      USS      PSS      RSS 
mailnull     1        0   996.0K  1001.0K     1.5M 
dovecot      2        0     1.0M     1.1M     3.2M 
memcached     2        0     1.1M     1.3M     2.3M 
varnish      1        0     1.2M     1.5M     2.7M 
apache       5        0   716.0K     2.1M    27.5M 
lighttpd     6        0     2.6M     4.0M    30.1M 
dovenull     4        0     3.4M     4.1M    13.6M 
newrelic     2        0     4.6M     4.9M     6.5M 
media        7        0     4.6M     5.8M    18.3M 
redis        5        0     5.7M     6.1M    10.5M 
ldap         1        0     7.6M     7.7M     8.4M 
user         9        0    10.3M    12.6M    24.1M 
postgres     7        0    11.8M    14.0M    25.8M 
named        1        0    27.1M    27.1M    27.9M 
ntop         1        0    30.2M    31.8M    35.0M 
mongod       1        0    46.8M    46.8M    47.7M 
elasticsearch     1        0   201.7M   205.0M   212.7M 
graylog2     1        0   262.1M   265.5M   273.4M 
nobody      20        0   434.3M   488.6M   789.4M 
mysql        1        0   489.6M   489.8M   492.1M 
root        58        0   628.2M   695.7M   847.9M 
---------------------------------------------------
           136        0     2.1G     2.3G     2.8G


$ vzubc -c
----------------------------------------------------------------
CT 16167     | HELD Bar% Lim%| MAXH Bar% Lim%| BAR | LIM | FAIL
-------------+---------------+---------------+-----+-----+------
  lockedpages|  18M 0.2% 0.2%|  18M 0.2% 0.2%|   8G|   8G|    - 
  privvmpages|7.51G  93%  93%|7.52G  94%  94%|   8G|   8G| 11.3K
      numproc| 495    2%   2%| 504    2%   2%|31.8K|31.8K|    - 
    physpages|3.55G  44%  44%|3.56G  44%  44%|   8G|   8G|    - 
  vmguarpages|   -    -    - |   -    -    - |   8G|   8G|    - 
 oomguarpages|1.82G  22%  22%|1.82G  22%  22%|   8G|   8G|    - 
       numpty|   4    2%   2%|   4    2%   2%| 255 | 255 |    - 
   numsiginfo|   -    -    - |  12    1%   1%|   1K|   1K|    - 
----------------------------------------------------------------

Also in the administrative side of my server I can see

CPU USAGE - 13.33% DISK SPACE USAGE - 2.54% / 1536GB RAM USAGE - 28.64% / 8GB

Continuous errors I'm getting on Command line are

Unable to Fork, Cannot allocate memory Segmentation Fault

As well as continuous failure of Tailwatchd failing and a few other services failing here and there. I used the Tweak Setting to change memory from 512MB to 4096MB to Unlimited MB to see if it changed anything, with no change. I also changed the Conserve Memory option both to on and off to see if there were any changes, with none.

Also, I tried to check the vz container settings. I have 1 config for 0.conf but nothing for 16167.conf. I tried to adjust setting in 0.conf with no luck, then created 16167.conf and adjusted settings which continued not to show up after a server restart. I experimented with the differet ve templates from 1G all the way to 4G, and again no improvement.

Any help or point in the right direction would be greatly appreciated. I've tried to make any corrections I could and all the research I could before asking the community, but I think at this pint, I need some help with it. Thanks in advance.

Answer 1

To resolve this issues, You will have to increase privvmpages value for your VM. You can increase it through your main node with the following command.

vzctl set ${cid} --privvmpages 1024M:2048M --save 

With the above command you will get 1024MB Guaranteed, 2048MB Burstable memory. Change it with your requirement and check it again.

Answer 2

I would suggest to tune your mysql for your system memory (half of it) and then also tune the apache through cpanel... most times it is the sql part.

you do also have elastic search. and your root memory is too high.

But on the current system you do have 5,7GB free memory. Are you sure you are out of mem?

It seems your provider has oversold the memory (the only thought i have about the segfault you do have)