CODEWITHSUNDEEP
mysqllinuxioswap
AL-Kateb
AL-Kateb

Reputation: 2962

swap space used while physical memory is free

i recently have migrated between 2 servers (the newest has lower specs), and it freezes all the time even though there is no load on the server, below are my specs:

HP DL120G5 / Intel Quad-Core Xeon X3210 / 8GB RAM

free -m output:

             total       used       free     shared    buffers     cached
Mem:          7863       7603        260          0        176       5736
-/+ buffers/cache:       1690       6173
Swap:         4094        412       3681

as you can see there is 412 mb ysed in swap while there is almost 80% of the physical ram available

I don't know if this should cause any trouble, but almost no swap was used in my old server so I'm thinking this does not seem right.

i have cPanel license so i contacted their support and they noted that i have high iowait, and yes when i ran sar i noticed sometimes it exceeds 60%, most often it's 20% but sometimes it reaches to 60% or even 70%

i don't really know how to diagnose that, i was suspecting my drive is slow and this might cause the latency so i ran a test using dd and the speed was 250 mb/s so i think the transfer speed is ok plus the hardware is supposed to be brand new.

the high load usually happens when i use gzip or tar to extract files (backup or restore a cpanel account).

one important thing to mention is that top is reporting that mysql is using 100% to 125% of the CPU and sometimes it reaches much more, if i trace the mysql process i keep getting this error continually:

setsockopt(376, SOL_IP, IP_TOS, [8], 4) = -1 EOPNOTSUPP (Operation not supported)

i don't know what that means nor did i get useful information googling it.

i forgot to mention that it's a web hosting server for what it's worth, so it has the standard setup for web hosting (apache,php,mysql .. etc)

so how do i properly diagnose this issue and find the solution, or what might be the possible causes?

Upvotes: 2

Views: 7476

Answers (1)

DocSalvager
DocSalvager

Reputation: 2254

As you may have realized by now, the free -m output shows 7603MiB (~7.6GiB) USED, not free.

You're out of memory and it has started swapping which will drastically slow things down. Since most applications are unaware that the virtual memory is now coming from much slower disk, the system may very well appear to "hang" with no feedback describing the problem.

From your description, the first process I'd kill in order to regain control would be the Mysql. If you have ssh/rsh/telnet connectivity to this box from another machine, you may have to login from that in order to get a usable commandline to kill from.

My first thought (hypothesis?) for what's happening is...

  • MySQL is trying to do something that is not supported as this machine is currently configured. It could be missing a library or an environment variable is not set or any number things.

  • That operation allocates some memory but is failing and not cleaning up the allocation when it does. If this were a shell script, it could be fixed by putting an event trap command at the beginning that runs a function that releases memory and cleans up.

  • The code is written to keep retrying on failure so it rapidly uses up all your memory. Refering back to the shell script illustration, the trap function might also prompt to see if you really want to keep retrying.

Not a complete answer but hopefully will help.

Upvotes: 1

Related Questions