CODEWITHSUNDEEP
phpwordpressapacheamazon-web-services
Glenak1911
Glenak1911

Reputation: 265

Wordpress site on Apache and AWS crashes without obvious leads

I recently migrated my Wordpress site to AWS for practice in self-hosting on the cloud. I have an EC2 instance in an autoscaling group behind an Application Load Balancer, and I have my Wordpress data on an EFS that gets mounted when an instance gets provisioned, and I am leveraging RDS for my database(MySQL).The site gets minimal traffic, besides me, maybe 2-3 visitors per week, so very much a personal project really.

Everything worked flawlessly until last week when my site went offline. Trying to access the url I received a 504 error, I wasn't able to ssh into the servers, and upon a reboot, I was able to see that there were a large amount of httpd instances are running. The site stays up for about 10 minutes before the servers become unresponsive. Looking through the logs on reboot, I see a lot of out of memory errors, specifically:

Cannot allocate memory: AH00481: fork: Unable to fork new process

I updated my httpd config to include custom prefork settings as noted in: Problems with Apache servers and A LOT of httpd processes

As well as : https://arstechnica.com/civis/viewtopic.php?t=1185478 I also thought it could be code related, but no code had changed for more than a week prior to this issue occurring.

I also scaled up my instance type up to a t2.small from a t2.micro, so my instance went from having 1 gig of RAM to about 2 gigs.

In the past, I've had sites go down due to brute force attacks so I checked the access logs for anything peculiar. I saw a few requests posting to xmlrpc.php, so I updated my .htaccess to block access to that file, as noted in: https://wordpress.stackexchange.com/questions/156522/restrict-access-to-xmlrpc-php

I also noticed healthy traffic from:

I also see entries in the logs that I concluded to be port scanning, as well as:

- - [06/May/2018:19:18:32 +0000] "POST /user/register?element_parents=account/mail/%23value&ajax_form=1&_wrapper_format=drupal_ajax HTTP/1.1" 404 20630 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64)"

- - [06/May/2018:19:14:02 +0000] "GET /jmx-console/HtmlAdaptor?action=inspectMBean&name=jboss.system%3Aservice%3DMainDeployer HTTP/1.1" 404 20630 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36"

Given that the return code is 404 for these requests, I don't think that they are anything to worry about.

I wonder if my Wordpress configuration, or EFS performance has anything to do with the issue, but everything is pretty much out of the box.

Does anybody have any ideas of other troubleshooting steps I can try, or anything I may have overlooked or done incorrectly?

Upvotes: 0

Views: 973

Answers (1)

chris
chris

Reputation: 37480

If your wordpress installation is on EFS, chances are you are running out of EFS credits - the total amount of I/O you have is directly related to the size of the EFS volume, and a small installation doesn't get a lot of EFS I/O - so little, in fact, that the health check alone can bring your site down.

Look at the BurstCreditBalance metric for your EFS volume to confirm.

AWS has recently updated the reference architecture for Wordpress - the v2 version has a trigger on credits that adds a 1GB dummy file to EFS if credits drop below a certain threshold.

Upvotes: 2

Related Questions