CODEWITHSUNDEEP
amazon-web-servicesdockeramazon-elastic-beanstalk
François-Xavier Gsell
François-Xavier Gsell

Reputation: 419

AWS Elastic BeanStalk Docker root file system switch to read-only

I use a docker application running on beanstalk for autoscaling.

The / file system within docker switch itself to read only at random times.

The application is generating a lot of logs but they are written to a dedicated volume and this one doesn't have the issue.

I tried to issue mount -o remount,rw / within the container, but I get "permission denied"

Upvotes: 3

Views: 1781

Answers (2)

Ishan Madhusanka
Ishan Madhusanka

Reputation: 791

We had the same issue with ElasticBeanstalk when we were using a heavy IO consuming application. The problem did not get recreated when we using the same application in an EC2 instance.

And then moving to ElasticContainerService resolved the issue for us.

Upvotes: 0

Dan Pierkowski
Dan Pierkowski

Reputation: 71

This happened to us twice recently, within about 48 hours. Both times we 'fixed' the problem by redeploying the docker image to Elastic Beanstalk. During the second redeploy EBS moved our app onto a new EC2 instance. The problem hasn't recurred since (it's been a couple of weeks).

Our theory is that the error was being caused by a failing disk. The kernel can remount a file system as read-only if corruption is detected. (This also might fit with the error you saw trying to remount the file system as read-write?). Since we've been moved onto new hardware we no longer have access to the old EC2 instance to investigate further.

Leaving this comment here in case this happens to someone else and they can continue the investigation.

Upvotes: 5

Related Questions