whatsthemenu
whatsthemenu

Reputation: 29

Cannot access ec2 after chmod -R 777 in root

I accidentally ran "sudo chmod -R 777 ./" on my Ubuntu ec2 AWS instance. After that it kicked me out and I cannot connect to the instance anymore, neither via FTP, nor via SSH. Do you have any tip for me how I can recover my instance?

Best John

Upvotes: 2

Views: 2112

Answers (2)

Mike Musni
Mike Musni

Reputation: 171

Here’s the revised text with improved grammar:


Here’s what I did:

  1. I logged in to the AWS web console.
  2. I selected the instance and connect to it.

Ideally, you should be able to access your EC2 instance using the web console. Once you are inside the instance, go up one directory by running:

cd ..

Then, run the command:

sudo chmod -R 0755 ec2-user

Upvotes: 0

TJ Zimmerman
TJ Zimmerman

Reputation: 3484

Whatever directory you were in when you ran chmod has had the permissions for all of the files and directories that begin with a . recursively changed.

Since your entire system stopped working after you ran this command, I assume you were in a non-trivial directory and managed to change the permissions for some essential binaries.

As a result, they are no longer functioning correctly which has rendered your system unusable. There are three ways to recover from this. You won't like the first one. The second one probably isn't an option given your question. And the third one is really annoying to execute.

First of all, I recommend that you delete the EC2 Instance and recreate it from scratch. Maybe this time recreate your manual changes with an CaC tool like Ansible to make this less painful in the future. :)

If that won't work for you, then restore the filesystem from a backup or snapshot. You have those...Right? Ah, no worries. I've been there before too. Sorry about that.

Now, onto the annoying option. You might be able to recover by stopping your EC2 instance and removing the EBS disk. Then provision a new EC2 instance in the same Availability Zone and attach your old disk volume to it at an unused mountpoint. Power on the EC2 instance, SSH into it, mount the old filesystem, and manually repair the permissions you recursively modified with chmod. After this, you should be able to stop that instance, add the disk back to your original instance, power it back on, and recover its state.

If you have modified a massive directory tree like /etc/ then you're probably in really rough shape. Possibly irrecoverable with any rational amount of effort for a Linux server. However, one potential other solution you might be able to explore is using chroot.

Follow my steps above to the point where you mount the old filesystem on the new EC2 Instance. Then mount the filesystem to a space on your new filesystem and run chroot /mnt/old/disk/location. If you manage to change into your own filesystem, then you can hopefully automatically recover by using apt-get to reinstall your pacakges. apt-get --reinstall install. After that, simply exit the chroot session and attach the disk to your old EC2 Instance and see where that gets you.

Let me know if you need any more help. I haven't had to recover from this situation specifically before, but I have recovered from destroyed root partitions on Linux filesystems on EC2 using this process before.

Upvotes: 2

Related Questions