Reputation: 3855

Docker push to AWS ECR hangs immediately and times out

I'm trying to push my first docker image to ECR. I've followed the steps provided by AWS and things seem to be going smoothly until the final push which immediately times out. Specifically, I pass my aws ecr credentials to docker and get a "login succeeded" message. I then tag the image which also works. pushing to the ecr repo I get no error message, just the following:

The push refers to repository [xxxxxxxxxxx.dkr.ecr.ca-central-1.amazonaws.com/reponame]
714c1b96dd83: Retrying in 1 second 
d2cdc77dd068: Retrying in 1 second 
30aad807caf5: Retrying in 1 second 
0559774c4ea2: Retrying in 1 second 
285b8616682f: Retrying in 1 second 
4aeea0ec2b15: Waiting 
1b1312f842d8: Waiting 
c310009e0ef3: Waiting 
a48777e566d3: Waiting 
2a0c9f28029a: Waiting 
EOF

It tries a bunch of times and then exits with no message. Any idea what's wrong?

Upvotes: 196

Answers (29)

Ppeters62

Reputation: 1

In case this also helps, in my case docker had paused to "save memory." I restarted docker and was able to connect.

It wasn't an issue with using the correct aws profile for me.

Upvotes: 0

Usman Ali

Reputation: 1

If you think that everything is fine from docker perspective and the registry do exists and docker is able to login then, do check the policy for your registry that fits your usecase. For instance, if your docker server is in one AWS account and the registry in another AWS account, then you will need to provide a policy to your registry and vice versa. For cross-AWS accounts, below is the solution.

https://repost.aws/knowledge-center/secondary-account-access-ecr

Also a video on this, https://youtu.be/YvqImD7pe9g

Upvotes: 0

Allen Jose

Reputation: 41

I fix this issue by using the commands

docker tag <Image-ID> 754431986477.dkr.ecr.ap-south-1.amazonaws.com/<aws repository name>:latest

docker push 754431986477.dkr.ecr.ap-south-1.amazonaws.com/<aws repository name>:latest

Upvotes: 0

Aman Goel

Reputation: 3541

If EOF is getting printed in console after couple of retrying statements then below solution is works:

add permission named "AmazonEC2ContainerRegistryFullAccess" to same IAM user.
add "ecr:UploadLayerPart" action in inline policy of IAM user.

then try:

docker push <ecr repo uri>

Above solution worked for me.

Upvotes: 0

Dave

Reputation: 1645

In my case I didn't specify a region in my policy:

- Effect: Allow
  Action:
    - ecr:*
  Resource: !Sub "arn:aws:ecr::${AWS::AccountId}:repository/my-image"

Should be:

- Effect: Allow
  Action:
    - ecr:*
  Resource: !Sub "arn:aws:ecr:eu-central-1:${AWS::AccountId}:repository/my-image"

Upvotes: 0

Aayush Mohta

Reputation: 64

In my case, it was a configuration issue, aws cli was configured for my other aws account and i was trying to push my image in my mother account

Figured it out with aws configure, and changed the necessary credentials.

Upvotes: 0

yakob abada

Reputation: 1287

The easiest way I have found is. Go to the console and create a repository. Click on the created repo to see the view push commands button. click and it will list all commands you need to build and push the image into the repo. you don't need to do any changes.

Upvotes: 0

King.Zevin

Reputation: 71

For me, I misconfigured the tag name like this:

$ docker tag <image_name>:latest xxxxxxxxxxx.dkr.ecr.ca-central-1.amazonaws.com/reponame/<additional_string>:latest
$ docker push xxxxxxxxxxx.dkr.ecr.ca-central-1.amazonaws.com/reponame/<additional_string>:latest
...timeout...

And I found the <additional_string> should be deleted:

$ docker tag <image_name>:latest xxxxxx.amazonaws.com/reponame:latest
$ docker push xxxxxx.amazonaws.com/reponame:latest
...success!...

Upvotes: 4

Rotem jackoby

Reputation: 22128

Haven't found any solution?

Maybe you are falling in the use case I had.

identifiers:

You working on different AWS accounts on the same terminal.
When jumping between AWS accounts, you're using export AWS_PROFILE (directly or via some tool or command) in the terminal level to export the profile.
You are able to push sometimes, and sometimes not.
You do manage to login with aws ecr get-login-password.

What was the cause?

When I switched between AWS accounts, although I had a valid session token for each account, BUT I forgot sometimes to export AWS_PROFILE back the relevant profile.

Upvotes: 0

Roopesh

Reputation: 51

The issue was resolved, when I created the docker repository first (in ECR) and then pushed it to ECR. Remember to create the docker repository, before running the docker push command.

Upvotes: 4

Zac Rogerson

Reputation: 171

I have to add for anyone else encountering this problem. Go to IAM and make sure you have put permissions. I don't want to say how long I wasted before figuring that out.

Edit to help @zac's answer: The policies that need to be attached are AmazonEC2ContainerRegistryFullAccess and AWSAppRunnerServicePolicyForECRAccess

Upvotes: 17

Elia Weiss

Reputation: 9896

I had this problem with sam deploy

sam delete --stack-name ...
sam deploy --guided

worked for me

Upvotes: 0

juanignaciosl

Reputation: 3573

I was following this documentation and hit this error. What addressed the problem was using the repository id instead of the account name.

aws ecs create-repository creates a repo, returning a repositoryUri. Then, the docker login, docker tag and docker push should be done using that repository url instead of the user one.

Upvotes: 0

user19303171

Reputation: 521

Make sure the name of your repository is the same name as your images. image:latest 756839881602.dkr.ecr.us-east-1.amazonaws.com/image:latest in this case my repository name is image and my image name is image as well. This worked for me.

Upvotes: 52

Hieu

Reputation: 267

Assuming you authenticated successfully to AWS and you have permissions to read, write to ECR, check if the repository does exist

aws ecr describe-repositories --repository-name reponame

If you catch an error RepositoryNotFoundException, then you will create to that repository with the following command

aws ecr create-repository --repository-name reponame

After that, try to push again, it will be fine!

Upvotes: 13

Stian

Reputation: 1391

Ensure you are using the correct profile and that the repository exists

Command to login with profile: aws ecr get-login-password --region <region> --profile=<profile-name> | docker login --username AWS --password-stdin <aws-account-id>.dkr.ecr.eu-west-1.amazonaws.com

Command to create repo if it does not exists: aws ecr describe-repositories --repository-names ${REPO_NAME} || aws ecr create-repository --repository-name ${REPO_NAME}(source)

Upvotes: 6

Bhaskar Reddy

Reputation: 91

Make sure your assumed aws role has the ability to push images to AWS ECR. Easiest is to check the role via the command:

aws sts get-caller-identity --profile=saml

Upvotes: 1

whatspoppin

Reputation: 383

For those who tried the solution above, and it didn't work, make sure the image name your are pushing is the same as the repository name.

Upvotes: 7

alf

Reputation: 18530

In my case I was creating the repo in us-east-2 and attempting to push to us-east-1, so docker couldn't find it.

Upvotes: 0

Param Natarajan

Reputation: 11

Please check cloud trail event logs , this is where all the api issues are clearly highlighted .

In my case it was because i had a - in my image name and hence it was throwing the following error in the cloud trail logs

"The repository with name 'myimage-nginx' does not exist in the registry with id '516583196897'

Please note the - in the image name.

Fixing the image name to remove the - resolved the issue for me.

Commands

docker tag nginx:latest 516583196897.dkr.ecr.ap-south-1.amazonaws.com/myimage:latest

docker push 516583196897.dkr.ecr.ap-south-1.amazonaws.com/myimage:latest

Upvotes: 1

aykcandem

Reputation: 1135

Check your aws permissions. In addition to AmazonEC2ContainerRegistryFullAccess permission, below actions has to be granted for the correct resource. Especially check "arn:aws:ecr:${REGION}:${ACCOUNT_ID}:repository/{$REGISTRY_NAME}" part.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "ecr:BatchGetImage",
        "ecr:BatchCheckLayerAvailability",
        "ecr:CompleteLayerUpload",
        "ecr:DescribeImages",
        "ecr:DescribeRepositories",
        "ecr:GetDownloadUrlForLayer",
        "ecr:InitiateLayerUpload",
        "ecr:ListImages",
        "ecr:PutImage",
        "ecr:UploadLayerPart"
      ],
      "Resource": "arn:aws:ecr:${REGION}:${ACCOUNT_ID}:repository/{$REGISTRY_NAME}"
    },
    {
      "Effect": "Allow",
      "Action": "ecr:GetAuthorizationToken",
      "Resource": "*"
    }
  ]
}

Upvotes: 12

sonlexqt

Reputation: 6469

In my case, the repository I wanted to push to didn't exist (For example, I tried pushing to my-app/backend:latest but only the my-app/cms repository exists). So make sure your repository exists in the AWS ECR Console in the right region. The error returned from AWS CLI (EOF) didn't help at all.

Upvotes: 18

Muhammad Nur Hazrul

Reputation: 11

If anyone is still stuck with the issue. I would highly recommend watching this short vid https://www.youtube.com/watch?v=89ZeXaZEf80&ab_channel=IdenticalCloud

Here are the steps I took to fix the issue (if you prefer not to watch the video):

Create a new IAM user with "Access keys" checked
Under permissions, click on "attach existing policies directly" and choose "AmazonEC2ContainerRegistryFullAccess"
Download the CSV file
Run "AWS configure" on your terminal and pass in the credentials from the CSV file
Set the location to the location you created your ECR (mine was us-east-1)
Go to ECR and follow the steps to push the image

Upvotes: 1

Dylan Pierce

Reputation: 4668

I also was able to login to the registry, yet the pushing of the image would just timeout.

The solution for me was to add AmazonEC2ContainerRegistryFullAccess to my IAM user.

After adding that permission to my IAM user account, I could docker push to the ECS registry just fine.

Upvotes: 5

Alena Melnikova

Reputation: 1521

You will get the same behaviour if you forget to create ECR repo before pushing.

Use CloudTrail to get a clue what is wrong.

Upvotes: 116

Stefan Eilers

Reputation: 131

In my case it was related to MFA (Multi-Factor-Authentication). I had to create a session token. The docker login seemed to be successful, but pushing does not work.

The following script is doing all for you and creates a aws profile "mfa" used to login: get_mfa_credentials.py

After executing, you can login with:

aws ecr get-login-password --region <YOUR_REGION> --profile mfa | docker login --username AWS --password-stdin <Your_REPO>

I do not know who wrote it, but I'm very grateful to this guy.

And thanks to AWS for bad tools that do not help.

Upvotes: 13

Kiryl Anokhin

Reputation: 459

Also make sure that you have configured correct policy for your user — for example, AmazonEC2ContainerRegistryFullAccess.

Upvotes: 30

0.618

Reputation: 11

For me, I had to delete the stack and re-deploy the stack. Then, I was able to push the docker image to ECR.

Upvotes: 0

Jesse McMullen-Crummey

Reputation: 3855

I figured out my issue. I wasn't using the correct credentials. I had a personal AWS account as my default credentials and needed to add my work profile to my credentials.

EDIT
If you have multiple aws profiles, you can mention the profile name at the docker login as below (assuming you have done aws configure --profile someprofile at earlier day),

aws ecr get-login-password --region us-east-1 --profile someprofile | docker login ....

Upvotes: 172

Docker push to AWS ECR hangs immediately and times out

Answers (29)

Commands

Related Questions