CODEWITHSUNDEEP
dockerdocker-composedocker-registry
hello
hello

Reputation: 11

Problems calling Docker's /v2/{name}/manifest/{tag} API

I am using the docker registry api to complete the push of the docker image. I have completed the push of all the previous blobs, but I encountered a problem when pushing the manifest at the end. I hope to get help here.

I checked the container log through the docker logs command and found that the error message was: err.code="manifest invalid" err.detail="invalid format length" err.message="manifest invalid"

I haven’t found a solution so far. I will describe my development environment and pseudo code below.

My docker registry image is of amd architecture, the machine is a cloud server of x86 architecture, and I access it locally through code I get the mirror manifest file through GET /v2/{name}/manifests/{tag}, After traversing all the layers according to the layers field in the list and uploading, this step is completed successfully

After all blobs are uploaded,

I use PUT /v2/{name}/manifests/{tag} to upload the final image manifest file My request body comes from the manifest file that I got from the source warehouse at the beginning, and my request header Content-Type is also consistent with it, but it fails in the end.

How can I modify my code to make it work properly

The issue is synchronized on github https://github.com/distribution/distribution/issues/3503

shell:

#! /bin/bash

getManifestsUrl="http://localhost:5001/v2/registry/manifests/1"
putManifestsUrl="http://localhost:5002/v2/registry/manifests/1"

manifests=$(curl -s -X GET "$getManifestsUrl")

echo "$manifests"

result=$(curl -X PUT -H "Content-Type:application/vnd.docker.distribution.manifest.v2+json" -d '"$manifests"' "$putManifestsUrl") 
echo "$result"

java:

     String name = "registry";
        String tag = "1";

        // get /v2/{name}/manifests/{tag} impl
        String url = String.format(fromHost + "/v2/%s/manifests/%s", name, tag);
        HttpHeaders httpHeaders = new HttpHeaders();
        httpHeaders.put(HttpHeaders.CONTENT_TYPE,
                Arrays.asList("application/vnd.docker.distribution.manifest.v2+json"));
        HttpEntity<String> httpEntity = new HttpEntity<>(null, httpHeaders);
        ResponseEntity<String> resp = restTemplate.exchange(url, HttpMethod.GET, httpEntity, String.class);
        // print "[application/vnd.docker.distribution.manifest.v1+prettyjws]"
        System.out.println(resp.getHeaders().get(HttpHeaders.CONTENT_TYPE));
        String manifests = resp.getBody();

        // put /v2/{name}/manifests/{tag} impl
        httpHeaders.clear();
        httpHeaders.put(HttpHeaders.CONTENT_TYPE,
                Arrays.asList("application/vnd.docker.distribution.manifest.v2+json"));
        httpEntity = new HttpEntity<>(manifests, httpHeaders);
        resp = restTemplate.exchange(toHost + "/v2/registry/manifests/1",
                HttpMethod.PUT, httpEntity, String.class);
        System.out.println(resp);

Upvotes: 1

Views: 1622

Answers (1)

BMitch
BMitch

Reputation: 264396

The problem here has to do with content-type header. When you do a GET, you tell the registry which content-type values you are able to accept with an Accept header, and the registry responds with one of those as the Content-Type header in the response. When you have no Accept header, the registry falls back to a compatibility mode with very old docker versions that are expecting a schema version 1 manifest, which is why you saw application/vnd.docker.distribution.manifest.v1+prettyjws.

When you push a manifest in the schema v1 format with a different content-type header, the registry is rejecting it since the manifest doesn't match content-type.

The fix should be the following in the GET:

httpHeaders.put(HttpHeaders.ACCEPT,
                Arrays.asList("application/vnd.docker.distribution.manifest.v2+json",
                "application/vnd.docker.distribution.manifest.list.v2+json",
                "application/vnd.oci.image.manifest.v1+json",
                "application/vnd.oci.image.index.v1+json"));

And then in your PUT request, you should pass the same content-type that you've received:

httpHeaders.put(HttpHeaders.CONTENT_TYPE,
                Arrays.asList(resp.getHeaders().get(HttpHeaders.CONTENT_TYPE)));

This handles the three main image types you'll see on registries today: docker schema 2 manifest, the docker manifest list, and the same pair for OCI.

Note that if you are copying images across repositories (and not simply retagging an image or modifying a manifest in the same repository), you'll need to recursively copy any manifests within the manifest list or OCI index before pushing that parent manifest list/index. And similarly copy any blobs before copying the manifest itself.

I've also got implementations of manifest get and put in Go which may help working through the logic in my regclient project.

Upvotes: 2

Related Questions