CODEWITHSUNDEEP
google-cloud-platformgoogle-cloud-functionsgoogle-cloud-buildgoogle-artifact-registry
Sertage
Sertage

Reputation: 3263

How to remove an image from Artifact Registry automatically

Using gcloud I can list and remove the images I want through those commands:

gcloud artifacts docker images list LOCATION/PROJECT-ID/RESPOSITORY-ID/IMAGE \
  --include-tags --filter="tags:IPLA*" --filter="create_time>2022-04-20T00:00:00"

and then

gcloud artifacts docker images delete LOCATION/PROJECT-ID/RESPOSITORY-ID/IMAGE:tag

I am trying to automate that so I can filter by tag name and date and run every day or week.

I've tried to use inside a cloud function, but I don't think that is allowed.

  const { spawn } = require("child_process");
  const listening = spawn('gcloud', ['artifacts', 'docker', 'images', 'list', 
     'LOCATION/PROJECT-ID/RESPOSITORY-ID/IMAGE',
     '--include-tags', 
     '--filter="tags:IPLA*"', 
     '--filter="create_time>2022-04-20T00:00:00"'
  ]);

  listening.stdout.on("data", data => {
      console.log(`stdout: ${data}`);
  });

  listening.stderr.on("data", data => {
      console.log(`stderr: ${data}`);
  });

  listening.on('error', (error) => {
      console.log(`error: ${error.message}`);
  });

I get this error when running the cloud function:

error: spawn gcloud ENOENT

I accept any other solution like trigger on cloud build, terraform as long is it can live on google cloud.

Upvotes: 3

Views: 4357

Answers (4)

Florian Ludewig
Florian Ludewig

Reputation: 6002

In my case, I wanted to delete all image but keep the x latest images, regardless of their creation time.

Calling the below script with ./script.sh location-docker.pkg.dev/project/repo/image will delete all images in location-docker.pkg.dev/project/repo/image except the three latest ones.

#!/bin/bash

IMAGES_TO_KEEP=3
IMAGE=$1

# Get shas of all images
DIGESTLIST=$(gcloud artifacts docker images list $IMAGE --include-tags --sort-by CREATE_TIME --format='value(DIGEST)')

# Create list from string result
IMAGE_LIST=()
for DIGEST in $DIGESTLIST; do
  IMAGE_LIST+=("${IMAGE}@${DIGEST}")
done

IMAGE_LIST_LENGTH=${#IMAGE_LIST[@]}
echo "Found ${IMAGE_LIST_LENGTH} images"

# Delete images and keep x number of images
echo "Starting to delete images"
for i in "${!IMAGE_LIST[@]}"; do
  if [ $i -ge $((IMAGE_LIST_LENGTH-IMAGES_TO_KEEP)) ]; then
    echo "Stop deleting images"
    break
  fi

  echo "Deleting image: ${IMAGE_LIST[$i]}"
  gcloud artifacts docker images delete "${IMAGE_LIST[$i]}" --quiet
done

Upvotes: 1

Beknazar Saitov
Beknazar Saitov

Reputation: 11

##!/usr/bin/env bash

_cleanup() {
  image_path="$2-docker.pkg.dev/$project_id/$1"
  echo "Starting to filter: $image_path"
  images=$(gcloud artifacts docker images list $image_path \
          --filter="UPDATE_TIME.date('%Y-%m-%d', Z)<=$(date --date="-1 years" +'%Y-%m-%d')" \
          --format='value(IMAGE)')
  if [ -z "$images" ]; then
    echo "No images to clean"
  else
    echo "Images found: $images"
    for each in $images; do
      echo "Deleting image: $image_path:$each"
      gcloud artifacts docker images delete "$images" --quiet
    done
  fi
}

project_id=$1
gcloud artifacts repositories list --format="value(REPOSITORY,LOCATION)" --project=$project_id | tee -a repo.txt

while read p; do
    sentence=$p
    stringarray=($sentence)
    _cleanup  ${stringarray[0]}  ${stringarray[1]}
done < repo.txt

echo
echo "DONE"
rm -rf repo.txt
echo "Deleteing repo.txt file"

Upvotes: 1

Sertage
Sertage

Reputation: 3263

Initially I started to look in the solution suggested by Guillaume, though it looked too overkill deploying a whole image just to clean the Artifact Registry. Ended up finding a lighter approach.

I create a shell script file to clean the images with the filters I wanted:

#!/usr/bin/env bash

_cleanup() {
  image_path="$location-docker.pkg.dev/$project_id/$repository_id/$image_name"
  echo "Starting to filter: $image_path"
  tags=$(gcloud artifacts docker images list $image_path \
    --include-tags \
    --filter="tags:IPLA* AND UPDATE_TIME.date('%Y-%m-%d', Z)<=$(date --date="-$older_than_days days" +'%Y-%m-%d')" \
    --format='value(TAGS)')
  if [ -z "$tags" ]; then
    echo "No images to clean"
  else
    echo "Images found: $tags"
    for tag in $tags; do
      echo "Deleting image: $image_path:$tag"
      gcloud artifacts docker images delete "$image_path:$tag" --quiet
    done
  fi
}
location=$1
project_id=$2
repository_id=$3
image_name=$4 #In this case I just want to clean the old branchs for same image
older_than_days=$5 #7 - Number of days in the repository
_cleanup

echo
echo "DONE"

Then I created a scheduled trigger on Cloud Build for the following cloudbuild.yaml file:

steps:

  - name: 'gcr.io/cloud-builders/gcloud'
    id: Clean up older versions
    entrypoint: 'bash'
    args: [ 'cleanup-old-images.sh', '$_LOCATION', '$PROJECT_ID','$_REPOSITORY_ID', '$_IMAGE_NAME', '$_OLDER_THAN_DAYS' ]

timeout: 1200s

Upvotes: 1

guillaume blaquiere
guillaume blaquiere

Reputation: 75990

You use Cloud Functions, a serverless product where you deploy your code that run somewhere, on something that you don't manage.

Here, in your code, you assume that gcloud is installed in the runtime. It's a mistake, you can't perform that assumption (that is wrong!)

However, you can use another serverless product where you manage your runtime environemnt: Cloud Run. The principle is to create your container (and therefore install what you want in it) and then deploy it. That time you can use gcloud command, because you know it exists on the VM.

However, it's not the right option. You have 2 better things

  • First of all, use something already done for you by a Google Cloud Developer Advocate (Seth Vargo). It's named GCR cleaner and remove images older than something
  • Or you can use directly the API to perform the exact same operation than GCLOUD bur without gcloud, by invoking the Artifact registry REST API. If you want to cheat and go faster, you can use the gcloud command with the --log-http parameter to display all the API call performed by the CLI. Copy the URL and parameters, and enjoy!!

Upvotes: 4

Related Questions