CODEWITHSUNDEEP
google-cloud-platformgoogle-cloud-storagegsutil
Chris Stryczynski
Chris Stryczynski

Reputation: 33861

How to retrieve the most recent file in cloud storage bucket?

Is this something that can be done with gsutil?

https://cloud.google.com/storage/docs/gsutil/commands/ls does not seem to mention any sorting functionality - only filtering by a date - which wouldn't work for my use case.

Upvotes: 18

Views: 25515

Answers (5)

Gajus
Gajus

Reputation: 73728

Most users will want to retrieve objects recursively, in which case you need to use -r and /** syntax, e.g.

gsutil ls -r -l 'gs://[bucket-name]/**' | sort -k 2

Upvotes: 0

Codemonkey
Codemonkey

Reputation: 4807

For my use case, I wanted to find the most recent directory in my bucket. I number them in ascending order (with leading zeros), so all I need to get the most recent one is this:

gsutil ls -l gs://[bucket-name] | sort | tail -n 1 | cut -d '/' -f 4
  1. list the directory
  2. sort alphabetically (probably unnecessary)
  3. take the last line
  4. tokenise it with "/" delimiter
  5. get the 4th token, which is the directory name

Upvotes: 0

Vladimir Djuricic
Vladimir Djuricic

Reputation: 4623

By using gsutil from a host machine this will populate the response array:

response=(`gsutil ls -l gs://some-bucket-name|sort -k 2|tail -2|head -1`)

Or by gsutil from docker container:

response=(`docker run --name some-container-name --rm --volumes-from gcloud-config -it google/cloud-sdk:latest gsutil ls -l gs://some-bucket-name|sort -k 2|tail -2|head -1`)

Afterwards, to get the whole response, run:

echo ${response[@]}

will print for example:

33 2021-08-11T09:24:55Z gs://some-bucket-name/filename-37.txt

Or to get separate info from the response, (e.g. filename)

echo ${response[2]}

will print the filename only

gs://some-bucket-name/filename-37.txt

Upvotes: 1

flowfleeflop
flowfleeflop

Reputation: 31

gsutil ls -l gs://<bucket-name> | sort -k 2 | tail -n 2 | head -1 | cut -d ' ' -f 7

It will not work well if there is less then two objects in the bucket though

Upvotes: 3

night-gold
night-gold

Reputation: 2421

Hello this still doesn't seems to exists, but there is a solution in this post: enter link description here

The command used is this one:

gsutil ls -l gs://[bucket-name]/ | sort -k 2

As it allow you to filter by date you can get the most recent result in the bucket and recuperating the last line using another pipe if you need.

Upvotes: 10

Related Questions