CODEWITHSUNDEEP
amazon-web-servicesamazon-s3aws-cli
Alex B
Alex B

Reputation: 2375

How do find smallest file size in s3 bucket using AWS CLI?

I'm trying to get min/max of file size in S3 bucket and I always get 3 values instead of one. I think it's controlled by pagination but I'm unable to disable it.

aws s3api list-objects-v2 
--bucket my-bucket-dev 
--prefix subscription/2019/04/01/23 
--output text  
--query 'sort_by(Contents,&Size)[:1].Size'

Result:

618
616
620

How do i get 616 as result without the need to sort locally?

NOTE: I tried sort(), min(), max() max_by(), --no-pagination, --page-size 100000

they all give at least 3 records as result.

Upvotes: 0

Views: 8932

Answers (3)

ciurlaro
ciurlaro

Reputation: 1014

I know that OP explicitly said

[...] without the need to sort locally

But in case anyone was interested, I personally find the following more readable:

aws s3 ls s3://bucket/dir/ --recursive | sort -k 3 -n -r | (head -n 1 && tail -n 1)

PS:

aws s3 ls s3://bucket/dir/ --recursive --human-readable

to make the result more readable

Upvotes: 1

Alex B
Alex B

Reputation: 2375

once i put --output json instead of --output text it gave me min size.

[
    616
]

another good point about local --query from @Dunedan - it looks like internaly it retrieves all files and applies sort_by() locally (so I'm paying for it - there are no cost savings vs other methods)

Upvotes: 1

John Rotenstein
John Rotenstein

Reputation: 269340

aws s3api list-objects-v2 --bucket my-bucket --query 'sort_by(Contents,&Size)[0].Size'

The sort_by() sorts by size, so you just want the first element by using [0].

Upvotes: 3

Related Questions