CODEWITHSUNDEEP
amazon-web-servicespowershellamazon-s3foreach
ericOnline
ericOnline

Reputation: 2008

Native PowerShell ForEach loop to delete files in S3 bucket

I'm trying to put together a simple PowerShell script to delete files from an AWS S3 bucket. I'm unable to split the filename correctly from items in the loop.

How is the filename correctly selected?

Example record names:

2021-06-07 16:08:15    1876349 20210502210533.csv
2021-06-07 16:44:53    1858461 210502210533.csv
2021-06-07 16:18:39  276597534 20210424203918.csv

Example script:

$all_files=aws s3 ls s3://bucket.host.com/folder1/folder2/folder3/folder4/ --profile dev

foreach($file in $all_files)
{
    aws s3 rm s3://bucket.host.com/folder1/folder2/folder3/folder4/$file.Split(' ')[6] --profile dev
}

Results:

delete: s3 ls s3://bucket.host.com/folder1/folder2/folder3/folder4/2021-06-07 16:44:53    1858461 20210502210533.csv.Split
delete: s3 ls s3://bucket.host.com/folder1/folder2/folder3/folder4/2021-06-07 16:18:39  276597534 20210424203918.csv.Split
delete: s3 ls s3://bucket.host.com/folder1/folder2/folder3/folder4/2021-06-07 15:50:41  276597534 20210424204122.csv.Split

Notice how the entire record is present (no split occurring). How do I reliably split this?

EDIT 1:

$all_files=aws s3 ls s3://bucket.host.com/folder1/folder2/folder3/folder4/ --profile dev

PS C:\Users\me> $all_files
2021-06-08 02:50:37 4637885036 20210425202931.csv
2021-06-08 02:53:23 4753217891 20210426204043P.csv
2021-06-08 02:59:10 4838159267 20210426204346.csv
2021-06-08 02:58:07 4871146830 20210426204407.csv
2021-06-08 03:00:24 4641073848 20210427203146.csv
2021-06-08 02:52:29 4633473584 20210427203836.csv
2021-06-08 02:57:55 4633473584 20210427204657.csv
2021-06-08 02:56:25 4633473584 20210428203618.csv
2021-06-08 02:53:30 4633473584 20210429204253.csv

PS C:\Users\me> ForEach($file in $all_files){$command = 'aws s3 rm s3://bucket.host.com/folder1/folder2/folder3/folder4/' + $file.Split(' ')[-1] + ' --profile dev' & $command}

At line:1 char:177
+ ... folder3/folder4/' + $file.Split(' ')[-1] + ' --profile vis_dev' & $comman ...
+                                                                 ~
Unexpected token '&' in expression or statement.
    + CategoryInfo          : ParserError: (:) [], ParentContainsErrorRecordException
    + FullyQualifiedErrorId : UnexpectedToken

If I leave the & $command off the end, then run Invoke-Expression $command as a second expression, the last file is deleted.

How is the & $command run as part of the loop?

Upvotes: 0

Views: 1176

Answers (1)

Steven
Steven

Reputation: 7087

Try this:

$All_Files = @(
    '2021-06-07 16:08:15    1876349 20210502210533.csv'
    '2021-06-07 16:44:53    1858461 210502210533.csv'
    '2021-06-07 16:18:39  276597534 20210424203918.csv'
)

ForEach($File in $All_Files)
{
    'aws s3 rm s3://bucket.host.com/folder1/folder2/folder3/folder4/' + $file.Split(' ')[-1]
}

If you just use normal string concatenation this should output what you're looking for.

If you are going to split using the string .Split() method you're reliance on the index [6] is problematic. [-1] will simply give you the last element in the resulting array.

If you are looking to execute the resulting string as a command I would assign it to a variable and include other required arguments, then use the call operator (&) to execute it.

ForEach($File in $All_Files)
{
    $command = 'aws s3 rm s3://bucket.host.com/folder1/folder2/folder3/folder4/' + $file.Split(' ')[-1] + ' --profile dev'
    & $command
}

Upvotes: 1

Related Questions