CODEWITHSUNDEEP
powershellamazon-web-servicesamazon-s3aws-powershell
Sam Gomari
Sam Gomari

Reputation: 765

Error in PowerShell due to copying the content of an S3 bucket

I copy the content of an S3 bucket to a local directory, however I get an error output from the powershell.

Copy-S3Object : The requested range is not satisfiable

It is pointing to this command: 

Copy-S3Object -BucketName $bucket -Key $object.Key -LocalFile $localFilePath -Region $region

Why do I get this error ? Note that the desired files that are needed to be copied do indeed get copied locally.

Upvotes: 2

Views: 7547

Answers (3)

Asdfg
Asdfg

Reputation: 12253

This happened to me when I tried to download the file which had more than one dots in it. Simplifying the file name, fixed the error.

File name that gave me error: myfile-18.10.exe
File name that worked: myfile-1810.exe

Upvotes: 0

Kelly Davis
Kelly Davis

Reputation: 364

this seems to occur with folders that do not contain files since copy wants to copy files. i accepted this error and trapped it.

catch  [Amazon.S3.AmazonS3Exception]
{

  # get error record
  [Management.Automation.ErrorRecord]$e = $_

  # retrieve information about runtime error
  $info = [PSCustomObject]@{
    Exception = $e.Exception.Message
    Reason    = $e.CategoryInfo.Reason
    Target    = $e.CategoryInfo.TargetName
    Script    = $e.InvocationInfo.ScriptName
    Line      = $e.InvocationInfo.ScriptLineNumber
    Column    = $e.InvocationInfo.OffsetInLine
    ErrorCode = $e.Exception.ErrorCode
  }
  if ($info.ErrorCode="InvalidRange") { #do nothing 
  } Else {
  # output information. Post-process collected info, and log info (optional)
  write-host $info -ForegroundColor Red}
}

  }

Upvotes: 3

Steve Roberts
Steve Roberts

Reputation: 734

I can't say why you are getting that error returned from S3, but I can tell you that if you are copying multiple objects you probably want to use the -LocalFolder parameter, not -LocalFile. -LocalFolder will preserve the prefixes as subpaths.

When downloading one or more objects from S3, the Read-S3Object cmdlet works the same as Copy-S3Object, but uses -KeyPrefix to specify the common prefix the objects share, and -Folder to indicate the folder they should be downloaded to.

This also reminds me I need to check why we used -LocalFolder on Copy-, and -Folder on Read- although I suspect aliases may also be available to make them consistent.

HTH

(Edit): I spent some time this morning reviewing the cmdlet code and it doesn't appear to me the cmdlet would work as-is on a multi-object download, even though it has a -LocalFolder parameter. If you have a single object to download, then using -Key/-LocalFile is the correct parameter combination. If -LocalFolder is passed, the cmdlet sets up internally to do a single file download instead of treating -Key as a common key prefix to a set of objects. So, I think we have a bug here that I'm looking into.

In the meantime, I would use Read-S3Object to do your downloads. It supports both single (-Key) or multi-object download (-KeyPrefix) modes. https://docs.aws.amazon.com/powershell/latest/reference/index.html?page=Read-S3Object.html&tocid=Read-S3Object

Upvotes: 3

Related Questions