CODEWITHSUNDEEP
pythonamazon-s3rasterio
sorryMike
sorryMike

Reputation: 789

Open raster using rasterio and s3

I'm storing rasters in Amazon S3 bucket and would like to open specific object using installed rasterio library in my Django docker container.

AWS_ACCESS_KEY_ID and AWS_ACCESS_KEY_ID are incluced in docker-compose.yml env variables.

Inside docker container and trying:

$ python manage.py shell
$ import rasterio
$ with rasterio.open(url_to_s3_object) as file:
$ ....print(file)

I'm receiving error:

Traceback (most recent call last):
  File "rasterio/_base.pyx", line 72, in rasterio._base.DatasetReader.start (rasterio/_base.c:2847)
  File "rasterio/_base.pyx", line 74, in rasterio._base.DatasetReader.start (rasterio/_base.c:2799)
  File "rasterio/_err.pyx", line 196, in rasterio._err.CPLErrors.check (rasterio/_err.c:1773)
  rasterio._err.CPLE_OpenFailed: {URL TO S3 OBJECT} does not exist in the file system, and is not recognised as a supported dataset name.

What is the solution for that?

In local machine everything works fine.

Upvotes: 4

Views: 2088

Answers (2)

Sotosoul
Sotosoul

Reputation: 1

If you don't want to involve GDAL, you can simply read the file using boto3, load it as in-memory BytesIO object, and then open it with rasterio:

import io
import boto3
import rasterio

session = boto3.Session()
client = session.client('s3')

obj = client.get_object(
   Bucket='your-bucket-name',
   Key='path/to/file.tif'
)

raw_data: bytes = obj['Body'].read()

with rasterio.open(io.BytesIO(raw_data)) as ds:
    band = ds.read(1)
    ....
    # do more stuff

Upvotes: 0

sorryMike
sorryMike

Reputation: 789

Found a solution for that. rasterio lib works correct with s3 under GDAL 2.1.0 at least - https://github.com/rasterio/rasterio/blob/master/docs/topics/vsi.rst#aws-s3 Upgrading GDAL solves the problem.

Upvotes: 3

Related Questions