CODEWITHSUNDEEP
pythonamazon-s3
user3619789
user3619789

Reputation: 95

Read a JSON from S3

I am new to python and following the answers from an existing_post I am trying to read a json file from amazon S3 like this:

  import boto3
  import os
    BUCKET = 'my_bucket'
    FILE_TO_READ = 'file.json'
    client = boto3.client('s3',
                           aws_access_key_id=os.environ.my_key,
                           aws_secret_access_key=os.environ.my_secret_key
                         )
    result = client.get_object(Bucket=BUCKET, Key='file') 
    text = result["Body"].read().decode()
    print(text['key']) # Use your desired JSON Key for your value

However I am receiving the following error:

NoSuchKey: An error occurred (NoSuchKey) when calling the GetObject operation: The specified key does not exist.

Can someone please tell me what am I doing wrong?

Upvotes: 2

Views: 7646

Answers (2)

chris_b
chris_b

Reputation: 138

The error is saying that that your key does not exist. I see that you already have the key "file.json" so I assume it exists. I would assume that you have your filed stored like this bucket/same_file/file.json . If this is the case try this:

import json
import boto3
s3_obj =boto3.client('s3')
s3_clientobj = s3_obj.get_object(Bucket='your_bucket', Key='file/file.json')
s3_clientdata = s3_clientobj['Body'].read().decode('utf-8')
print("printing s3_clientdata")
print(s3_clientdata)

If you want to do data manipualation, a more pythonic soution would be:

fs = s3fs.S3FileSystem()
with fs.open('yourbucket/file/your_json_file.json', 'rb') as f:
    s3_clientdata = json.load(f)
#Convert to df
df = pd.DataFrame(s3_clientdata,index=[0])

Upvotes: 5

Caliuf
Caliuf

Reputation: 63

I see you are retrieving the object with key "file" in

result = client.get_object(Bucket=BUCKET, Key='file')

I think you where meaning something like:

result = client.get_object(Bucket=BUCKET, Key=FILE_TO_READ)

See the get_object reference for further details

Upvotes: 0

Related Questions