CODEWITHSUNDEEP
jsonpython-3.xamazon-web-servicesamazon-s3boto3
ioquitteoo
ioquitteoo

Reputation: 55

Extract element from JSON file in S3 bucket using boto3

I have created an AWS Python Lambda which simulates data and sends it as messages to a relevant AWS IoT topic.

Instead of reading the Client_ID from the os.environ in the lambda I am wanting to pull them from the JSON file that I have stored in S3 using boto3

Upvotes: 0

Views: 1754

Answers (1)

Coin Graham
Coin Graham

Reputation: 1584

You should consider using "S3 Select" which allows you to query a file in S3 directly without having to download the file to the system. In boto3 it's called select_object_content. I built out a sample below from your info and the boto3 page.

response = client.select_object_content(
    Bucket='mybucketname',
    Key='simulated/config/IoT-sim-config.json',
    Expression="SELECT s.* FROM S3Object s WHERE s.client_id = 'Sim_1'", # You will need to fiddle with the quotes on the SQL here.
    ExpressionType='SQL',
    InputSerialization={
        'JSON': {
            'Type': 'DOCUMENT'
        }
    },
    OutputSerialization={
        'JSON': {
            'RecordDelimiter': ','
        }
    }
)

Upvotes: 1

Related Questions