CODEWITHSUNDEEP
pythoncsvamazon-s3
user13540211
user13540211

Reputation: 53

Read data from a CSV file in S3 bucket and store it in a dictionary in python

I am trying to read a csv file from S3 bucket and store its content into a dictionary. Sample csv file data. I want to use my first row as key and subsequent rows as value.

name,origin,dest
xxx,uk,france
yyyy,norway,finland
zzzz,denmark,canada

I am using the below code which is storing the entire row in a dictionary. But I want to loop through each row and store each field in a row as key value pair.

{'name':'xxx','origin':'uk','dest':'france'}

s3 = boto3.client('s3')
obj = s3.get_object(Bucket = 'bucket_name', Key = 'logs/log.csv')
lines = obj['Body'].read().decode("utf-8").replace("'", '"')
lines = lines.splitlines()
if (isinstance(lines, str)):
        lines = (lines)

docData = {}
for line in lines:
        docData['content'] = str(line)

print(docData)

Upvotes: 0

Views: 5939

Answers (1)

snakecharmerb
snakecharmerb

Reputation: 55599

Assuming that

s3 = boto3.client('s3')
obj = s3.get_object(Bucket = 'bucket_name', Key = 'logs/log.csv')
lines = obj['Body'].read().decode("utf-8")

produces the csv's contents as a string, then you can use the standard library's csv module to get a list of dicts.

import csv
import io

buf = io.StringIO(lines)
reader = csv.DictReader(buf)
rows = list(reader)

Upvotes: 3

Related Questions