CODEWITHSUNDEEP
pythonamazon-web-servicesaws-lambda
addicted
addicted

Reputation: 3051

AWS Lambda - Python - reading csv file in S3-uploaded packaged zip function

I have a lambda function in Python 3.6 which I compressed into a zip file to upload to S3 (total function folder size is 180MB ,that's why). Inside the zip I have 1 csv file ('example.csv') which I want to read in the lambda handler function.

How to read this file? I tried:

filename = 'example.csv'
filepath = os.environ['LAMBDA_TASK_ROOT'] + '/' + filename
df = pd.read_csv(filepath, dtype=str)

# Failed with OSError: Initializing from file failed

Example of the content of my lambda function folder:

root:
 -- lambda_function.py
 -- example.csv
 -- bunch of library folders

Content of my csv file:

  id | value | something | else
-----------------------------------
  0  |  lol  |    ok     |  bye
  1  |  omg  |    foo    |  bar
  2  |  thx  |    baz    |  qux

What is the path to my csv file?

Upvotes: 0

Views: 2697

Answers (1)

Nitishkumar Singh
Nitishkumar Singh

Reputation: 1839

I am assuming you are using boto3, in the documentation there is download_file method available to download file in local. 

import boto3
import zipfile
def lambda_handler(event, context):
    s3 = boto3.resource('s3')
    s3.Bucket('<first_bucket>').download_file('<file_name>.zip', '/tmp/<file_name>.zip')

    # put unzip code
    zip_ref = zipfile.ZipFile('/tmp/<file_name>.zip', 'r')
    zip_ref.extractall(directory_to_extract_to)
    zip_ref.close()
    #handle CSV file reading and rest of operation

After the above code, you can put your csv handling code to read and perform required operation on it.

Upvotes: 1

Related Questions