CODEWITHSUNDEEP
amazon-web-servicesamazon-s3aws-lambda
user190549
user190549

Reputation: 447

Reading file line by line from S3 on Lambda Trigger

I am trying to read a file line by line from S3 on Lambda Trigger. Below is my code

import boto3


def lambda_handler(event, context):
"""Read file from s3 on trigger."""
s3 = boto3.client("s3")
if event:
    file_obj = event["Records"][0]
    bucketname = str(file_obj['s3']['bucket']['name'])
    filename = str(file_obj['s3']['object']['key'])
    print("Filename: ", filename)
    fileObj = s3.get_object(Bucket="aws-lambda-hive-trigger", Key=filename)
    print(fileObj)
    file_content = fileObj["Body"].read().split(b'\n')
    for lines in file_content:
        print(lines)
return 'Thanks'

I am not getting any error but when i am checking on cloudwatch logs , no content is getting printed. How should i obtain the file content. Below is text of my file:-

    0000701869-18-00001910-K      INDEPENDENCE HOLDING CO                                                                                                 2018032620180326155547155547155547   0                                                                 <SUBMISSION>
<ACCESSION-NUMBER>0000701869-18-000019
<TYPE>10-K
<PUBLIC-DOCUMENT-COUNT>195
<PERIOD>20171231
<FILING-DATE>20180326
<DATE-OF-FILING-DATE-CHANGE>20180326
<FILER>
<COMPANY-DATA>
<CONFORMED-NAME>INDEPENDENCE HOLDING CO
<CIK>0000701869
<ASSIGNED-SIC>6311
<IRS-NUMBER>581407235
<STATE-OF-INCORPORATION>DE
<FISCAL-YEAR-END>1231
</COMPANY-DATA>
<FILING-VALUES>
<FORM-TYPE>10-K
<ACT>34
<FILE-NUMBER>001-32244
<FILM-NUMBER>18712741
</FILING-VALUES>
<BUSINESS-ADDRESS>
<STREET1>96 CUMMINGS POINT RD
<CITY>STAMFORD
<STATE>CT
<ZIP>06902
<PHONE>2033588000
</BUSINESS-ADDRESS>
<MAIL-ADDRESS>
<STREET1>96 CUMMINGS POINT ROAD
<CITY>STAMFORD
<STATE>CT
<ZIP>06902
</MAIL-ADDRESS>
<FORMER-COMPANY>
<FORMER-CONFORMED-NAME>STAMFORD CAPITAL GROUP INC
<DATE-CHANGED>19910925
</FORMER-COMPANY>
<FORMER-COMPANY>
<FORMER-CONFORMED-NAME>INDEPENDENCE HOLDING CO/
<DATE-CHANGED>19871115
</FORMER-COMPANY>
</FILER>
<DOCUMENT>
<TYPE>10-K
<SEQUENCE>1
<FILENAME>ihc-20171231.htm
<DESCRIPTION>INDEPENDENCE HOLDING COMPANY - FORM 10-K SEC FILING
<TEXT>
<XBRL>
<?xml version='1.0' encoding='us-ascii' standalone='no'?>
<html xmlns='http://www.w3.org/1999/xhtml' xmlns:xbrli='http://www.xbrl.org/2003/instance' xmlns:us-gaap='http://fasb.org/us-gaap/2017-01-31' xmlns:nonnum='http://www.xbrl.org/dtr/type/non-numeric' xmlns:num='http://www.xbrl.org/dtr/type/numeric' xmlns:dei='http://xbrl.sec.gov/dei/2014-01-31' xmlns:fil='http://Ihcgroup.com/20171231' xmlns:i='http://www.xbrl.org/2003/instance' xmlns:xbrldi='http://xbrl.org/2006/xbrldi' xmlns:iso4217='http://www.xbrl.org/2003/iso4217' xmlns:i

    x='http://www.xbrl.org/2013/inlineXBRL' xmlns:ixt='http://www.xbrl.org/inlineXBRL/transformation/2015-02-26' xmlns:ixt-sec='http://www.sec.gov/inlineXBRL/transformation/2015-08-31' xmlns:utr='http://www.xbrl.org/2009/utr' xmlns:link='http://www.xbrl.org/2003/linkbase' xmlns:xlink='http://www.w3.org/1999/xlink' xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance'>
    <head>
    <!-- Produced by EDGARsuite software, Advanced Computer Innovations, Inc., Copyright (C) 2008-2018 [PPXCKFGGAJGLVL0HCGXZ]. www.edgarsuite.com -->
    <title>INDEPENDENCE HOLDING COMPANY - Form 10-K SEC filing</title>
    <meta content='text/html; charset=UTF-8' http-equiv='Content-Type' />
    </head>
    <body>

This a sample of a file, my file size is 28 mb, I am not getting any error but when i am checking on cloudwatch logs , no content is getting printed for large files How should i obtain the file content. I am getting file content printed in cloudwatch for smaller files but when i upload 28 mb actual raw file i am not able to see any filecontent getting printed in cloudwatch and it says timeout in 3 sec.

Upvotes: 0

Views: 692

Answers (1)

alexis-donoghue
alexis-donoghue

Reputation: 3387

3s is too little of a timeout for this to work. Transferring 28 Mb will take more than that, assuming you have given function minimum memory.

Increase timeout to 30s and try again. You can monitor time it takes for function to process via Cloudwatch, so you can increase or lower timeout as needed.

Upvotes: 1

Related Questions