CODEWITHSUNDEEP
python-3.xamazon-web-servicesamazon-s3boto3
smgeneralist
smgeneralist

Reputation: 105

Saving HTML in memory to S3 AWS Python Boto3

import boto3
from io import StringIO
s3 = boto3.client('s3')

display = # Altair Charting

buff = StringIO(display.save(str_obj, 'html'))

s3.upload_fileobj(buff.read(), BUCKET, 'index.html')

I think I full example would complicate the issue, so I left Altair charting commented out.

Anyway, I've tried to implement the vast amount suggestions for saving objects and files to AWS S3 buckets, but I'm not understanding.

Saving to disk is as simple as:

display.save('index.html')

Getting this onto S3 seems extremely difficult comparison. I'm not sure what I am missing here. Perhaps the upload_fileobj is not the correct method, but I've gone around and around trying to make this work.

The specific error with this method is ValueError: Fileobj must implement read

UPDATE:

buff = StringIO(display.save('str.html'))

s3.put_object(
   Bucket=BUCKET, 
   Key=f'{DASHBOARD}{mkt_type}/{symbol}/index.html',
   Body=buff.read()
   )

Results in the 0 Byte file index.html on my Bucket

UPDATE 2:

str_obj = StringIO()
display.save(str_obj, 'html')
buff = str_obj.read()

s3.put_object(
    Bucket=BUCKET, 
    Key=f'{DASHBOARD}{mkt_type}/{symbol}/index.html',
    Body=buff
    )

This also doesn't work. I just can't believe saving a file to S3 is this complicated. Hindsight note: I didn't have the getvalue() method required for buff

SOLUTION: This is not the first time I've struggled with S3 files, so I'm probably leaving this for my own future reference. That said, it is still not clear to me why I couldn't save the '.html' file in string form.

import boto3
from io import StringIO
s3 = boto3.client('s3')

display = # Altair Charting

str_obj = StringIO() # instantiate in-memory string object
display.save(str_obj, 'html') # saving to memory string object
buf = str_obj.getvalue().encode() # convert in-memory string to bytes

# Upload as bytes
s3.put_object(
    Bucket=BUCKET, 
    Key=f'{DASHBOARD}{mkt_type}/{symbol}/index.html', 
    Body=buf
    )

Upvotes: 5

Views: 5117

Answers (1)

Samuel
Samuel

Reputation: 3801

From boto3 docs: put_object

Body=b'bytes'|file,

Which means that Body should be file handle or byte string. So there are (at least) 2 possible ways to upload:

By passing file handle to Body:

with open('index.hml', 'rb') as f:
    s3.put_object(Bucket=BUCKET, Key=f'{DASHBOARD}{mkt_type}/{symbol}/index.html', Body=f)

By passing bytestring to Body (supposing display.save() return string):

buf = display.save('str.html').encode() # converting str to bytes
s3.put_object(Bucket=BUCKET, Key=f'{DASHBOARD}{mkt_type}/{symbol}/index.html', Body=buf)

Upvotes: 3

Related Questions