CODEWITHSUNDEEP
pythongoogle-cloud-platformgoogle-cloud-storage
jack
jack

Reputation: 861

How to create a Json file in google storage from DataFrame?

I have the following code (Python 2.7):

import pandas as pd
....
pd.DataFrame(response2.json())['results'].to_json('orders.json', orient='records')

This saves a orders.json file in my local project. I want the file to be written to my Bucket on Google Storage and not to my local machine.

Is it possible to generate the file directly to the storage or must I create it locally and then upload it?

I know it should be something like:

from google.cloud import storage
storage_client = storage.Client()
bucket = storage_client.get_bucket("MyTestBucket")
blob = bucket.blob("folderName/" + blob?!)
blob.upload_from_filename("orders.json")  # Or something with writing the DataFrame?

Can someone help with that?

Upvotes: 1

Views: 1539

Answers (1)

Ryan Yuan
Ryan Yuan

Reputation: 2566

I found using datalab.storage is more convenient to write data to Google Cloud Storage.

import datalab.storage as storage
import pandas as pd

# Define sample dataframe to write
df = pd.DataFrame(data=[{1,'product1','description1'},{2,'product2','description2'}],columns=['id','name','description'])

# Define your bucket and write to GCS
storage.Bucket('{your-bucket-name}').item('{folder}/{your-file}.json').write_to(df.to_json(),'text/json')

Upvotes: 2

Related Questions