CODEWITHSUNDEEP
jsonpython-3.xgoogle-cloud-functionsgoogle-cloud-storage
Simon Breton
Simon Breton

Reputation: 2876

Write and read json data cloud functions and cloud storage

I have a first cloud function where I'm writing the response from an API in cloud storage:

  api_response = service.accounts().api().execute()
  bucket_name = 'my_bucket'
  date = datetime.today().strftime('%Y-%m-%d %H:%M:%S')
  bucket = storage_client.bucket(bucket_name)
  destination_blob_name = 'stuff_here/{}'.format(date)
  blob = bucket.blob(destination_blob_name)
  blob.upload_from_string("{}".format(api_response))

When looking at my file in cloud storage I have something like this:

{'items': [
{'path': 'foo/12066407/stuff/12396182', 'accountId': '34858', 'containerId': '475663', 'name': 'blablabla', 'usageContext': ['stuff'], 'fingerprint': '1621433559139', 'Url': 'https://randomurl.com/#/randomPath/path/12066407/id?apiLink=blabla'}]}

I have only one object in the array here for the example. All the objects are consistent in the real use case

Then I'd like to be able to read this file from another cloud function.

I have the following code:

bucket = client.get_bucket('my_bucket')
blob = bucket.get_blob('filename')
content = blob.download_as_string().decode()
print(StringIO(content))
print(json.dumps(content))
my_df = pd.read_json(StringIO(content))

For the first print I have Content: <_io.StringIO object at 0x3ef722b287d0> . For the second print I have a replicate of what I have in my file and when trying to pass it in the df I have the following error ValueError: Expected object or value.

Ultimately I want to be able to extract one of the key-value pair.

Should I write the result of the API differently in the first function or how can I read it with the second function?

Upvotes: 0

Views: 1188

Answers (1)

guillaume blaquiere
guillaume blaquiere

Reputation: 75735

Many things.

First of all, StringIO return an object that you can read and write. So, if you print object, you have the memory reference in the output, it's totally fine.

Then, you have a string, that you want to parse in JSON. you have to use json.loads(content) and not the dumps (dumps is for writing a dict to string in a JSON format).

Finally, the JSON if your file (in Cloud Storage) seems weird: the key and value MUST have double quote and you have only simple quote in your sample. You can't load a broken JSON in your code, be sure that the API answer is a valid JSON before writing it to GCS.

Upvotes: 2

Related Questions