CODEWITHSUNDEEP
jsonformatgoogle-cloud-dataprep
hnandarusdy
hnandarusdy

Reputation: 452

import json file in google dataprep

is there anyone here know how to import json file in google data prep? I can upload a json files there, but it can't create columns.

here is the screenshot:enter image description here

as you can see it displays all json into 1 column.

I did some research and looks to me that JSON should be supported with additional steps

here is the article: https://cloud.google.com/dataprep/docs/html/Supported-File-Formats_57344528

and here is the additional step article: https://cloud.google.com/dataprep/docs/html/Splitrows-Transform_57344654.

I did try to split rows but it is still not working. Is there any particular format of JSON I have to use?

would be nice if you could give me a simple example of json file that works as expected.

thanks.

Upvotes: 0

Views: 1162

Answers (2)

terchris
terchris

Reputation: 595

I'm testing dataprep and I use a really simple json file. But dataprep just import is as a string. Reading the doc it says that "NOTE: Cloud Dataprep requires that JSON files be submitted with one valid JSON object per line."

This makes no sense. In order to use a tool for data preparation you will have to use another tool first to prepare the data.

https://cloud.google.com/dataprep/docs/html/Initial-Parsing-Steps_57344625

Upvotes: 1

hnandarusdy
hnandarusdy

Reputation: 452

Found the answer.

example data:

{"name":"jonSnow","age":21,"dob":"1988-01-01","tags":["a","b"],"address":{"postcode":3122,"state":"vic"}}
{"name":"aom","age":21,"dob":"1988-02-01","tags":["c","d"],"address":{"postcode":3000,"state":"Vic"}}

when it has been added as a dataset, these are the steps:(this has been by default suggested by dataprep)

  1. split rows.

    splitrows col: column1 on: '\n' quote: '\"' quoteEscapeChar: '\"'

  2. create column using un

    unnest col: column1 keys: 'name', 'age', 'dob', 'tags', 'address.postcode', 'address.state'

From what I understand, google data prep will receive your json as a string. So it really depends on how you import the dataset.

Upvotes: 0

Related Questions