CODEWITHSUNDEEP
pythonjsoncsv
Arun Karthick
Arun Karthick

Reputation: 336

Trying to convert a big tsv file to json

I've a tsv file, which I need to convert it into a json file. I'm using this python script which is exporting a empty json file. 

import json
data={}
with open('data.json', 'w') as outfile,open("data.tsv","r") as f:
for line in f:
   sp=line.split()
   data.setdefault("data",[])
json.dump(data, outfile)

Upvotes: 0

Views: 2727

Answers (2)

giotskhada
giotskhada

Reputation: 2452

You never use the sp in your code.

To properly convert the tsv, you should read the first line separately, to get the "column names", then read the following lines and populate a list of dictionaries.

Here's what your code should look like:

import json
data=[{}]
with open('data.json', 'w') as outfile, open("data.tsv","r") as f:
firstline = f.readline()
columns = firstline.split()
lines = f.readlines()[1:]
for line in lines:
    values = line.split()
    entry = dict(zip(columns, values))
    data.append(entry)
json.dump(data, outfile)

This will output a file containing a list of tsv rows as objects.

Upvotes: 1

Nusrath
Nusrath

Reputation: 527

This can be done by pandas , but am not sure about performance

df.to_json

 df = pd.read_csv('data.tsv',sep='\t') # read your tsv file 
 df.to_json('data.json') #save it as json . refer orient='values' or 'columns' as per your requirements

Upvotes: 2

Related Questions