CODEWITHSUNDEEP
pythonjson
Alexander Starbuck
Alexander Starbuck

Reputation: 1159

How to fetch JSON data from API, format / encode / write to a file?

I need to fetch some data from a weather API, extract certain info and send it to std. output (in my case this is the console/terminal; I am playing around with python API scripting and do not yet have a web site/app do show fetched data).

Example Python code from the API provider (simple to understand and works):

import urllib2
import json

API_KEY='mysuperawesomekey'


f = urllib2.urlopen('http://api.wunderground.com/api/' + API_KEY + '/geolookup/conditions/q/IA/Cedar_Rapids.json')

json_string = f.read()
parsed_json = json.loads(json_string)
location = parsed_json['location']['city']
temp_f = parsed_json['current_observation']['temp_f']

print "Current temperature in %s is: %s" % (location, temp_f)

f.close()

Since I am on a free plan, I don't want to use up my API requests but rather fetch the JSON data, save it to a file and use in another script to practice. I found a couple of solutions here on StackOverflow but neither seem to work completely (nice formatting of the file):

Attempt 1. to save the fetched data, add. to the orig. code above:

import io
import codecs

with open('data.json', 'w') as outfile:
json.dump(json_string, outfile, indent=4, sort_keys=True, separators=(",", ':'))

Attempt 2:

with codecs.open('data.json', 'w', 'utf8') as jasonfile:
  jasonfile.write(json.dumps(parsed_json, sort_keys = True, ensure_ascii=False))

Both of my attempts work ("kind of") as I do get a .json file. But, upon inspecting it in my editor (Atom) I am seeing this (first few soft-break lines):

Output:

"\n{\n  \"response\": {\n  \"version\":\"0.1\",\n\"termsofService\":\"http://www.wunderground.com/weather/api/d/terms.html\",\n  \"features\": {\n  \"geolookup\": 1\n  ,\n  \"conditions\": 1\n  }\n\t}\n\t\t,\t\"location\": {\n\t\t\"type\":\"CITY\",\n\t\t\"country\":\"US\",\n\t\t\"country_iso3166\":\"US\",\n\t\t\"country_name\":\"USA\",\n\t\t\"state\":\"IA\",\n\t\t\"city\":\"Cedar Rapids\",\n\t\t\"tz_short\":\"CDT\",\n\t\t\"tz_long\":\"America/Chicago\",\n\t\t\"lat\":\"41.97171021\",\n\t\t\"lon\":\"-91.65871429\",\n\t\t\"zip\":\...

It is all on one line, with newlines and tabs showing. I have a couple of questions:

  1. How can I check what format / kind of data-type API is returning? (FOr example is it just a raw string which needs to be JSON-iffied?)
  2. Is my JSON not written in a human readable format because of encoding, delimiters fault?
  3. Line json_string = f.read() from example code seems to "mimic" working with a "real" internal file object, is this correct?

Many thanks!

Upvotes: 2

Views: 2165

Answers (1)

Alexander Starbuck
Alexander Starbuck

Reputation: 1159

Seems that it was quite a simple solution. In my original code, I was saving a "non-parsed" variable to a file:

import urllib2
import json

API_KEY='key'

f = urllib2.urlopen('http://api.wunderground.com/api/' 
+ API_KEY + '/geolookup/conditions/q/IA/Cedar_Rapids.json')

# Saving the below variable into a .json file gives all the JSON data
# on a single line
json_string = f.read()

with open('data.json', 'w') as outfile:
    json.dump(json_string, outfile, indent=4, sort_keys=True, separators=(",", ':'))

which, of course, resulted in a file with JSON data on a single line. What I should have been doing is saving the parsed variable:

import urllib2
import json

API_KEY='key'

f = urllib2.urlopen('http://api.wunderground.com/api/' + API_KEY + '/geolookup/conditions/q/IA/Cedar_Rapids.json')

json_string = f.read()

# THIS one is correct!
parsed_json = json.loads(json_string)

with open('data.json', 'w') as outfile:
  json.dump(parsed_json, outfile, indent=4, sort_keys=True, separators=(",", ':'))

Upvotes: 1

Related Questions