CODEWITHSUNDEEP
pythonjson
erikvm
erikvm

Reputation: 928

How to replace None values in JSON response?

I'm fetching financial information from an api endpoint and when I get a 200 response through

r = requests.get(url)
data = r.json()

It'll return None for all null values. How do I convert all null/None values to 0? Since it's financial data, the JSON is usually quite massive (300k-400k lines, some with deep nested nulls) so I can't do a try/except block on each TypeError.

An extract of the json response looks something like this:

{'0': 
'Highlights': {'QuarterlyRevenueGrowthYOY': 0.671, 'GrossProfitTTM': 3750684, 'DilutedEpsTTM': 0.2, 'QuarterlyEarningsGrowthYOY': 0.95
5}, 'Valuation': {'TrailingPE': 60.75, 'ForwardPE': 0, 'PriceSalesTTM': 2.0817, 'PriceBookMRQ': 4.207, 'EnterpriseValueRevenue': 1.
806, 'EnterpriseValueEbitda': 0.0952}, 'Technicals': {'Beta': None, '52WeekHigh': 12.35, '52WeekLow': 7.84, '50DayMA': 11.0197, '20
0DayMA': 10.2209, 'SharesShort': 0, 'SharesShortPriorMonth': 0, 'ShortRatio': 0, 'ShortPercent': 0}, 'SplitsDividends': {'ForwardAn
nualDividendRate': 0.18, 'ForwardAnnualDividendYield': 0.0151, 'PayoutRatio': 0.9, 'DividendDate': '0000-00-00', 'ExDividendDate':
'2020-06-11', 'LastSplitFactor': '', 'LastSplitDate': '0000-00-00'}, 'Earnings': {'Last_0': {'date': '2020-06-30', 'epsActual': 0.1
9, 'epsEstimate': None, 'epsDifference': None, 'surprisePercent': None}, 'Last_1': {'date': '2019-12-31', 'epsActual': 1.86, 'epsEs
timate': None, 'epsDifference': None, 'surprisePercent': None}, 'Last_2': {'date': '2019-06-30', 'epsActual': -0.82, 'epsEstimate':
 None, 'epsDifference': None, 'surprisePercent': None}, 'Last_3': {'date': '0000-00-00', 'epsActual': 0, 'epsEstimate': 0, 'epsDiff
erence': 0, 'surprisePercent': 0}}, 'Financials': {'Balance_Sheet': {'currency_symbol': 'EUR', 'quarterly_last_0': {'date': '2020-0
6-30', 'filing_date': None, 'totalAssets': '12810000.00', 'intangibleAssets': '281000.00', 'otherCurrentAssets': '60000.00', 'total
Liab': '4225000.00', 'totalStockholderEquity': '8585000.00', 'deferredLongTermLiab': '74000.00', 'otherCurrentLiab': '1274000.00',
'commonStock': '80000.00', 'retainedEarnings': '311000.00', 'otherLiab': '200000.00', 'goodWill': '3381000.00', 'otherAssets': '730
00.00', 'cash': '4983000.00', 'totalCurrentLiabilities': '4025000.00', 'shortLongTermDebt': None,
... 
}

Yeah you get the point.. a ton of None all over the place. Any quick fixes for this?

Upvotes: 1

Views: 1277

Answers (2)

erikvm
erikvm

Reputation: 928

Found a way to do it, @Charles Duffy, thanks for the inspiration - borrowed some but couldn't get it quite to work. The final code looks like this if anyone would need it in the future

from collections.abc import Mapping, Iterable


def replace_none_values(noneVal, replaceVal='0.00'): # not sure if this is bad practice 
    if noneVal is None:
        return replaceVal
    if isinstance(noneVal, Mapping):
        return {k: replace_none_values(v, replaceVal) for k, v in noneVal.items()}
    elif not isinstance(noneVal, str) and isinstance(noneVal, Iterable):
        return [replace_none_values(v, replaceVal) for v in noneVal]
    return noneVal

Upvotes: 0

Charles Duffy
Charles Duffy

Reputation: 295308

def recursive_replace(obj, findVal, replaceVal):
  for k, v in obj.items():
    if v == findVal:
      obj[k] = replaceVal
    elif isinstance(v, dict):
      obj[k] = recursive_replace(obj[k], findVal, replaceVal)
  return obj

result = recursive_replace(json.loads(yourdata), None, 0)

Upvotes: 1

Related Questions