Reputation: 9
Python 2.7 Dict is overwriting itself
Hey guys my dict phistory is overwriting it self each time this function is called I expect it to add a new entry to the dict.
def pHistory(): #Creates/loads a Dict of historical prices then saves it after the update phase 2
global phistory
global dataentries
phistory = {}
dataentries = 0
print phistory
print dataentries
def loaddata():
global dataentriesa
with open('StockDb.FUMatt','r') as y:
vhistory = y.read()
print vhistory
phistory = vhistory
print phistory
with open('count.FUMatt','r') as ya:
strdataa = ya.read()
print strdataa
dataentriesa = int(strdataa)
print dataentriesa
def save(): #function to save database to file
with open('StockDb.FUMatt','w') as z:
savith = json.dumps(phistory)
z.write(savith)
print savith
with open('count.FUMatt','w') as j:
strdata = str(dataentries)
j.write(strdata)
print strdata
loaddata()
dataentries = dataentriesa
a = stockdf.keys()
print dataentries
print phistory
for b in a:
name = stockdf[b]['name']
price = stockdf[b]['price']
time = stockdf[b]['time']
historyupdate = { b: {dataentries: {time:price}}}
phistory.update(historyupdate)
c = phistory.keys()
#print c
dataentries = dataentries + 1
save()
when this code is run the count is working fine for log entries, how ever lets say when dataentrie 2 is added to the dict all of the info from data entrie 1 is overwritten. am I overlooking something?
the json format is
{
"EVL": {
"35": {
"2018-08-13 14:00:01": "440.716"
}
},
"TCHS": {
"35": {
"2018-08-13 14:00:01": "994.678"
}
},
"FHG": {
"35": {
"2018-08-13 14:00:01": "404.863"
}
},
"WLT": {
"35": {
"2018-08-13 14:00:01": "988.793"
}
},
"BAG": {
"35": {
"2018-08-13 14:00:01": "2346.422"
}
},
"MSG": {
"35": {
"2018-08-13 14:00:01": "305.658"
}
},
"TMI": {
"35": {
"2018-08-13 14:00:01": "1021.110"
}
},
"TGP": {
"35": {
"2018-08-13 14:00:01": "195.230"
}
},
"IOU": {
"35": {
"2018-08-13 14:00:01": "535.950"
}
},
"LSC": {
"35": {
"2018-08-13 14:00:01": "989.317"
}
},
"TSBC": {
"35": {
"2018-08-13 14:00:01": "496.127"
}
},
"PRN": {
"35": {
"2018-08-13 14:00:01": "851.644"
}
},
"TCP": {
"35": {
"2018-08-13 14:00:01": "242.966"
}
},
"TCM": {
"35": {
"2018-08-13 14:00:01": "553.274"
}
},
"WSSB": {
"35": {
"2018-08-13 14:00:01": "65.845"
}
},
"IIL": {
"35": {
"2018-08-13 14:00:01": "85.831"
}
},
"TCB": {
"35": {
"2018-08-13 14:00:01": "428.470"
}
},
"TCC": {
"35": {
"2018-08-13 14:00:01": "550.622"
}
},
"EWM": {
"35": {
"2018-08-13 14:00:01": "871.508"
}
},
"YAZ": {
"35": {
"2018-08-13 14:00:01": "51.068"
}
},
"TCSE": {
"35": {
"2018-08-13 14:00:01": "19067.656"
}
},
"CNC": {
"35": {
"2018-08-13 14:00:01": "642.367"
}
},
"GRN": {
"35": {
"2018-08-13 14:00:01": "267.374"
}
},
"SLAG": {
"35": {
"2018-08-13 14:00:01": "234.068"
}
},
"HRG": {
"35": {
"2018-08-13 14:00:01": "426.689"
}
},
"ISTC": {
"35": {
"2018-08-13 14:00:01": "1010.354"
}
},
"SYM": {
"35": {
"2018-08-13 14:00:01": "1184.627"
}
},
"SYS": {
"35": {
"2018-08-13 14:00:01": "660.124"
}
},
"ELBT": {
"35": {
"2018-08-13 14:00:01": "125.758"
}
},
"MCS": {
"35": {
"2018-08-13 14:00:01": "1720.868"
}
},
"TCT": {
"35": {
"2018-08-13 14:00:01": "375.333"
}
}
} {
"EVL": {
"36": {
"2018-08-13 14:15:01": "440.425"
}
},
"TCHS": {
"36": {
"2018-08-13 14:15:01": "994.438"
}
},
"FHG": {
"36": {
"2018-08-13 14:15:01": "404.876"
}
},
"WLT": {
"36": {
"2018-08-13 14:15:01": "988.940"
}
},
"BAG": {
"36": {
"2018-08-13 14:15:01": "2348.502"
}
},
"TCT": {
"36": {
"2018-08-13 14:15:01": "375.416"
}
},
"MSG": {
"36": {
"2018-08-13 14:15:01": "305.709"
}
},
"TMI": {
"36": {
"2018-08-13 14:15:01": "1021.672"
}
},
"TGP": {
"36": {
"2018-08-13 14:15:01": "195.230"
}
},
"LSC": {
"36": {
"2018-08-13 14:15:01": "989.401"
}
},
"TSBC": {
"36": {
"2018-08-13 14:15:01": "496.085"
}
},
"PRN": {
"36": {
"2018-08-13 14:15:01": "851.735"
}
},
"TCP": {
"36": {
"2018-08-13 14:15:01": "242.994"
}
},
"TCM": {
"36": {
"2018-08-13 14:15:01": "552.139"
}
},
"WSSB": {
"36": {
"2018-08-13 14:15:01": "65.845"
}
},
"IIL": {
"36": {
"2018-08-13 14:15:01": "85.833"
}
},
"TCB": {
"36": {
"2018-08-13 14:15:01": "428.538"
}
},
"TCC": {
"36": {
"2018-08-13 14:15:01": "550.983"
}
},
"EWM": {
"36": {
"2018-08-13 14:15:01": "871.608"
}
},
"YAZ": {
"36": {
"2018-08-13 14:15:01": "51.065"
}
},
"TCSE": {
"36": {
"2018-08-13 14:15:01": "19069.714"
}
},
"CNC": {
"36": {
"2018-08-13 14:15:01": "642.603"
}
},
"GRN": {
"36": {
"2018-08-13 14:15:01": "267.333"
}
},
"SLAG": {
"36": {
"2018-08-13 14:15:01": "233.866"
}
},
"HRG": {
"36": {
"2018-08-13 14:15:01": "426.800"
}
},
"ISTC": {
"36": {
"2018-08-13 14:15:01": "1010.045"
}
},
"SYM": {
"36": {
"2018-08-13 14:15:01": "1184.777"
}
},
"SYS": {
"36": {
"2018-08-13 14:15:01": "660.208"
}
},
"ELBT": {
"36": {
"2018-08-13 14:15:01": "125.717"
}
},
"MCS": {
"36": {
"2018-08-13 14:15:01": "1721.041"
}
},
"IOU": {
"36": {
"2018-08-13 14:15:01": "535.890"
}
}
}
this is with your append change where definitely making progress but optimally it doesn't separate them like this just separates them buy the count number
counts format is literally just an integer for what ever count where at so for instance right now where at
36 but technically been incremented to 37 already for the next run of phistory count will get loaded loading the 37 into dataentries
{
"EVL": {
"35": {
"2018-08-13 14:00:01": "440.716"
}
"36": {
"2018-08-13 14:00:01": "440.716"
}
},
"TCHS": {
"35": {
"2018-08-13 14:00:01": "994.678"
}
"36": {
"2018-08-13 14:00:01": "440.716"
}
},
this is the desired format
i think i can take b plug it into something like
b = u'EVL'
import unicodedata
unicodedata.normalize('NFKD', b).encode('ascii','ignore')
'EVL'
Upvotes: 1
Views: 89
Answers (1)
Reputation: 134
Check if this works
for b in a:
name = stockdf[b]['name']
price = stockdf[b]['price']
time = stockdf[b]['time']
if not b in phistory:
phistory[b] = {}
phistory[b][dataentries] = {time:price}
c = phistory.keys()
#print c
dataentries = dataentries + 1
save()
This should result in the kind of behavior you are looking for.
Please run tests on smaller samples of your database, it will make debugging this much quicker.
Interestingly, since we're rewriting/updating phistory every time we call the function, we may actually WANT to use 'w' instead of 'a' here. Otherwise we'll basically end up with the history of every iteration of phistory stored in one file, which could get massive. I recommend making the above change, and then testing both 'w' and 'a' separately, as I think 'w' might actually give you what you want here.
EDIT: Answering final question
So this is the way you were creating the dict and updating it. update() does what it says on the tin, it updates a (k,v) pair in a dict. It doesn't append things unless they don't exist already. Since the top-level key "EVL" already exists, all of its values are overwritten when you pass in update({'EVL' : some_values})
The way I did it was creating an empty entry if there wasn't one, then updating it by basically telling it what value to put in that new key. the syntax
dict[key] = X
will either update that key or create a new one if that key doesn't exist, but
dict[key][key2] = X
relies on dict[key] existing, because otherwise it won't know what the parent of [key2] is to add it in the first place. That's why we had to explicitly create the top-level key if it didn't exist. Then after that
dict[key_that_exists][key2] = X
will either update the (key2, v) pair or create it if it doesn't exist.
Hope that helps :)
Upvotes: 2
Related Questions
- Python dict being dumped to JSON file not clearing out previous file contents
- When I modify a value in a json of nested dicts it is removing other dicts
- Python JSON unhashable type: 'dict'
- Why do my python dictionaries keep overwriting
- While trying to append Python dict to JSON it writes only once
- Python overwriting dictionary instead of appending
- python: dictionary getting overwritten
- Weird ouput on updating json dictionary in python
- Dictionaries overwriting in Python
- Unexpected output when saving/loading dictionary to/from json