Reputation: 1
I have a simple program, to calcualte a pandas df, but it seems the dataframe assignments are done with object reference and not with the values.
My assumption is that, it is happening because of object reference and I tried both shallow copy and deep copy but nothing changed. Also in this example, I have one record in json_records, but we can have multiple records also.
Below is the code block, can someone please help me. I am struggling with this.
import os, json
import pandas as pd
pd.set_option('display.max_colwidth', None)
json_records = [
{
"Metadata": {
"ElementId": 1
},
"Data": {
"SrcExternalId": 1,
"col1": "a",
"col2": "b"
}
}
]
tgt_details = pd.DataFrame({"ElementId": [1, 1, 2], "TgtId": [1, 2, 2],"Details":["a", None, None], "Property": ["Item1", None, None]})
df = pd.DataFrame(columns = ["Property", "MsgJsons"])
for record in json_records:
fil_tgt_details = tgt_details[tgt_details["ElementId"] == record["Metadata"]["ElementId"]]
for index, fil_tgt_detail in fil_tgt_details.iterrows() :
upd_record = record
upd_record["TgtId"] = fil_tgt_detail["TgtId"]
# print(upd_record)
if fil_tgt_detail["Details"]:
print("Creating new df")
print(upd_record)
tgtProperty = fil_tgt_detail["Property"]
recordExists = df[df["Property"] == tgtProperty]
recordExists.reset_index(inplace = True, drop = True)
if recordExists.empty :
dfRecord = {"Property": tgtProperty,
"MsgJsons":[[upd_record]]}
df = pd.concat([df, pd.DataFrame(dfRecord)], ignore_index=True)
else :
df.loc[df["Property"] == tgtProperty]["MsgJsons"].values[0].append(upd_record)
else:
print("Do something else")
print(df)
The program should return me the dataframe:
Property | MsgJsons |
---|---|
Item1 | [{'Metadata': {'ElementId': 1}, 'Data': {'SrcExternalId': 1, 'col1': 'a', 'col2': 'b'}, 'TgtId': 1}] |
But insted it is giving me this in return:
Property | MsgJsons |
---|---|
Item1 | [{'Metadata': {'ElementId': 1}, 'Data': {'SrcExternalId': 1, 'col1': 'a', 'col2': 'b'}, 'TgtId': 2}] |
Upvotes: 0
Views: 31