souradippal
souradippal

Reputation: 1

Python pandas dataframe is using variable reference not the value

I have a simple program, to calcualte a pandas df, but it seems the dataframe assignments are done with object reference and not with the values.

My assumption is that, it is happening because of object reference and I tried both shallow copy and deep copy but nothing changed. Also in this example, I have one record in json_records, but we can have multiple records also.

Below is the code block, can someone please help me. I am struggling with this.

import os, json
import pandas as pd

pd.set_option('display.max_colwidth', None)

json_records = [
  {
    "Metadata": {
      "ElementId": 1
    },
    "Data": {
      "SrcExternalId": 1,
      "col1": "a",
      "col2": "b"
    }
  }
]


tgt_details = pd.DataFrame({"ElementId": [1, 1, 2], "TgtId": [1, 2, 2],"Details":["a", None, None], "Property": ["Item1", None, None]})

df = pd.DataFrame(columns = ["Property", "MsgJsons"])

for record in json_records:

  fil_tgt_details = tgt_details[tgt_details["ElementId"] == record["Metadata"]["ElementId"]]

  for index, fil_tgt_detail in fil_tgt_details.iterrows() :
    upd_record = record
    upd_record["TgtId"] = fil_tgt_detail["TgtId"]
    
    # print(upd_record)

    if fil_tgt_detail["Details"]: 
      print("Creating new df")
      print(upd_record)
      tgtProperty = fil_tgt_detail["Property"]

      recordExists = df[df["Property"] == tgtProperty]
      recordExists.reset_index(inplace = True, drop = True)

      if recordExists.empty :

        dfRecord = {"Property": tgtProperty,
                  "MsgJsons":[[upd_record]]}

        df = pd.concat([df, pd.DataFrame(dfRecord)], ignore_index=True)
      else :
        df.loc[df["Property"] == tgtProperty]["MsgJsons"].values[0].append(upd_record)
    else:
      print("Do something else")

print(df)

The program should return me the dataframe:

Property MsgJsons
Item1 [{'Metadata': {'ElementId': 1}, 'Data': {'SrcExternalId': 1, 'col1': 'a', 'col2': 'b'}, 'TgtId': 1}]

But insted it is giving me this in return:

Property MsgJsons
Item1 [{'Metadata': {'ElementId': 1}, 'Data': {'SrcExternalId': 1, 'col1': 'a', 'col2': 'b'}, 'TgtId': 2}]

Upvotes: 0

Views: 31

Answers (0)

Related Questions