CODEWITHSUNDEEP
python-3.xpandasdataframe
Danish
Danish

Reputation: 2871

Sort the list of dictionaries based on date column of dataframe in pandas

I have a input list and dataframe as shown below.

[{"type": "linear",
  "from": "2020-02-04T20:00:00.000Z",
  "to": "2020-02-03T20:00:00.000Z",
  "days":3,
  "coef":[0.1,0.1,0.1,0.1,0.1,0.1]
  },
 {"type": "quadratic",
  "from": "2020-02-03T20:00:00.000Z",
  "to": "2020-02-10T20:00:00.000Z",
  "days":3,
  "coef":[0.1,0.1,0.1,0.1,0.1,0.1]
  },
 {"type": "polynomial",
  "from": "2020-02-05T20:00:00.000Z",
  "to": "2020-02-03T20:00:00.000Z",
  "days":3,
  "coef":[0.1,0.1,0.1,0.1,0.1,0.1]
  }]

df:

Date                t_factor     
2020-02-01             5             
2020-02-02             23              
2020-02-03             14           
2020-02-04             23
2020-02-05             23  
2020-02-06             23          
2020-02-07             30            
2020-02-08             29            
2020-02-09             100
2020-03-10             38
2020-03-11             38               
2020-03-12             38                    
2020-03-13             70           
2020-03-14             70

Step1: Sort the list based on the value of "from" key in dictionary

[
 {"type": "quadratic",
      "from": "2020-02-03T20:00:00.000Z",
      "to": "2020-02-10T20:00:00.000Z",
      "days":3,
      "coef":[0.1,0.1,0.1,0.1,0.1,0.1]
      },
{"type": "linear",
      "from": "2020-02-04T20:00:00.000Z",
      "to": "2020-02-03T20:00:00.000Z",
      "days":3,
      "coef":[0.1,0.1,0.1,0.1,0.1,0.1]
      },
     {"type": "polynomial",
      "from": "2020-02-05T20:00:00.000Z",
      "to": "2020-02-03T20:00:00.000Z",
      "days":3,
      "coef":[0.1,0.1,0.1,0.1,0.1,0.1]
      }]

Step2:add a dictionary with value of "from" key as minimum date of df and "to" should be "from" date the first dictionary in the sorted list. "days" = 0, "coef":[0.1,0.1,0.1,0.1,0.1,0.1].

{"type": "df_first",
      "from": "2020-02-01T20:00:00.000Z",
      "to": "2020-02-03T20:00:00.000Z",
      "days":0,
      "coef":[0.1,0.1,0.1,0.1,0.1,0.1]
      }

Step3:add a dictionary with value of "from" key as 7 days after minimum date of df and "to" should be one days after from

{"type": "df_mid",
      "from": "2020-02-08T20:00:00.000Z",
      "to": "2020-02-09T20:00:00.000Z",
      "days":0,
      "coef":[0.1,0.1,0.1,0.1,0.1,0.1]
      }

Step4:add a dictionary with value of "from" key as maximum date of df and "to" should be same as well as "from".

{"type": "df_last",
      "from": "2020-02-14T20:00:00.000Z",
      "to": "2020-02-14T20:00:00.000Z",
      "days":0,
      "coef":[0.1,0.1,0.1,0.1,0.1,0.1]
      }

Step 5: Sort all the dictionary based on "from" date.

Expected Output:

[{"type": "df_first",
      "from": "2020-02-01T20:00:00.000Z",
      "to": "2020-02-03T20:00:00.000Z",
      "days":0,
      "coef":[0.1,0.1,0.1,0.1,0.1,0.1]
      },
     {"type": "quadratic",
      "from": "2020-02-03T20:00:00.000Z",
      "to": "2020-02-10T20:00:00.000Z",
      "days":3,
      "coef":[0.1,0.1,0.1,0.1,0.1,0.1]
      },
{"type": "linear",
      "from": "2020-02-04T20:00:00.000Z",
      "to": "2020-02-03T20:00:00.000Z",
      "days":3,
      "coef":[0.1,0.1,0.1,0.1,0.1,0.1]
      },

     {"type": "polynomial",
      "from": "2020-02-05T20:00:00.000Z",
      "to": "2020-02-03T20:00:00.000Z",
      "days":3,
      "coef":[0.1,0.1,0.1,0.1,0.1,0.1]
      },
{"type": "df_mid",
      "from": "2020-02-08T20:00:00.000Z",
      "to": "2020-02-09T20:00:00.000Z",
      "days":0,
      "coef":[0.1,0.1,0.1,0.1,0.1,0.1]
      },

{"type": "df_last",
      "from": "2020-02-14T20:00:00.000Z",
      "to": "2020-02-14T20:00:00.000Z",
      "days":0,
      "coef":[0.1,0.1,0.1,0.1,0.1,0.1]
      }
]

Step 6:

Replace the "to" value of each dictionary with "from" value of next dictionary. "to" value of last dictionary be as it is.

Expected Final output:

[{"type": "df_first",
          "from": "2020-02-01T20:00:00.000Z",
          "to": "2020-02-03T20:00:00.000Z",
          "days":0,
          "coef":[0.1,0.1,0.1,0.1,0.1,0.1]
          },
         {"type": "quadratic",
          "from": "2020-02-03T20:00:00.000Z",
          "to": "2020-02-04T20:00:00.000Z",
          "days":3,
          "coef":[0.1,0.1,0.1,0.1,0.1,0.1]
          },
    {"type": "linear",
          "from": "2020-02-04T20:00:00.000Z",
          "to": "2020-02-05T20:00:00.000Z",
          "days":3,
          "coef":[0.1,0.1,0.1,0.1,0.1,0.1]
          },
    
         {"type": "polynomial",
          "from": "2020-02-05T20:00:00.000Z",
          "to": "2020-02-08T20:00:00.000Z",
          "days":3,
          "coef":[0.1,0.1,0.1,0.1,0.1,0.1]
          },
    {"type": "df_mid",
          "from": "2020-02-08T20:00:00.000Z",
          "to": "2020-02-14T20:00:00.000Z",
          "days":0,
          "coef":[0.1,0.1,0.1,0.1,0.1,0.1]
          },
    
    {"type": "df_last",
          "from": "2020-02-14T20:00:00.000Z",
          "to": "2020-02-14T20:00:00.000Z",
          "days":0,
          "coef":[0.1,0.1,0.1,0.1,0.1,0.1]
          }
    ]

Upvotes: 1

Views: 74

Answers (1)

Shubham Sharma
Shubham Sharma

Reputation: 71687

Define a function add_dct that takes arguments as list of dictionaries lst with _type, _from and _to and appends a new dictionary to lst:

dmin, dmax = df['Date'].min(), df['Date'].max()
def add_dct(lst, _type, _from, _to):
    lst.append({
        'type': _type,
        'from': _from if isinstance(_from, str) else _from.strftime("%Y-%m-%dT20:%M:%S.000Z"),
        'to': _to if isinstance(_to, str) else _to.strftime("%Y-%m-%dT20:%M:%S.000Z"),
        'days': 0,
        "coef":[0.1,0.1,0.1,0.1,0.1,0.1]
    })

Follow this steps as according to your predefined requirements:

# STEP 1
lst = sorted(lst, key=lambda d: pd.Timestamp(d['from']))

# STEP 2
add_dct(lst, 'df_first', dmin, lst[0]['from'])

# STEP 3
add_dct(lst, 'df_mid', dmin + pd.Timedelta(days=7), dmin + pd.Timedelta(days=8))

# STEP 4
add_dct(lst, 'df_last', dmax, dmax)

# STEP 5
lst = sorted(lst, key=lambda d: pd.Timestamp(d['from']))

Result:

[{'type': 'df_first',
  'from': '2020-02-01T20:00:00.000Z',
  'to': '2020-02-03T20:00:00.000Z',
  'days': 0,
  'coef': [0.1, 0.1, 0.1, 0.1, 0.1, 0.1]},
 {'type': 'quadratic',
  'from': '2020-02-03T20:00:00.000Z',
  'to': '2020-02-10T20:00:00.000Z',
  'days': 3,
  'coef': [0.1, 0.1, 0.1, 0.1, 0.1, 0.1]},
 {'type': 'linear',
  'from': '2020-02-04T20:00:00.000Z',
  'to': '2020-02-03T20:00:00.000Z',
  'days': 3,
  'coef': [0.1, 0.1, 0.1, 0.1, 0.1, 0.1]},
 {'type': 'polynomial',
  'from': '2020-02-05T20:00:00.000Z',
  'to': '2020-02-03T20:00:00.000Z',
  'days': 3,
  'coef': [0.1, 0.1, 0.1, 0.1, 0.1, 0.1]},
 {'type': 'df_mid',
  'from': '2020-02-08T20:00:00.000Z',
  'to': '2020-02-09T20:00:00.000Z',
  'days': 0,
  'coef': [0.1, 0.1, 0.1, 0.1, 0.1, 0.1]},
 {'type': 'df_last',
  'from': '2020-03-14T20:00:00.000Z',
  'to': '2020-03-14T20:00:00.000Z',
  'days': 0,
  'coef': [0.1, 0.1, 0.1, 0.1, 0.1, 0.1]}]

Upvotes: 1

Related Questions