CODEWITHSUNDEEP
pythonpandas
cekar
cekar

Reputation: 366

Pandas correct values in lists

I have a dataframe like this:

data = {"pos":[["A1","A2"],
               ["B1","B2"]],
        "value"  :[[20,1000],
                 [20,50]]}

which creates with:

df = pd.DataFrame(data)

this dataframe:

        pos       value
0  [A1, A2]  [20, 1000]
1  [B1, B2]    [20, 50]

What I need is to cut values >= 1000 in half, fill both split values in the list at the entire position and also clone the corresponding position in column["pos"].

Like that:

        pos       value           new_value
0  [A1, A2]  [20, 1000]  [20, 500.0, 500.0]
1  [B1, B2]    [20, 50]            [20, 50]

Therefore I created this function:

def splitMax():
    loop_list = df.value.tolist()
    new_list = copy.deepcopy(loop_list)

    for idx, nested_list in enumerate(loop_list):
        for idx_nested, list_el in enumerate(nested_list):
            if list_el >= 1000 and list_el <=2000:

                new_list[idx][idx_nested] = list_el/2
                new_list[idx].insert(idx_nested+1,list_el/2)
    df["new_value"] = new_list

But now it is getting complicated: What I would like to have at the end is this below.

   old_pos   old_values        new_pos      new_values
0  [A1, A2]  [20, 1000]   [A1, A2, A2]  [20, 500, 500]
1  [B1, B2]    [20, 50]  [B1, B2, nan]     [20, 50, 0]

My problem is now that when there is a third line with values [1000, 1000] then all already calculated rows need a correction again.

Here for the second row: "pos_new" = [B1, B2, nan] then has to become [B1,B2,nan,nan] resp. new_value = [20, 50, 0, 0]

Upvotes: 1

Views: 234

Answers (1)

ecortazar
ecortazar

Reputation: 1422

I ended up with a solution in two steps that should help

[EDITED]

data = {"pos":[["A1","A2"],
               ["B1","B2"]],
        "value"  :[[20,1000],
                 [20,50]]}

df = pd.DataFrame(data)

limit = 1000

# New (proposed) implementation of your function
def new_row(row):
    row.loc['new_value'] = np.concatenate([(i, ) if i<limit else (i/2, i/2) for i in row[1]]) 
    row.loc['new_pos'] = np.concatenate([(row[0][pos],) if i<limit else (row[0][pos], row[0][pos]) for pos, i in enumerate(row[1])]) 
    return row

df = df.apply(new_row, axis = 1)

# To fix mismatched lengths
max_elements = df['new_value'].apply(len).max()

df['new_value'] = df['new_value'].apply(lambda x: np.append(x, [0] * (max_elements-len(x))))
df['new_pos'] = df['new_pos'].apply(lambda x: np.append(x, [np.nan] * (max_elements-len(x))))

Output

    pos         value       new_value               new_pos
0   [A1, A2]    [20, 1000]  [20.0, 500.0, 500.0]    [A1, A2, A2]
1   [B1, B2]    [20, 50]    [20, 50, 0]             [B1, B2, nan]

Upvotes: 1

Related Questions