CODEWITHSUNDEEP
pythonpandasassignmethod-chaining
dmeu
dmeu

Reputation: 4052

Python pandas method chaining: assign column from strsplit

I have a problem with the assign method, when I want to create a new column from a split of another column. If I select a value of the split method, I'll get the error ValueError: Length of values does not match length of index. If I just apply split, without selecting (index) any value, I get a new column with a list inside.

Here is the output if I do not index the output of the split method

(
    pd.DataFrame({
        "Gene": ["G1", "G1", "G2", "G2"],
        "Sample": ["H1_T1", "H2_T1", "H1_T1", "H2_T1"]
    })
    .assign(Timepoint = lambda x: x.Sample.str.split("_")[1])
)
    Gene    Sample  Timepoint
0   G1  H1_T1   [H1, T1]
1   G1  H2_T1   [H2, T1]
2   G2  H1_T1   [H1, T1]
3   G2  H2_T1   [H2, T1]

Here is an example where I would like to select the T1 or T2 value from the Sample column and gives an error:

(
    pd.DataFrame({
        "Gene": ["G1", "G1", "G2", "G2"],
        "Sample": ["H1_T1", "H2_T1", "H1_T1", "H2_T1"]
    })
    .assign(Timepoint = lambda x: x.Sample.str.split("_")[1])
)

The error I get from this is:

/home/user/anaconda3/lib/python3.4/site-packages/pandas/core/series.py in _sanitize_index(data, index, copy)
   2739 
   2740     if len(data) != len(index):
-> 2741         raise ValueError('Length of values does not match length of '
   2742                          'index')
   2743 

ValueError: Length of values does not match length of index

Upvotes: 4

Views: 1478

Answers (1)

EdChum
EdChum

Reputation: 393963

IIUC then you want an additional call to str to select the elements:

In [234]:
pd.DataFrame({
        "Gene": ["G1", "G1", "G2", "G2"],
        "Sample": ["H1_T1", "H2_T1", "H1_T1", "H2_T1"]
    }).assign(Timepoint = lambda x: x.Sample.str.split("_").str[1])

Out[234]:
  Gene Sample Timepoint
0   G1  H1_T1        T1
1   G1  H2_T1        T1
2   G2  H1_T1        T1
3   G2  H2_T1        T1

If we modify your df slightly and see the output

In [237]:
df = pd.DataFrame({
        "Gene": ["G1", "G1", "G2", "G2"],
        "Sample": ["H1_T1", "H2_T2", "H1_T3", "H2_T4"]
    })

df['Sample'].str.split("_")

Out[237]:
0    [H1, T1]
1    [H2, T2]
2    [H1, T3]
3    [H2, T4]
dtype: object

So what you attempted was the following:

In [238]:
df['Sample'].str.split("_")[1]

Out[238]:
['H2', 'T2']

You can see that what this did was to select the second row, what you want is to select the second element of each row:

In [239]:
df['Sample'].str.split("_").str[1]

Out[239]:
0    T1
1    T2
2    T3
3    T4
dtype: object

Upvotes: 4

Related Questions