Right way to reverse a pandas DataFrame?

Question

Here is my code:

import pandas as pd

data = pd.DataFrame({'Odd':[1,3,5,6,7,9], 'Even':[0,2,4,6,8,10]})

for i in reversed(data):
    print(data['Odd'], data['Even'])

When I run this code, i get the following error:

Traceback (most recent call last):
  File "C:\Python33\lib\site-packages\pandas\core\generic.py", line 665, in _get_item_cache
    return cache[item]
KeyError: 5

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\*****\Documents\******\********\****.py", line 5, in <module>
    for i in reversed(data):
  File "C:\Python33\lib\site-packages\pandas\core\frame.py", line 2003, in __getitem__
    return self._get_item_cache(key)
  File "C:\Python33\lib\site-packages\pandas\core\generic.py", line 667, in _get_item_cache
    values = self._data.get(item)
  File "C:\Python33\lib\site-packages\pandas\core\internals.py", line 1656, in get
    _, block = self._find_block(item)
  File "C:\Python33\lib\site-packages\pandas\core\internals.py", line 1936, in _find_block
    self._check_have(item)
  File "C:\Python33\lib\site-packages\pandas\core\internals.py", line 1943, in _check_have
    raise KeyError('no item named %s' % com.pprint_thing(item))
KeyError: 'no item named 5'

Why am I getting this error?
How can I fix that?
What is the right way to reverse pandas.DataFrame?

Answer 1

If your DF index is non-numeric, this works:

df = df.reset_index().sort_index(ascending=False).set_index('index') # reverses DF order

So reset_index() will give you sortable numbers and then you reverse it and set the index again.

Answer 2

This method is explicit and clear:

df.loc[reversed(df.index)]

Answer 3

What is the right way to reverse a pandas DataFrame?

TL;DR: df[::-1]

This is the best method for reversing a DataFrame, because 1) it is constant run time, i.e. O(1) 2) it's a single operation, and 3) concise/readable (assuming familiarity with slice notation).

---

Long Version

I've found the ol' slicing trick df[::-1] (or the equivalent df.loc[::-1]¹) to be the most concise and idiomatic way of reversing a DataFrame. This mirrors the python list reversal syntax lst[::-1] and is clear in its intent. With the loc syntax, you are also able to slice columns if required, so it is a bit more flexible.

Some points to consider while handling the index:

• "what if I want to reverse the index as well?"

  • you're already done. df[::-1] reverses both the index and values.

• "what if I want to drop the index from the result?"

  • you can call .reset_index(drop=True) at the end.

• "what if I want to keep the index untouched (IOW, only reverse the data, not the index)?"

  • this is somewhat unconventional because it implies the index isn't really relevant to the data. Perhaps consider removing it entirely? Although what you're asking for can technically be achieved using either df[:] = df[::-1] which creates an in-place update to df, or df.loc[::-1].set_index(df.index), which returns a copy.

_{1: df.loc[::-1] and df.iloc[::-1] are equivalent since the slicing syntax remains the same, whether you're reversing by position (iloc) or label (loc).}

---

The Proof is in the Pudding

X-axis represents the dataset size. Y-axis represents time taken to reverse. No method scales as well as the slicing trick, it's all the way at the bottom of the graph. Benchmarking code for reference, plots generated using perfplot.

---

Comments on other solutions

• df.reindex(index=df.index[::-1]) is clearly a popular solution, but on first glance, how obvious is it to an unfamiliar reader that this code is "reversing a DataFrame"? Additionally, this is reversing the index, then using that intermediate result to reindex, so this is essentially a TWO step operation (when it could've been just one).

• df.sort_index(ascending=False) may work in most cases where you have a simple range index, but this assumes your index was sorted in ascending order and so doesn't generalize well.

• PLEASE do not use iterrows. I see some options suggesting iterating in reverse. Whatever your use case, there is likely a vectorized method available, but if there isn't then you can use something a little more reasonable such as list comprehensions. See How to iterate over rows in a DataFrame in Pandas for more detail on why iterrows is an antipattern.

Answer 4

One way to do this if dealing with sorted range index is:

data = data.sort_index(ascending=False)

This approach has the benefits of (1) being a single line, (2) not requiring a utility function, and most importantly (3) not actually changing any of the data in the dataframe.

Caveat: this works by sorting the index in descending order and so may not always be appropriate or generalize for any given Dataframe.

Answer 5

None of the existing answers resets the index after reversing the dataframe.

For this, do the following:

 data[::-1].reset_index()

Here's a utility function that also removes the old index column, as per @Tim's comment:

def reset_my_index(df):
  res = df[::-1].reset_index(drop=True)
  return(res)

Simply pass your dataframe into the function

Answer 6

This works:

    for i,r in data[::-1].iterrows():
        print(r['Odd'], r['Even'])

Answer 7

You can reverse the rows in an even simpler way:

df[::-1]

Answer 8

data.reindex(index=data.index[::-1])

or simply:

data.iloc[::-1]

will reverse your data frame, if you want to have a for loop which goes from down to up you may do:

for idx in reversed(data.index):
    print(idx, data.loc[idx, 'Even'], data.loc[idx, 'Odd'])

or

for idx in reversed(data.index):
    print(idx, data.Even[idx], data.Odd[idx])

You are getting an error because reversed first calls data.__len__() which returns 6. Then it tries to call data[j - 1] for j in range(6, 0, -1), and the first call would be data[5]; but in pandas dataframe data[5] means column 5, and there is no column 5 so it will throw an exception. ( see docs )