Multi-dimensional/Nested DataFrame/Dataset/Panel in Pandas

Question

I would like to store some multidimensional data in a pandas dataframe or panel such that I would like to be able to return for example:

1. All the times for Runner A, Race A
2. All the times(and names) for Race A for a certain year say 2015

Example data would look something like this, note that not all runners will have data for all years or all races.

Could anyone suggest a good way to do this with Pandas or any other way?

Name     | Gender | Age
Runner A | Male   | 35
    Race A
        Year | Time     
        2015 | 2:35:09  
        2014 | 2:47:34
        2013 | 2:50:12
    Race B
        Year | Time
        2013 | 1:32:07

Runner B | Male | 29
    Race A
        Year | Time
        2015 | 3:05:56

Runner C | Female | 32
    Race B
        Year | Time
        1998 | 1:29:43

Answer 1

Simple approach:

runners = pd.DataFrame(
    [
        ['Bob',   'Male', 1980],
        ['Tom',   'Male', 1986],
        ['Amy', 'Female', 1966],
    ],
    columns=['Name', 'Gender', 'BirthYear']
)

races = pd.DataFrame(
    [
        ['A', 2015, 'Bob', '2:35:09'],
        ['A', 2014, 'Bob', '2:47:34'],
        ['A', 2013, 'Bob', '2:50:12'],
        ['B', 2013, 'Bob', '1:32:07'],
        ['A', 2015, 'Tom', '3:05:56'],
        ['B', 1998, 'Amy', '1:29:43'],
    ],
    columns=['Race', 'Year', 'Name', 'Time']
)


print races.loc[(races.Name == 'Bob') & (races.Race == 'A')][['Time']]
print 
print races.loc[(races.Year == 2015) & (races.Race == 'A')][['Name', 'Time']]

      Time
0  2:35:09
1  2:47:34
2  2:50:12

  Name     Time
0  Bob  2:35:09
4  Tom  3:05:56

get back all data

df = races.merge(runners)

get age at race time

df['Age'] = df.Year - df.BirthYear
print df

  Race  Year Name     Time  Gender  BirthYear  Age
0    A  2015  Bob  2:35:09    Male       1980   35
1    A  2014  Bob  2:47:34    Male       1980   34
2    A  2013  Bob  2:50:12    Male       1980   33
3    B  2013  Bob  1:32:07    Male       1980   33
4    A  2015  Tom  3:05:56    Male       1986   29
5    B  1998  Amy  1:29:43  Female       1966   32

Answer 2

I think you can use Multiindex and then select data by slicers:

import pandas as pd

df = pd.DataFrame({'Time': {('Runner A', 'Male', 35, 'Race A', 2014): '2:47:34', ('Runner C', 'Female', 32, 'Race B', 1998): '1:29:43', ('Runner B', 'Male', 29, 'Race A', 2015): '3:05:56', ('Runner A', 'Male', 35, 'Race A', 2013): '2:50:12', ('Runner A', 'Male', 35, 'Race B', 2013): '1:32:07', ('Runner A', 'Male', 35, 'Race A', 2015): '2:35:09'}})
print (df)
                                   Time
Runner A Male   35 Race A 2013  2:50:12
                          2014  2:47:34
                          2015  2:35:09
                   Race B 2013  1:32:07
Runner B Male   29 Race A 2015  3:05:56
Runner C Female 32 Race B 1998  1:29:43

#index has to be fully lexsorted
df.sort_index(inplace=True)
print (df)
                                   Time
Runner A Male   35 Race A 2013  2:50:12
                          2014  2:47:34
                          2015  2:35:09
                   Race B 2013  1:32:07
Runner B Male   29 Race A 2015  3:05:56
Runner C Female 32 Race B 1998  1:29:43

idx = pd.IndexSlice
print (df.loc[idx['Runner A',:,:,'Race A',:],:])
                                 Time
Runner A Male 35 Race A 2013  2:50:12
                        2014  2:47:34
                        2015  2:35:09

print (df.loc[idx[:,:,:,'Race A',2015],:])
                                 Time
Runner A Male 35 Race A 2015  2:35:09
Runner B Male 29 Race A 2015  3:05:56