CODEWITHSUNDEEP
pythonpandasrandom
ℕʘʘḆḽḘ
ℕʘʘḆḽḘ

Reputation: 19375

how to get a random sample in a multiindex pandas dataframe?

I have a dataframe that is indexed according to the following variables: NAME - date. Name is some sort of bizarre ID, and date is.. a date.

The data is very large and I would like to inspect the data I have for several random choices of NAME.

That is,

  1. pick a random NAME among the possible ones
  2. inspect the data for this NAME, ordered by time.

I dont know how to do that. I see that we can use get_level_values, but I dont have a specific NAME in mind, I just want to call random samples many times.

Any help appreciated! Thanks!

Upvotes: 3

Views: 1505

Answers (2)

maxymoo
maxymoo

Reputation: 36545

You could forget your multi-index, and just use isin with sample:

import random
df = df.reset_index()
df[df['NAME'].isin(random.sample(list(df['NAME'].unique()),5))]

Upvotes: 1

Jarad
Jarad

Reputation: 18913

import pandas as pd
import numpy as np
import random
import string

df = pd.DataFrame(data={'NAME': [''.join(random.choice(string.ascii_uppercase + string.ascii_lowercase + string.digits) for _ in range(17)) for _ in range(10)],
            'Date': pd.date_range('1/01/2016', periods=10),
            'Whatever': np.random.randint(20, 50, 10)},
                  columns=['NAME', 'Date', 'Whatever']).set_index(['NAME', 'Date'])

random_df = df[df.index.get_loc(np.random.choice(df.index.levels[0])) == True].sort_index(level=1)
print(random_df)

Returns a df that looks like this:

                              Whatever
NAME              Date                
xg71zOEQVOEfCZ2ne 2016-01-01        35
qLCXuEerCXi6gmF1Y 2016-01-02        26
0vDe7x8TIb5FRv7hV 2016-01-03        40
Ddc6FGKBdtcLqT53O 2016-01-04        31
IYcrKG9pjt7mHH3qn 2016-01-05        44
lAWObNTC8yXPMY3v5 2016-01-06        49
k90QWdPc5qFSCFi1c 2016-01-07        22
BWQoHo8lUyEwK9Nuf 2016-01-08        42
Xt0bxUerTan0i1eGw 2016-01-09        22
tc7PYCzpyGmYLbnxu 2016-01-10        46

A random_df that looks like this:

                              Whatever
NAME              Date                
IYcrKG9pjt7mHH3qn 2016-01-05        44

Upvotes: 2

Related Questions