CODEWITHSUNDEEP
pythonarraysnumpydataframe
Michael
Michael

Reputation: 7377

Merge (numpy) arrays based on date

I have N arrays each structured as the following

Array 1: [['2014-01-01', '2014-01-03' ...], [1.1, 0.5, ...]]
Array 2: [['2014-01-01', '2014-01-02' ...], [1.4, 0.9, ...]]
Array 3: [['2014-01-02', '2014-01-04' ...], [0.8, 1.5, ...]]

And I want to get some type of data frame as the following

date            1-data    2-data
2014-01-01      1.1       1.4
2014-01-02      0         0.9
2014-01-03      0.5       0
2014-01-04      0         0

The problem, as you can see from the example, is that some dates are excluded from each array (i.e the dates aren't the same across all of the arrays). I am struggling finding a quick, pythonic way to merge all of my arrays into a dataframe, and filling missing data with zeros.

Upvotes: 3

Views: 1263

Answers (1)

Zero
Zero

Reputation: 76917

This should solve it, using merge function and outer method

>>> import pandas as pd
>>> import numpy as np
>>> d1 = pd.DataFrame(np.array([['2014-01-01', '2014-01-03'], [1.1, 0.5]])).T
>>> d2 = pd.DataFrame(np.array([['2014-01-01', '2014-01-02'], [1.4, 0.9]])).T
>>> d3 = pd.DataFrame(np.array([['2014-01-02', '2014-01-04'], [0.8, 1.5]])).T
>>> d1.columns = d2.columns = d3.columns = ['t','v']
>>> pd.DataFrame(np.array(d1.merge(d2, on='t', how='outer').
...                          merge(d3, on='t', how='outer').
...                          sort('t')),
...                          columns=['date','1-data','2-data','3-data'])
... 
         date 1-data 2-data 3-data
0  2014-01-01    1.1    1.4    NaN
1  2014-01-02    NaN    0.9    0.8
2  2014-01-03    0.5    NaN    NaN
3  2014-01-04    NaN    NaN    1.5

[4 rows x 4 columns]

Upvotes: 2

Related Questions