Why am I getting the error "OverflowError: Python int too large to convert to C long" from pandas/matplotlib?

Question

I have a pandas dataframe with dates and maximum and minimum values called solflux:

    date        max   min
0   2015-01-01  148  10.5
1   2015-02-01  142   9.5
2   2015-03-01  140   9.0
3   2015-04-01  135   7.5
4   2015-05-01  132   7.5
...

I want to plot these values as errorbars in matplotlib. I use the following to find the values I want

dates = solflux['date']
deltas = (solflux['max'] - solflux['min'])/2
means = solflux['min'] + deltas

and then plot with

import matplotlib.pyplot as plt
f = plt.figure()
ax = f.add_subplot(111)
ax.errorbar(dates, means, marker='+', yerr=deltas)

I then get this error:

C:\Anaconda3\lib\site-packages\matplotlib\axes\_axes.py:545: UserWarning: No labelled objects found. Use label='...' kwarg on individu
al plots.
  warnings.warn("No labelled objects found. "
Traceback (most recent call last):
  File "C:\Anaconda3\lib\site-packages\matplotlib\backends\backend_qt5agg.py", line 197, in __draw_idle_agg
    FigureCanvasAgg.draw(self)
  File "C:\Anaconda3\lib\site-packages\matplotlib\backends\backend_agg.py", line 464, in draw
    self.figure.draw(self.renderer)
  File "C:\Anaconda3\lib\site-packages\matplotlib\artist.py", line 63, in draw_wrapper
    draw(artist, renderer, *args, **kwargs)
  File "C:\Anaconda3\lib\site-packages\matplotlib\figure.py", line 1143, in draw
    renderer, self, dsu, self.suppressComposite)
  File "C:\Anaconda3\lib\site-packages\matplotlib\image.py", line 139, in _draw_list_compositing_images
    a.draw(renderer)
  File "C:\Anaconda3\lib\site-packages\matplotlib\artist.py", line 63, in draw_wrapper
    draw(artist, renderer, *args, **kwargs)
  File "C:\Anaconda3\lib\site-packages\matplotlib\axes\_base.py", line 2409, in draw
    mimage._draw_list_compositing_images(renderer, self, dsu)
  File "C:\Anaconda3\lib\site-packages\matplotlib\image.py", line 139, in _draw_list_compositing_images
    a.draw(renderer)
  File "C:\Anaconda3\lib\site-packages\matplotlib\artist.py", line 63, in draw_wrapper
    draw(artist, renderer, *args, **kwargs)
  File "C:\Anaconda3\lib\site-packages\matplotlib\axis.py", line 1136, in draw
    ticks_to_draw = self._update_ticks(renderer)
  File "C:\Anaconda3\lib\site-packages\matplotlib\axis.py", line 969, in _update_ticks
    tick_tups = [t for t in self.iter_ticks()]
  File "C:\Anaconda3\lib\site-packages\matplotlib\axis.py", line 969, in <listcomp>
    tick_tups = [t for t in self.iter_ticks()]
  File "C:\Anaconda3\lib\site-packages\matplotlib\axis.py", line 912, in iter_ticks
    majorLocs = self.major.locator()
  File "C:\Anaconda3\lib\site-packages\matplotlib\dates.py", line 983, in __call__
    self.refresh()
  File "C:\Anaconda3\lib\site-packages\matplotlib\dates.py", line 1003, in refresh
    dmin, dmax = self.viewlim_to_dt()
  File "C:\Anaconda3\lib\site-packages\matplotlib\dates.py", line 760, in viewlim_to_dt
    return num2date(vmin, self.tz), num2date(vmax, self.tz)
  File "C:\Anaconda3\lib\site-packages\matplotlib\dates.py", line 401, in num2date
    return _from_ordinalf(x, tz)
  File "C:\Anaconda3\lib\site-packages\matplotlib\dates.py", line 254, in _from_ordinalf
    dt = datetime.datetime.fromordinal(ix).replace(tzinfo=UTC)
OverflowError: Python int too large to convert to C long

The strange thing is that if I ignore the yerr part and just run

ax.plot(dates, means, marker='+')

everything works as expected. I suspect this is a bug in matplotlib, but was wondering if anyone had any suggestions?

I'm running Python 3.6.0, Matplotlib 2.0.0 and Pandas 0.19.2 which is all up to date as far as I am aware.

Answer 1

I also got an error when I tried to run your example code, but it wasn't the same one you posted. The error I see is:

ValueError: invalid literal for float(): 2015-01-01

The error trace you posted also seems to be concerned with date conversions, though, so we may be dealing with basically the same problem - are you dates stored as date objects? I'm able to run your example code successfully after explicitly converting dates to a Datetime object, using to_datetime():

    date        max   min
0   2015-01-01  148  10.5
1   2015-02-01  142   9.5
2   2015-03-01  140   9.0
3   2015-04-01  135   7.5
4   2015-05-01  132   7.5

solflux = pd.read_clipboard()
dates = pd.to_datetime(solflux['date'])
deltas = (solflux['max'] - solflux['min'])/2
means = solflux['min'] + deltas

f = plt.figure()
ax = f.add_subplot(111)
ax.errorbar(dates, means, marker='+', yerr=deltas)

Note, however, that you can achieve the same result using native Pandas functionality, and the tick labels end up a bit neater this way:

solflux.date = pd.to_datetime(solflux.date)
solflux['deltas'] = (solflux['max'] - solflux['min'])/2
solflux['means'] = solflux['min'] + solflux.deltas

Data frame:

        date  max   min  deltas  means
0 2015-01-01  148  10.5   68.75  79.25
1 2015-02-01  142   9.5   66.25  75.75
2 2015-03-01  140   9.0   65.50  74.50
3 2015-04-01  135   7.5   63.75  71.25
4 2015-05-01  132   7.5   62.25  69.75

Now use plot() (optionally set date as index, you can also just set x='date' inside plot()):

solflux.set_index('date').plot(y='means', yerr='deltas', legend=False)