Pandas dataframe, ValueError: shape mismatch: objects cannot be broadcast to a single shape

Question

I have this code that is used to track how late specific deliveries were, and how late they were by. I categorized them as so: early deliveries, on time, and late deliveries. I am able to graph these results if I include every material number. However, when I specific by material number, I run into an error( listed below) I have also provided what is exactly printed in the terminal. It seems as if the dataframe has created two rows labeled different things, and has the count from there, thus I am unable to graph because there are two values, So how can I fix my code to simply extract the "count number" and use that number to graph the bargraph

import matplotlib.pyplot as plt; plt.rcdefaults()
import numpy as np
import matplotlib.pyplot as plt
import pandas as pd


Material= 'Material'
DELIVERY_DATE = 'Delivery Date'
DESIRED_DATE = 'source desired delivery date'
DELAYED_DAYS = 'Delayed Days'



StartYear = input("Start Year? ")
StartYear = int(StartYear)
EndYear = input("End Year? ")
EndYear = int(EndYear) 




DELIVERY_DATE = 'Delivery Date'
DESIRED_DATE = 'source desired delivery date'
DELAYED_DAYS = 'Delayed Days'


df = pd.read_csv('otdo.csv')

df['Delivery Date'] = pd.to_datetime(df['Delivery Date'], format='%m/%d/%Y')
df['source desired delivery date'] = pd.to_datetime(df['source desired delivery date'], format='%m/%d/%Y')


late_threshold = pd.Timedelta(days=0)
late_threshold2 = pd.Timedelta(days=10)

df[DELIVERY_DATE] = pd.to_datetime(df[DELIVERY_DATE])
df[DESIRED_DATE] = pd.to_datetime(df[DESIRED_DATE])
df[DELAYED_DAYS] = df[DELIVERY_DATE] - df[DESIRED_DATE]


df2 = df[(df['Delivery Date'].dt.year >= int(StartYear)) & (df['Delivery Date'].dt.year <= int(EndYear))]



df3 = df2[ df2[DELAYED_DAYS] > late_threshold] 
df3 =  df3[late_threshold2 > df3[DELAYED_DAYS]]
df3 = df3.loc[df['Material'].str.contains('20080810', na=False)]




df4 = df2[ df2[DELAYED_DAYS] > late_threshold2] 
df4 = df4.loc[df['Material'].str.contains('20080810', na=False)]


df5 = df2[df2[DELAYED_DAYS] <= late_threshold] 
df5 = df5.loc[df['Material'].str.contains('20080810', na=False)]


df6 = df2.loc[df['Material'].str.contains('20080810', na=False)]




df7 = df2[ df2[DELAYED_DAYS] > late_threshold] 
df7 =  df7[late_threshold2 > df7[DELAYED_DAYS]]



df8 = df2[ df2[DELAYED_DAYS] > late_threshold2] 


df9 = df2[df2[DELAYED_DAYS] <= late_threshold] 


zero = df2.count() 
zero2 = df3.count()
zero3 = df4.count() 
zero4 = df5.count() 
zero5 = df7.count()
zero7 = df9.count() 

hey = zero7.iloc[1:1]
print(hey)
print(zero7)

objects = ('1', '2', '3')
y_pos = np.arange(len(objects))
values = [zero5, zero4, zero7]

plt.bar(y_pos, values, align='center', alpha=0.2)
plt.xticks(y_pos, objects)

plt.show()

This is what is produced in the terminal:

Start Year? 2014
End Year? 2018
Series([], dtype: int64)
Material        4936
Delayed Days    4936
dtype: int64
Traceback (most recent call last):
  File "C:\Users\khalha\eclipse-workspace\Test3\Test3\gagada.py", line 118, in <module>
    plt.bar(y_pos, values, align='center', alpha=0.2)
  File "C:\Users\khalha\AppData\Local\Programs\Python\Python37\lib\site-packages\matplotlib\pyplot.py", line 2770, in bar
    ret = ax.bar(*args, **kwargs)
  File "C:\Users\khalha\AppData\Local\Programs\Python\Python37\lib\site-packages\matplotlib\__init__.py", line 1855, in inner
    return func(ax, *args, **kwargs)
  File "C:\Users\khalha\AppData\Local\Programs\Python\Python37\lib\site-packages\matplotlib\axes\_axes.py", line 2233, in bar
    np.atleast_1d(x), height, width, y, linewidth)
  File "C:\Users\khalha\AppData\Local\Programs\Python\Python37\lib\site-packages\numpy\lib\stride_tricks.py", line 249, in broadcast_arrays
    shape = _broadcast_shape(*args)
  File "C:\Users\khalha\AppData\Local\Programs\Python\Python37\lib\site-packages\numpy\lib\stride_tricks.py", line 184, in _broadcast_shape
    b = np.broadcast(*args[:32])
ValueError: shape mismatch: objects cannot be broadcast to a single shape

Csv file:

Material    Delivery Date   source desired delivery date
3334678 12/31/2014  12/31/2014
233433  12/31/2014  12/31/2014
3434343 1/5/2015    1/5/2015
3334567 1/5/2015    1/6/2015
546456  2/11/2015   2/21/2015
221295  4/10/2015   4/10/2015

Answer 1

The error message says that in

plt.bar(y_pos, values...

matplotlib expects as bar heights a 1D-array, but with values you provide a list of dataframes, which can't be broadcast to a simple 1D-array.

You should use a list of scalars instead to make this work.

E.g.

values = [zero5.Material, zero4.Material, zero7.Material]

if I understand your data model correct.
Note that if you want to plot two arrays, i.e. two bars at each y_pos for comparison reasons, this is done by calling plt.bar(...) two times. First with the one array, second with the other, adding some y-offset to the y-pos array. See this for an example.

However - I'd recommend you to create not too many further dataframes derived from your csv-import, but one single dataframe, which contains the boolean results depending on your threshold times, presumably already cast to 'int' for calculating the sum , e.g.:

df2['thresh1'] = (df2[DELAYED_DAYS] > late_threshold).astype(int)
df2['thresh2'] = (df2[DELAYED_DAYS] > late_threshold).astype(int)

This gives you the opportunity to calculate in one line

zeros = df2.sum()

what you called zeros.
Then a first test could be

zeros.plot(kind='bar')