Seaborn: Violinplot experiences difficulty with too many variables?

Question

I wanted to use seaborn to visualize my entire Pandas dataframe with violinplots, and I thought I had made the necessary corrections to generate a large graph for the sizable number of 270 variables my dataframe possessed.

However, no matter what I do, the violinplots only display their inner mini-boxplots (as another question here describes) for each variable, and not their kde's:

fig, ax = plt.subplots(figsize=(50,5))
ax.set_ylim(-6, 6)

a = sns.violinplot(x='variable', y='value', data=pd.melt(train_norm), ax=ax)
a.set_xticklabels(a.get_xticklabels(), rotation=90);

plt.savefig('massive_violinplot.png', figsize=(50,5), dpi=220)

(apologies for the cropped graph, the whole thing is too big to post)

Whereas the following code, using the same pd.Dataframe, but only showing the first six variables, displays correctly:

fig, ax = plt.subplots(figsize=(10,5))
ax.set_ylim(-6, 6)

a = sns.violinplot(x='variable', y='value', data=pd.melt(train_norm.iloc[:,:6]), ax=ax)
a.set_xticklabels(a.get_xticklabels(), rotation=90);

plt.savefig('massive_violinplot.png', figsize=(10,5), dpi=220)

How could I get a graph like the above for all the variables, filled with proper violinplots showing their kde's?

Answer 1

This is not related to the number of variables or the plot size but to the huge differences in the distributions of the variables. I can't access your data right now so I will ilustrate it with a made up dataset. You can follow along with your dataset, selecting the three variables with more dispersion and the three with less dispersion. As a dispersion measurement you can use the variance or even the data range (if you don't have crazy long tails) or something different, I am not sure what would work better.

rs = np.random.RandomState(42)
data = rs.randn(100, 6)
data[:, :3] *= 20
df = pd.DataFrame(data)

See what happens if we plot the density with common axes so they are directly comparable.

df.plot(kind='kde', subplots=True, layout=(3, 2), sharex=True, sharey=True)
plt.tight_layout()

This is more or less the same you can see in the seaborn violin plot but of course transposed.

sns.violinplot(x='variable', y='value', data=pd.melt(df))

This is usually great for comparing the variables because you can look at the differences in width as differences in density. Unfortunately the violin for the variables with more dispersion are so narrow that you can't see the width at all and you lose any sense of the shape. On the other hand the variables with less dispersion appear too short (actually in your dataset some of them are just horizontal lines).

For the first problem you can make the violins use all the available horizontal space by using scale='width' but then you no longer can compare the density across variables. The width is the same at the peaks but the density is not.

sns.violinplot(x='variable', y='value', data=pd.melt(df), scale='width')

By the way, this is what matplotlib's violin plot does by default.

plt.violinplot(df.T)

For the second problem I think your only option is to normalize or standardize the variables in some way.

sns.violinplot(x='variable', y='value', data=pd.melt((df - df.mean()) / df.std()))

Now you have a clearer view of each variable separately (how many modes they have, how skewed they are, how long the tails are...) but you can compare neither the scale nor the dispersion across variables.

The moral of the story is that you can't see everything at once, you have to pick and choose depending on what you are looking for in the data.