Linear regression graph interpretation

Question

I have a histogram showing frequency of some data. I have two type of files: Pdbs and Uniprots. Each Uniprot file is associated with a certain number of Pdbs. So this histogram shows how many Uniprot files are associated with 0 Pdb files, 1 Pdb file, 2 Pdb files ... 80 Pdb files. Y-axis is in a log scale.

I did a regression on the same dataset and this is the result.

Here is the code I'm using for the regression graph:

# Fitting Simple Linear Regression to the Training set
from sklearn.linear_model import LinearRegression
regressor = LinearRegression()
x = np.array(x).reshape((-1, 1))
y = np.array(y)
regressor.fit(x, y)

# Predicting the Test set results
y = regressor.predict(x)

# Visualizing the Training set results
plt.scatter(x, y, color = 'red')
plt.plot(x, regressor.predict(x), color = 'blue')
plt.title('Uniprot vs Pdb')
plt.xlabel('Pdbs')
plt.ylabel('Uniprot')
plt.savefig('regression_test.png')
plt.show()

Can you help me interpret the regression graph? I can understand that as the number of Pdbs increases, there will be less Uniprots associated with them. But why is it going negative on the y-axis? Is this normal?

Answer 1

The correct way to interpret this linear regression is "this linear regression is 90% meaningless." In fact, some of that 90% is worse than meaningless, it's downright misleading, as you have pointed out with the negative y values. OTOH, there is about 10% of it that we can interpret to good effect, but you have to know what you're looking for.

The Why: Amongst other often less apparent things, one of the assumptions of a linear regression model is that the data are more-or-less linear. If your data aren't linear with some very regular "noise" added in, then all bets are off. Your data aren't linear. They're not even close. So all bets are off.

Since all bets are off, it is helpful to examine the sort of things that we might have otherwise wanted to do with a linear regression model. The hardest thing is extrapolation, which is predicting y outside of the original x range. Your model's abilities at extrapolation are pretty well illustrated by its behavior at the endpoints. This is where you noticed "hey, my graph is all negative!". This is, in a very simplistic sense, because you took a linear model, fit it to data that did not satisfy the "linear" assumption, and then tried to make it do the hardest thing for a model to do. The second hardest thing for a model to do is interpolation which is making predictions inside the original x range. This linear regression isn't very good at that either. Further down the list is, if we simply look at the slope of the linear regression line, we can get a general idea of whether our data are increasing or decreasing. Note that even this bet is off if your data aren't linear. However, it generally works out in a not-entirely-useless sort of way for large classes of even non-linear real-world data. So, this one thing, your linear regression model gets kind of right. Your data are decreasing, and the linear model is also decreasing. That's the 10% I spoke of previously.

What to do: Try to fit a better model. You say that you log-transformed your original data, but it doesn't look like that helped much. In general, the whole point of "transforming" data is to make it look linear. The log transform is helpful for exponential data. If your starting data didn't look exponential-like, then the log transform probably isn't going to help. Since you are trying to do density estimation, you almost certainly want to fit a probability distribution to this stuff, for which you don't even need to do a transform to make the data linear. Here is another Stack Overflow answer with details about how to fit a beta distribution to data. However, there are many options.

Answer 2

Can you help me interpret the regression graph?

Linear Regression tries to built a line between x-variables and a target y-variable which assimates the 'real' value in the most closed possible way (graph you find also here: https://en.wikipedia.org/wiki/Linear_regression):

the line here is the blue line, and the original points are the black lines. The goal is to minimize the error (black dots to blue line) for all black dots.

The regression line is the blue line. That means you can describe a uniprot with a linear equatation y = m*x +b , which has a constant value m=0.1 (example) and b=0.2 (example) and x=Pdbs.

I can understand that as the number of Pdbs increases, there will be less Uniprots associated with them. But why is it going negative on the y-axis?

This is normal, you could plot this line until -10000000 Pdbs or whateever, it is just a equation. Not a real line.

But there is one mistake in your plot, you need to plot the original black dots also or not?

y = regressor.predict(x)
plt.scatter(x, y, color = 'red')

This is wrong, you should add the original values to it, to get the plot from my graphic, something like:

y = df['Uniprot']
plt.scatter(x, y, color = 'red')

should help to understand it.