CODEWITHSUNDEEP
pythonlinear-regressionstatsmodels
Nurislom Rakhmatullaev
Nurislom Rakhmatullaev

Reputation: 315

Which formula is used for t-value and standard error in statsmodels OLS

I want to understand how the python statsmodels library works. So when I am trying to get results using formulas from econometrics for OLS t-values and SEE or bse I am getting not the same answers as it is in statsmodels. (OLS with zero intercept) I have :

x = [1,2,3]
y = [7,3,5]

And received the same results for R^2, residual as it is in statsmodels with this code:

def ols(x, y):
    # OLS
    df = pd.DataFrame(data={'x':x, 'y':y})
    coeff = sum(df['y'] * df['x']) / sum(df['x'] ** 2)
    df['predict'] = df['x'] * coeff
    # R^2
    n = len(df)
    rss = sum((df['y'] - df['predict']) ** 2 )
    tss = sum((df['y']) ** 2)
    r2 = 1 - rss/tss
    # Residaual
    resid = (df['y'] - df['predict']).values
    
    return coeff, r2, resid, df

This is my statsmodels oblect

ols_obj = OLS(y, x).fit()

print('coeff', (ols_obj.predict(x)/x)[0])
print('R^2', ols_obj.rsquared)
print('resid', ols_obj.resid)

print('t', ols_obj.tvalues)
print('param', ols_obj.params[0], '| bse', ols_obj.bse[0], '| param/bse', ols_obj.params[0]/ols_obj.bse)

coeff 2.0
R^2 0.6746987951807228
resid [ 5. -1. -1.]
t [2.03670031]
param 2.0 | bse 0.9819805060619657 | param/bse [2.03670031]

This is my function:

coeff, r2, resid, df = ols(x, y)

print('coeff', coeff)
print('R^2', r2)
print('resid', resid)

coeff 2.0
R^2 0.6746987951807228
resid [ 5. -1. -1.]

But for t values, I am getting wrong numbers
From econometrics I used formula for standard error
SE(b) = sqrt( ( sum(resid^2) / (n-2) ) / sum( (x - mean(x) ) **2 ) )

SE(b) = 3.6742346141747673

What I am doing wrong?

Upvotes: 3

Views: 2394

Answers (2)

Nurislom Rakhmatullaev
Nurislom Rakhmatullaev

Reputation: 315

I found a formula of Standart error or SE(b) when there is no intercept.

For (nD x-axis) matrix, it looks like this:

SSE = np.dot(residual.T, residual)
DFE = len(x) - 2
MSE = SSE/DFE
inverted = np.linalg.inv(x.T, x)
covariance = inverted * MSE
bse = np.diag(covariance)
tvalues = coeff/bse

For 1D array x, it looks like this:

bse = math.sqrt(sum(resid**2) / ((len(df)-1) * sum( (df.x ** 2 ))))
tvalue = coeff/bse

Upvotes: 1

ce.teuf
ce.teuf

Reputation: 786

I believe, like @Josef, that this is the formula with an intercept. If you allow yourself to follow matrix development Wiki link:

import statsmodels.api as sm

x = np.array([1,2,3])
y = np.array([7,3,5])
resx = sm.OLS(y, x).fit()

# residual variance
res_variance = (1/(3-1))*sum(resx.resid**2) 
# estimator stand. Err.
beta_se = np.sqrt((res_variance)*(1 / (x.T @ x))) # x.T @ x is a scalar here. use np.linalg.Inv otherwise

new_tval = resx.params / beta_se # 2.036700..

is the same as

resx.tvalues # 2.036700..

Upvotes: 4

Related Questions