CODEWITHSUNDEEP
rforecastingevaluation
Rayan
Rayan

Reputation: 11

how can I find AIC, BIC, R2 and RMSE for forecasting model in r?

I want to do forecasting for air quality data, I have applied auto.arima and neural network models. R2, RMSE, MAPE, AIC and BIC are the criteria to evaluate and choose the best model.

I use for arima:

no <- ts(NO, start=c(2017,06,01), frequency=365)
model <- auto.arima(no, D=1)
finalmodel <- arima(no, order = c(0, 0, 2), seasonal = list(order = c(0,1,0), period = 365))
Forecastmodel<- forecast(finalmodel, h=365)

in this model I could have only AIC and BIC

and for Neural Network:

no <- ts(NO, start=c(2017,06,01), frequency=365)
model <- nnetar(no)
forecast <- forecast(model, h=365)

So, how can I find the criteria in r for each model? And is there any other recommendation for a forecasting model? Hopefully, I've used the right codes

Upvotes: 1

Views: 1224

Answers (1)

Rob Hyndman
Rob Hyndman

Reputation: 31810

Here is an example.

library(forecast)
# Create example data 
no <- ts(rnorm(100))
# Fit an ARIMA model
model <- auto.arima(no, D=1)
summary(model)
#> Series: no 
#> ARIMA(0,0,0) with zero mean 
#> 
#> sigma^2 estimated as 0.9299:  log likelihood=-138.26
#> AIC=278.52   AICc=278.56   BIC=281.13
#> 
#> Training set error measures:
#>                       ME      RMSE       MAE MPE MAPE      MASE        ACF1
#> Training set -0.09963973 0.9643283 0.7635588 100  100 0.6833166 -0.07970921
# Calculate forecasts
fc <- forecast(model)
# Compare forecasts against a test set
test <- ts(rnorm(20), start=101)
accuracy(fc, test)
#>                       ME      RMSE       MAE MPE MAPE      MASE        ACF1
#> Training set -0.09963973 0.9643283 0.7635588 100  100 0.6833166 -0.07970921
#> Test set     -0.23460384 0.8855929 0.7824913 100  100 0.7002595  0.14526351
#>              Theil's U
#> Training set        NA
#> Test set     0.8087404

Created on 2021-06-12 by the reprex package (v2.0.0)

Note that you do not have to re-estimate the model chosen by auto.arima(). It is already saved in model.

Given you appear to have daily data, you might consider other modelling options. See https://otexts.com/fpp2/weekly.html#daily-and-sub-daily-data for some suggestions.

Upvotes: 2

Related Questions