CODEWITHSUNDEEP
rforecasting
Shivam Sarin
Shivam Sarin

Reputation: 551

Accuracy Function

I am doing a TS Analysis. What is the difference between these two accuracies:

fit<-auto.arima(tsdata)
fcast<-forecast(fit,6)
accuracy(fcast)             #### First Accuracy

fit<-auto.arima(tsdata)
fcast<-forecast(fit,6)
accuracy(fcast,actual values)  #### Second Accuracy

How does the accuracy function work when I don't specify the actual values in the accuracy function as in the first case.

Secondly what is the right approach to calculate accuracy?

Upvotes: 2

Views: 16113

Answers (1)

Adi Sarid
Adi Sarid

Reputation: 819

In this answer I'm assuming you are using the function from the forecast package. The answer lies within accuracy's description:

Returns range of summary measures of the forecast accuracy. If x is provided, the function measures out-of-sample (test set) forecast accuracy based on x-f. If x is not provided, the function only produces in-sample (training set) accuracy measures of the forecasts based on f["x"]-fitted(f). All measures are defined and discussed in Hyndman and Koehler (2006).

In your case x being the second argument of the function. So, in short accuracy(fcst) provides an estimation of the prediction error, based on the training set.

For example: lets assume you have 12 months and predicting 6 ahead. Then if you use accuracy(fcst) you get the error of the model over the 12 months (only).

Now, let's assume x = real demand for the 6 months you are forecasting. And that you didn't use this data to build the Arima model. In this case accuracy(fcst, x) gives you the test set error, which is a better measure to what you will get in the future using this model (compared to the train set error).

The best practice is to use a test set error because this measure is less prone to bias (you will most likely get "better" prediction results on the training set then on a "hideout" test set, but these results will be a sort of "overfitting"). If you have a test set, you should use the test set as the second argument.

Upvotes: 6

Related Questions