CODEWITHSUNDEEP
rtime-seriesbayesian
Sharma
Sharma

Reputation: 338

R bsts predictions are not consistent

Whenever I run the predict function multiple times on a bsts model using the same prediction data, I get different answers. So my question is, is there a way to return consistent answers given I keep my predictor dataset the same?

Example using the iris data set (I know it's not time series but it will illustrate my point)

iris_train <- iris[1:100,1:3]
iris_test <- iris[101:150,1:3]

ss <- AddLocalLinearTrend(list(), y = iris_train$Sepal.Length)

iris_bsts <- bsts(formula = Sepal.Length ~ ., data = iris_train, 

state.specification = ss, 
                  family = 'gaussian', seed = 1, niter = 500)
burn <- SuggestBurn(0.1,iris_bsts)

Now if I run this following line say, 10 times, each result is different:

iris_predict <- predict(iris_bsts, newdata = iris_test, burn = burn)
iris_predict$mean

I understand that it is running MCMC simulations, but I require consistent results and have therefore tried:

  1. Setting the seed in bsts and before predict
  2. Setting the state space standard deviation to near 0, which just creates unstable results.

And neither seem to work. Any help would be appreciated!

Upvotes: 1

Views: 1126

Answers (3)

Kelig
Kelig

Reputation: 1

I have come across the same issue. The problem comes from setting the seed within the model definition only.

To solve your problem, you have to set a seed within the predict function such as:

iris_predict <- predict(iris_bsts, newdata = iris_test, burn = burn, seed=X)

Hope this helps.

Upvotes: 0

R. Vessenes
R. Vessenes

Reputation: 26

I encountered the same problem. To fix it, you need to set the random seed in the embedded C code. I forked the packaged and made the modifications here: BSTS.

For package installation only, download bsts_0.7.1.1.tar.gz in the build folder. If you already have bsts installed, replace it with this version via:

remove.packages("bsts")
# assumes working directory is whre file is located
install.packages("bsts_0.7.1.1.tar.gz", repos=NULL, tyype="source")

If you do not have bsts installed, please install it first to ensure all dependencies are there. (This may require installing Rtools, Boom, and BoomSpikeSlab individually.)

This package version only modifies the predict function from bsts, all code should work as is. It automatically sets the random seed to 1 each time predict is called. If you want predictions to vary, you'll need to explicitly set the predict parameter each time.

Upvotes: 1

CPak
CPak

Reputation: 13581

You can make a function to specify seed each time (set.seed was unnecessary...):

reproducible_predict <- function(S) {
    iris_bsts <- bsts(formula = Sepal.Length ~ ., data = iris_train, state.specification = ss, seed = S, family = 'gaussian', niter = 500)
    burn <- SuggestBurn(0.1,iris_bsts)
    iris_predict <- predict(iris_bsts, newdata = iris_test, burn = burn)
    return(iris_predict$mean)
}

reproducible_predict(1)
[1] 7.043592 6.212780 6.789205 6.563942 6.746156
reproducible_predict(1)
[1] 7.043592 6.212780 6.789205 6.563942 6.746156

reproducible_predict(200)
[1] 7.013679 6.173846 6.763944 6.567651 6.715257
reproducible_predict(200)
[1] 7.013679 6.173846 6.763944 6.567651 6.715257

Upvotes: 0

Related Questions