CODEWITHSUNDEEP
rregressionsvmpredict
user21515
user21515

Reputation: 31

How to reproduce predict.svm() in R when doing regression?

I have trained a svm model in R using eps-regression and a radial kernel in the e1017 package. I can make predictions for new observations when using the function predict() but I'm interested in reproducing this outside a R environment. To do so I need to know how does R predict new observations.

There is an example of how to do so when using svm to do classification (here) but it doesn't work when doing regression due to the fact that the predictive function has a different formula: In the case of classification the predictive function is given by 

$\hat{f}(x)= \sum^N_{i=1} \hat{\alpha}_i y_i K(x,x_i)+\hat{\beta}_0 $,

where $y_i$ is already contained in m$coefs

whereas in the case of regression the predictive function is given by 

$\hat{f}(x)= \sum^N_{i=1} \hat{\alpha}_i K(x,x_i) $

both formulas according to Hastie, Tibshirani, Friedman (2001).

I have also read here that \hat{\alpha}_i are the support vectors so I tried to use them in my predictive function, instead of the coeficients used in the classification case.

As an example one could use:

library(e1071)

x <- seq(0.1, 5, by = 0.05) 
y <- log(x) + rnorm(x, sd = 0.2)

m   <- svm(y~x) 
new <- predict(m, x)

k<-function(x,x1,gamma){   
   return(exp(-gamma*sum((x-x1)^2))) }

f<-function(x,m){   
   return(t(m$SV) %*% as.matrix(apply(m$SV,1,k,x,m$gamma))) }

my.predict<-function(m,x){   
            apply(x,1,f,m)}

x<-as.matrix(x) 
my.predict(m,x)[1:10]
new[1:10]

Can someone explain where is this going wrong and/or recommend literature on the subject?

Thanks,
Marta

Upvotes: 1

Views: 817

Answers (1)

user21515
user21515

Reputation: 31

Here is the solution:

library(e1071)

x <- seq(0.1, 5, by = 0.05)
y <- log(x) + rnorm(x, sd = 0.2)
m   <- svm(y~x,scale=F,kernel="radial")
k<-function(x,x1,gamma){
  return(exp(-gamma*(x-x1)^2))
}
f<-function(x,m,xi){
  return(apply(xi,1,function(z) t(m$coefs) %*% (k(m$SV,z,m$gamma))))
}

my.predict<-function(m,x,xi){
  xi<-as.matrix(xi)
  return(f(x,m,xi)-m$rho)
}
my.predict(m,x,x)[1:10] 
new <- predict(m, x)
new[1:10]

Upvotes: 1

Related Questions