CODEWITHSUNDEEP
rtime-seriesforecastingdata-preprocessing
Krellex
Krellex

Reputation: 753

Data values NaN after normalization is applied and structure changes

I am trying to normalize my values before using the neural net function however, when normalizing my values they turn into NaN and I go from how the values are inside my dataDelay variable to a single observation with 88 variables instead of the original amount.

library(neuralnet)
library(grid)
library(MASS)
library(ggplot2)
library(reshape2)
library(gridExtra)
library(neuralnet)

normalize <- function(x){
  return ((x - min(x)) / (max(x) - min (x)))
}


data <-
  structure(
    list(
      `USD/EUR` = c(
        1.373,
        1.386,
        1.3768,
        1.3718,
        1.3774,
        1.3672,
        1.3872,
        1.3932,
        1.3911,
        1.3838,
        1.4171,
        1.4164,
        1.3947,
        1.3675,
        1.3801,
        1.3744,
        1.3759,
        1.3743,
        1.3787,
        1.3595,
        1.3599,
        1.3624,
        1.3523,
        1.3506,
        1.3521
      )
    ),
    row.names = c(NA,-25L),
    class = c("tbl_df",
              "tbl", "data.frame")
  )


#time series delay
dataDelay <- embed(data[[1]], 4)[, 4:1]
#normalizing values
currencyNorm <- as.data.frame(lapply(dataDelay, normalize))

Input <- subset(dataDelay, select = c(dataDelay[1], dataDelay[2], dataDelay[3]))
Output <- subset(dataDelay, select = c(dataDelay[4]))

##NN model
currency_model <- neuralnet(Output~Input, hidden = 1, data = dataDelay)

dataDelay output:

      [,1]   [,2]   [,3]   [,4]
[1,] 1.3730 1.3860 1.3768 1.3718
[2,] 1.3860 1.3768 1.3718 1.3774
[3,] 1.3768 1.3718 1.3774 1.3672
[4,] 1.3718 1.3774 1.3672 1.3872
[5,] 1.3774 1.3672 1.3872 1.3932
[6,] 1.3672 1.3872 1.3932 1.3911

After normalizing:

NaN. NaN..1 NaN..2 NaN..3 NaN..4 NaN..5 NaN..6 NaN..7 NaN..8 NaN..9 NaN..10 NaN..11 NaN..12
1  NaN    NaN    NaN    NaN    NaN    NaN    NaN    NaN    NaN    NaN     NaN     NaN     NaN...

Applying to full dataset:

normalize <- function(x){
  return ((x - min(x)) / (max(x) - min (x)))
}


exchangeData <- read.csv("ExchangeUSDcsv.csv")
data <- exchangeData[,3]
data <- as.data.frame(data)

currencyNorm <- embed(normalize(data[[1]]), 4)[, 4:1]
head(currencyNorm)

currencyNorm <- as.data.frame(currencyNorm)

[ PROBLEM AFTER APPLYING SOLUTION CODE ON FULL DATASET ]

Full dataset: https://www.dropbox.com/s/17exy1968lsidsc/ExchangeUSDcsv.csv?dl=0

Output when applied to full dataset:

     [,1] [,2] [,3] [,4]
[1,]   NA   NA   NA   NA
[2,]   NA   NA   NA   NA
[3,]   NA   NA   NA   NA
[4,]   NA   NA   NA   NA
[5,]   NA   NA   NA   NA
[6,]   NA   NA   NA   NA

Upvotes: 2

Views: 894

Answers (2)

Rui Barradas
Rui Barradas

Reputation: 76460

I believe that the simplest way is to normalize from the original data.

currencyNorm <- embed(normalize(data[[1]]), 4)[, 4:1]

But if this is a XY Problem then maybe the following code is more to the point.
It builds a neural net from currencyNorm with one hidden layer. To extract the subsets Input and Output is not needed, the formula V4 ~ . models the 4th column V4 on all other columns.

library(neuralnet)

currencyNorm <- embed(normalize(data[[1]]), 4)[, 4:1]
currencyNorm <- as.data.frame(currencyNorm)
##NN model
currency_model <- neuralnet(V4 ~ ., hidden = 1, data = currencyNorm)

In order to predict with the model, you will have to have 3 values, one for each of V1, V2 and V3.

set.seed(2021)   # make the results reproducible
new <- data.frame(V1 = runif(1), V2 = runif(1), V3 = runif(1))

predict(currency_model, newdata = new)
#          [,1]
#[1,] 0.6168927

Or a new data set with many rows.

new2 <- data.frame(V1 = runif(5), V2 = runif(5), V3 = runif(5))
predict(currency_model, newdata = new2)

Upvotes: 1

Ronak Shah
Ronak Shah

Reputation: 389055

If you want to do column wise normalisation of matrix use apply :

currencyNorm <- data.frame(apply(dataDelay, 2, normalize))

To normalize complete data as whole :

currencyNorm <- normalize(dataDelay)

Upvotes: 2

Related Questions