CODEWITHSUNDEEP
streammachine-learningdata-miningnormalization
T.Sh
T.Sh

Reputation: 390

normalization methods for stream data

I am using Clustream algorithm and I have figured out that I need to normalize my data. I decided to use min-max algorithm to do this, but I think in this way the values of new coming data objects will be calculated differently as the values of min and max may change. Do you think that I'm correct? If so, which algorithm shall I use?

Upvotes: 12

Views: 2436

Answers (2)

Vinicius Woloszyn
Vinicius Woloszyn

Reputation: 314

Instead to compute the global min-max based on the whole data, you can use a local nomarlization based on a sliding window (e.g. using just the last 15 secconds of data). This approach is very commom to compute Local Mean Filter on signal and image processing.

I hope it can help you.

Upvotes: 7

fatih
fatih

Reputation: 1395

When normalizing stream data you need to use the statistical properties of the train set. During streaming you just need to cut too big/low values to a min/max value. There is no other way, it's a stream, you know.

But as a tradeoff, you can continuously collect the statistical properties of all your data and retrain your model from time to time to adapt to evolving data. I don't know Clustream but after short googling: it seems to be an algorithm to help to make such tradeoffs.

Upvotes: 0

Related Questions