CODEWITHSUNDEEP
optimizationmachine-learningdataset
Mohamed Ali JAMAOUI
Mohamed Ali JAMAOUI

Reputation: 14699

What is the advantage of the paperboat format in performance optimization of ML?

The paperBoat format claims to provide a better dataset representation for machine learning routines. I'd like to understand the nature of its optimization. I understand that using an integer representation for model attributes means a faster processing of the data set, what are the other improvements.

Also, how to tune an ML algorithm to work with this file format.

Upvotes: 0

Views: 80

Answers (1)

Nikita Astrakhantsev
Nikita Astrakhantsev

Reputation: 4749

I don't know if this format really provides better representation, but I can speculate why it can be more efficient.

First, as they state at format description, "Having data of the same precision consecutive enables hardware vectorization."; consider also wikipedia: "Vector processing techniques have since been added to almost all modern CPU designs".

Second, their format allows you to mix sparse and non-sparse features, but since all sparse features are placed consequently, it is possible to easily take them as a sparse matrix and optimize methods for learning like conjugate gradient.

how to tune an ML algorithm to work with this file format?

What do you mean by ML algorithm tuning? The learning algorithm doesn't know and doesn't need to know anything about file format of the dataset; and you can't increase or decrease accuracy if you know file format. In theory, you can speed up the concrete optimization algorithm (like Gradient descent) if you can rely on some properties of data (and, I guess, Ismion PaperBoat does it), but I don't think that you can tune it by yourself.

Upvotes: 1

Related Questions