CODEWITHSUNDEEP
javaapache-commonsapache-commons-math
unnik
unnik

Reputation: 1153

Memory Requirement of Commons Math: The Apache Commons Mathematics Library

I would like to know if all the operations in Apache Commons Math are in memory operations. I am particularly interested in OLSMultipleLinearRegression and using this for big data.

Also is there any existing JAVA API for running regression on big data.

Upvotes: 0

Views: 125

Answers (2)

Phil Steitz
Phil Steitz

Reputation: 644

It is correct that OLSMultipleLinearRegression stores the full design matrix in memory. There is another OLS implementation class, however, MillerUpdatingRegression, in the same Commons Math package (version 3.0 onward) that does not store the input dataset in memory.

Upvotes: 0

Stephen C
Stephen C

Reputation: 718788

My reading of the javadocs for the OLSMultipleLinearRegression class is that it is entirely in memory. For a start, the model and the sample data must be supplied as primitive arrays which precludes any possibility of representations that don't fit in memory ... at least at the point where you provide the inputs.

Upvotes: 1

Related Questions