CODEWITHSUNDEEP
apache-sparkmemory-managementdatagridrddignite
D V N
D V N

Reputation: 41

How do I compute Memory required to create igniteRDD

Can anyone please help me understanding, How is the Memory calculation done, to store A String, a long variable, and a int variable in IgniteRDD. I was going to forums and find different answers, and I am totally confused on how to calculate the Memory requirement for my application.

I am trying to calculate 48 Billion records with 2 Strings, 1 int and 2 long variables.

Any help will be greatly appreciated.

Regards, D V Nithin

Upvotes: 1

Views: 43

Answers (1)

Evgenii Zhuravlev
Evgenii Zhuravlev

Reputation: 3017

See https://apacheignite.readme.io/docs/capacity-planning

Also, for estimating memory size - I would recommend loading various sized data several times and measuring Memory consumed by it. It will help you to understand approximate needed memory size. These metrics will help you in this: https://apacheignite.readme.io/docs/memory-metrics

Upvotes: 1

Related Questions