CODEWITHSUNDEEP
javaarraysstoring-information
anonymous_coder
anonymous_coder

Reputation: 39

Java overall arrays limit?

I am implementing a program that has about 2,000,000 (2 million) arrays each of size 16,512 (128 x 129) of integers. I only need to call 200 arrays at a time (that is 3.3 MB), but I wonder if I can expand the program to have more than 2 million (say 200 million) but I still only need to call 200 arrays at a time. So what is the limit of making more and more arrays while I don't use more than 200 arrays at a time?

Upvotes: 0

Views: 751

Answers (3)

duffymo
duffymo

Reputation: 308743

I highly doubt that, unless you're running on a 64 bit machine with a lot of RAM and a very generous heap.

Let's calculate the memory you'll need for your data: 

2,000,000*128*129*8/1024/1024/1024 = 30.8GB.

You'll need additional RAM for the JVM, the rest of your program, and the operating system.

Sounds like a poorly conceived solution to me.

If you mean "I only have 200 arrays in memory at a time" you can certainly do that, but you'll have to move the rest out to secondary storage or a relational database. Query for them, use them, GC them. It might not be the best solution, but it's hard to tell based on the little you've posted.

Update:

Does "trigger" mean "database trigger"?

Yes, you can store them on the disk. I can't guarantee that it'll perform. Your hard drive can certainly handle 30GB of data; it's feasible that it'll accomodate 300GB if it's large enough.

Just remember that you have to think about how you'll manage RAM. GC thrashing might be a problem. A good caching solution might be your friend here. Don't write one yourself.

What happens if that hard drive fails and you lose all that data? Do you back it up? Can your app afford to be down if the disk fails? Think about those scenarios, too. Good luck.

Upvotes: 3

NPE
NPE

Reputation: 500167

As long as you don't keep references to arrays you no longer need, there is no hard limit. Old arrays will automatically get garbage collected, so you can keep allocating and abandoning arrays pretty much ad infinitum.

There is, of course, a limit on how many arrays you can keep around at any given time. This is limited by the amount of memory available to the JVM.

Upvotes: 0

erikxiv
erikxiv

Reputation: 4075

As long as you increase max heap size to make sure your application doesn't run out of memory, you shuold be fine.

Upvotes: 0

Related Questions