CODEWITHSUNDEEP
c++clinuxfedora
Osiris Xu
Osiris Xu

Reputation: 713

How to compile a program to make it capable to use >4GB memory on 32-bit Linux?

The whole code is written in C, C++, and Fortran. Is it possible to make it to use more than 4GB memory. Now it is always crashed when it reaches 3GB memory.

If it is possible, how to set up the compiling options (or configure flags)?

We can use gcc, g++, ...or intel compilers

our OS: Fedora 12 x32

cat /proc/cpuinfo
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe nx lm constant_tsc arch_perfmon pebs bts aperfmperf pni dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm sse4_1 xsave lahf_lm tpr_shadow vnmi flexpriority
bogomips : 5319.72
clflush size : 64
cache_alignment : 64
address sizes : 36 bits physical, 48 bits virtual

Upvotes: 4

Views: 2112

Answers (4)

Potatoswatter
Potatoswatter

Reputation: 137850

You need to partition its working set into chunks of < 3 GB, and handle each chunk in a separate process. Connect the processes by pipes or sockets.

This is pretty similar to developing it into a network/cluster application, which might not be a bad idea if you want scalability.

Upvotes: 2

Zan Lynx
Zan Lynx

Reputation: 54335

I believe that if you put a file in a tmpfs filesystem (or hugetlbfs) and in your program map small (1 - 2 GB) pieces of it at a time, you can work with more than 4 GB of data at once.

The map operations aren't all that fast so you will take a performance hit if you jump through your memory too randomly.

Upvotes: 3

Dirk is no longer here
Dirk is no longer here

Reputation: 368351

Unfortunately you are hitting the limit that

R> 2^32
[1] 4294967296

which is your 4 gb limit. So you simply cannot index beyond for a single application, no matter what the OS.

This is one of the reasons many of us switched to 64 bit versions of our OSs. Linux has supported this since the late 1990s. Just switch to FC (or Ubuntu or ...) in 64bit.

One possible alternative is installing more RAM (which Linux will handle) and segmenting your task over several instances of the application, effectively running it in parallel. But that may not be worth the trouble...

Upvotes: 8

Jess
Jess

Reputation: 3049

32-bit integers can't count higher than 4Gb:

http://www.linuxquestions.org/questions/linux-general-1/32-bit-os-and-4gb-memory-limit-707762/

Upvotes: 1

Related Questions