CODEWITHSUNDEEP
c++memory-mapped-filesmemory-mapping
duong_dajgja
duong_dajgja

Reputation: 4276

Memory mapping a huge file in 32 bit software running on 64 bit OS

I have performance problems with reading huge files. I would like to use memory mapping to solve the problems. Input files are up to 10 GB of size, my program (written in C++) is built with 32 bit configuration, the target system is Windows 64 bit and has 24 GB of RAM. Is this possible to map whole a huge file to memory? If yes, how can my program access to the "high" address area (> 0xFFFFFFFF, theoritically, because my program is 32 bit, so pointers are 32 bit also)?

Upvotes: 0

Views: 1750

Answers (2)

Wyzard
Wyzard

Reputation: 34571

You can't. A 32-bit program uses 32-bit pointers that can't go past 4GB, even when running on a 64-bit OS.

One thing that may help a little, though, is to link your program with the /LARGEADDRESSAWARE option. By default, 32-bit Windows programs are only able to use 2GB of the address space, because some programs assume that a pointer's uppermost bit is always zero and use it as a flag to store additional information. That'd break if Windows started allocating memory above 2GB, so you have to link your program with a special option that tells Windows your program doesn't abuse that bit. This lets your program use the full 4GB of address space instead of being limited to 2GB.

Upvotes: 0

ElderBug
ElderBug

Reputation: 6145

In a 32-bit program, you will never be able to map the whole file at once in a single process, since the address space is 2GB long. What is possible is to map only part of the file at a given moment by playing with MapViewOfFile parameters. It is also possible to map the whole file at once with multiple processes mapping 1GB each (since separate processes have separate address spaces), but that would be really impractical to use.

Upvotes: 0

Related Questions