Stack Memory /STACK

Question

For the program I'm working on, I frequently need to read input from a text file which contains hundreds of thousands of integers. For the time being, I'm reading a handful of values and storing them in a vector. Whenever a value I need is not in the vector, I read from the input file again and flush out the old values to make room for the values I'm currently reading in.

I'd like to avoid a situation where I constantly need to read from the input file and I'm wondering how many values I can store in my vector before there will be a problem. max_size() returns 1073741823, so I'm thinking that I can store that many elements but I'm wondering where that memory is being used and if it's a good idea to have a vector that large.

When you create a vector as so:

int main(){
std::vector<int> vec;
vec.push_back(3);
vec.push_back(4);

return 0;
}

Is that vector now using stack memory? Since your vector contains 2 ints, does that mean that 8 bytes of stack memory is being used?

According to MSDN docs:

For x86 and x64 machines, the default stack size is 1 MB.

That does not seem like a lot of memory. What is an example of a situation where you would want to increase the stack memory? Is there any way in Visual Studio to monitor exactly how much stack and heap memory are currently being used?

Is there anything I can do to prevent constant reading from the input file in a situation like this?

Answer 1

Is that vector now using stack memory?

The vec object is on the stack, but it internally allocates its memory on the heap as it grows

EDIT

Also, instead of reading all the file and storing it in a vector, you could try using a memory mapped file. From what I understand (not having used them myself), you would benefit from page caching and file reading in kernel mode (as the OS will manage the loading of the file on demand).

Note that this is merely a suggestion on where to pursue your investigation (I think that it might be appropriate, but I am not familiar enough with memory mapped files to tell you more)

Answer 2

vector stores elements in the heap, not the stack. Whether you should really allocate that much heap memory is a different matter, but you won't blow your stack.