CODEWITHSUNDEEP
c++memoryvector
Łukasz Lew
Łukasz Lew

Reputation: 50278

Why this program reserves and keeps 2 GB of memory instead of 1 GB?

#include <vector>
typedef std::vector<char> vc;
typedef std::vector<vc> vvc;

vvc f() {
  const int N = (1 << 15);
  return vvc(N, vc(N)); // 1 GB
}

int main () {
  vvc v; 
  v = f();
  while(true);  // Why 2GB allocated ?
  return 0;
}

Compiled with both clang -O2 and g++ -O2. Same behavior.

Edit: There are multiple ways to fix this code pointed in some of the answers. But my point is to understand this code. Of course there is a temporary object, but it should disappear at the semicolon and 1GB of memory should be returned to the system. The question intends to ask why it does not happen.

Edit 2: The destructor of temporary object is indeed called before semicolon.

Upvotes: 9

Views: 431

Answers (5)

Balog Pal
Balog Pal

Reputation: 17163

You have assignment in your code that we preach against in vain for good reason ;-)

If you used initialization, that allows RVO optimization, and as others experimented it actually seem working too.

With assignment the function return an object, then it must be copied to target and only then nuke the temporary. While with data flow anal it could be possibly optimized, it's a case that hard to catch and considered rare to use. Thus you suffer the performance penalty on top of all the rest.

EDIT: to see that the temporary is indeed nuked at the proper point I suggest using debugger and single-step into functions. Alternatively look at assy listing. For the abstract machine the dtor call must happen before the next instruction.

However, optimized code uses the 'as if' rule more liberally. If the code has no way to tell the difference, it may postpone it a little, and you may just experience that effect.

C++11 note: in C++11 vector gained another op= that moves from the rvalue. That would cure this particular case, but it will take time till all compilers close the gap and especially all classes gain moving stuff. Pre-11 code looking for moving would use swap() in place of =.

Upvotes: 4

user184968
user184968

Reputation:

Edit: There are multiple ways to fix this code pointed in some of the answers. But my point is to understand this code. Of course there is a temporary object, but it should disappear at the semicolon and 1GB of memory should be returned to the system. The question intends to ask why it does not happen.

On Windows I compiled your test with gcc and ran. And I see that the value of Private Bytes is increasing after start of the program and then starts decreasing and then finally stops changing. So on this OS memory is returned to the system. By the way I made use of Process Explorer to get information about Private Bytes.

I assume you did your test on Linux and on this OS memory is not returned to the system. If it is not used it is finally moved to a paging area if I am right.

Upvotes: 1

celtschk
celtschk

Reputation: 19721

My guess is that you're looking at the operating system display of how much memory is allocated to the process, and that your compiler doesn't yet support C++11's move assignment.

Therefore what happens is probably the following:

  • In your function, you allocate the vector (consuming 1GB of memory).
  • Then, in your assignment, that vector is copied (consuming anmother 1GB of memory, which is adjacent to the first block).
  • After that, the original vector is destroyed. That frees the memory for future allocations in your program, but since that memory is before the memory of the retained vector, and for most operating systems the memory allocated to a process must be contiguous, the process can not give it back to the operating system.

So the result is that the operating system allocates 2GB to your process, of which 1GB is allocated to the vector, and 1GB is free for future allocation within your program (but not to allocations in another process).

Upvotes: 4

Boris
Boris

Reputation: 622

I've just tested on win32, vc7.

Your code allocates 2Gb. If change to this:

int main () {
  vvc v = f();
  while(true);

It needs only 1Gb.

I suppose the reason - is a copy operation between vectors.

v = f() - invokes constructor. Your case - default c'tor and copy (operator =) into empty object. Coping needs 2 Gb.

Internal actions of f() (creating vector and return) can use RVO, and there are no any coping and extra allocating.

Upvotes: 2

stefan bachert
stefan bachert

Reputation: 9608

I think the return value will be copied. Than temporary 2 copies of vvc exists. Each with 2^30 chars (= 1GB)

To my knowledge the compiler is allowed but not required to remove the copy operation

I found this link http://en.wikipedia.org/wiki/Return_value_optimization

Upvotes: 0

Related Questions