confused about max size of std::vector

Question

I ran into an issue in my code with std::vector, giving me: vector<T> too long. I'm using a vector of char in this case. The code is treating 3D (tomography) images, so I have a lot of voxels. I have exactly the same issue on windows using the VS compiler as on Mac using CLANG, not tested gcc yet.

To inspect the issue, I added the following lines:

    printf("max vector size %u\n", v.max_size() );
    printf("PTRDIFF_MAX %u, INTMAX_MAX %u\n", PTRDIFF_MAX, INTMAX_MAX );

Which gave me:

    max vector size 4294967295
    PTRDIFF_MAX 4294967295, INTMAX_MAX 4294967295

I have two questions about this:

1. Why is the max value so small? it looks like the max of an uint32. I was expecting it to be more in the range of size_t, which should be 18446744073709551615, right?
2. Why am I getting the vector<T> too long when my vector surpasses 2147483648 (i.e. half the stated maximum) number of values?

[edit]

Some responses:

• Thank you for pointing out my printf mistakes. I've converted to std::cout and now I get (which makes more sense on a 64bit system):

    max_size: 9223372036854775807
    PTRDIFF_MAX: 9223372036854775807
    INTMAX_MAX: 9223372036854775807
  

• The error vector<T> too long is a runtime error

• I traced it back to the line where I resize the vector

• When I use a vector of uint32_t instead of char, the problem arrises at the same vector length (at 4 times the memory usage).

• I have more than sufficient RAM, however, I don't know how allocation works upon a resize request. Perhaps, there isn't a sufficiently large continuous block of RAM available?

Answer 1

Why is the max value so small? it looks like the max of an uint32.

That is to be expected on 32 bit systems.

I was expecting it to be more in the range of size_t, which should be 18446744073709551615, right?

If PTRDIFF_MAX is 4294967295, then I find it surprising that SIZE_MAX would be as much as 18446744073709551615. That said, I also would find it surpising that PTRDIFF_MAX was 4294967295.

You're seeing surprising and meaningless output because the behaviour of the program is undefined which is because you used the wrong format specifier. %u is for unsigned int and only for unsigned int. %td specifier is for std::ptrdiff_t, PRIdMAX macro expands to the specifier for std::intmax_t and %zu is for std::size_t.

I recommend learning to use the C++ iostreams. It isn't quite as easy to accidentally cause undefined behaviour using them as it is when using the C standard I/O.

Why am I getting the vector too long when my vector surpasses 2147483648 (i.e. half the stated maximum) number of values?

I don't know what getting "vector too long" means, but it's typical that you don't have the entire address space available to your program. It's quite possible that half of it is reserved to the kernel.

max_size doesn't necessarily take such system limitations into consideration and is a theoretical limit that is typically not achievable in practice.