CODEWITHSUNDEEP
rust
Matthieu M.
Matthieu M.

Reputation: 300129

Maximum size of an array in 32 bits?

According to the Rust Reference:

The isize type is a signed integer type with the same number of bits as the platform's pointer type. The theoretical upper bound on object and array size is the maximum isize value. This ensures that isize can be used to calculate differences between pointers into an object or array and can address every byte within an object along with one byte past the end.

This obviously constrain an array to at most 2G elements on 32 bits system, however what is not clear is whether an array is also constrained to at most 2GB of memory.

In C or C++, you would be able to cast the pointers to the first and one past last elements to char* and obtain the difference of pointers from those two; effectively limiting the array to 2GB (lest it overflow intptr_t).

Is an array in 32 bits also limited to 2GB in Rust? Or not?

Upvotes: 12

Views: 4251

Answers (2)

Veedrac
Veedrac

Reputation: 60187

The internals of Vec do cap the value to 4GB, both in with_capacity and grow_capacity, using

let size = capacity.checked_mul(mem::size_of::<T>())
                   .expect("capacity overflow");

which will panic if the pointer overflows.

As such, Vec-allocated slices are also capped in this way in Rust. Given that this is because of an underlying restriction in the allocation API, I would be surprised if any typical type could circumvent this. And if they did, Index on slices would be unsafe due to pointer overflow. So I hope not.

It might still not be possible to allocate all 4GB for other reasons, though. In particular, allocate won't let you allocate more than 2GB (isize::MAX bytes), so Vec is restricted to that.

Upvotes: 7

Daniel
Daniel

Reputation: 16464

Rust uses LLVM as compiler backend. The LLVM instruction for pointer arithmetic (GetElementPtr) takes signed integer offsets and has undefined behavior on overflow, so it is impossible to index into arrays larger than 2GB when targeting a 32-bit platform.

To avoid undefined behavior, Rust will refuse to allocate more than 2 GB in a single allocation. See Rust issue #18726 for details.

Upvotes: 4

Related Questions