Allocating a std::vector with custom allocator and type unknown at runtime

Question

For the question, suppose the following code flow:

void * some_data = allocate_memory(const std::string & type)

fetch_data_from_somewhere(some_data);

send_data_to_client(some_data);

allocate_memory will just allocate an empty message, which will be filled (including potential further allocation of dynamic memory) via fetch_data_from_somewhere and then sent somewhere by send_data_to_client. The client then casts the data to the real type (which only he knows about).

I now have to write the function to allocate_memory given only a string identifying the type. With that information, I can get a data struct describing the message (e.g.: It's 80 bytes long, there is an integer in the first 4 bytes, a std::vector afterwards, etc.).

Imagine I know that I have to allocate a std::vector<T> where T is a type unknown to me (again, I know a string identifier of the type and can access information regarding its structure, size, etc.).

If I use void * some_memory = calloc(sizeof(std::vector<T>), 1), this seems to work for all T (except for booleans) with gcc, clang and MSVC++ compilers. Later on, the client can just do static_cast<std::vector<T>>(some_memory) and using all vector operations seems to work. This seems somewhat reasonable, although I can't tell whether it's somehow guaranteed to work.

However, if I want to pass my own allocator, it seems to me that I'm out of luck: I'd expect that the zero-allocated memory will not work, but I can't really do void * = new std::vector<T, MyAllocator>(0, my_allocator) or similar, because I cannot know T. If I needed a vector of type std::vector<T *> instead, I'd assume it would be possible to just use a std::vecotr<void *> since the data is untyped in memory anyways.

Is there any way to allocate std::vector<T, MyAllocator> with unknown type T at compile time (cross-platform...)?

It's clearly not intended use nor clean code, but for the sake of the question, let's suppose I can't do anything about it.

Answer 1

Just allocating a bunch of memory does not a vector make. Your calloc solution is wrong because it doesn't actually construct a vector, plus the memory you would construct it in has not been properly aligned. The fact that you've repeatedly observed this method "working" is pure bad luck; you can, in theory and in practice, expect it to catastrophically blow up.

Even if all you needed to do was allocate some memory, you still couldn't do it with a vector of unknown type because you don't know how large a vector<unknown-type> is. Certainly you can't construct one. Even your vector<T*>→vector<void*> idea doesn't work: you're forgetting that each vector specialisation is a completely distinct type, and relationships between value types do not create relationships between the resulting container types.

Your approach is too naive to successfully generate C++ objects in general. There is a reason that C++ gives us operator new and allocators that work in harmony. The old C-style approach of "give memory, use memory" just isn't enough. The fact that you then want to "send data to client" is a red flag too: you can't serialise complex C++ objects by dumping their immediate bytes to a network.

Finally, my obligatory warning that C++ is an abstraction, and that if you are trying to write "clever" source code that manipulates objects at the byte level, you need to be very careful. C++ expects you to literally tell it when you want an object to be created, and literally tell it when you want that object to be destroyed; failing to do so then pretending that the object exists is prone to all manner of weirdness, since compilers (including but not limited to during their optimisation phase) take full advantage of the complex, heavily semantic rules that make up the language. In general you really do need to "tell it what you mean" rather than trying to go behind its back. (Some limited exceptions to this rule exist, mostly leniencies surrounding objects of built-in type, and the ability to alias existing, properly-constructed objects using a char*; none of these exceptions apply to your requirement.)

Stick with the construction/allocation facilities provided, and write a serialiser.