CODEWITHSUNDEEP
c++linuxfilegccbinary
solinent
solinent

Reputation: 1653

How to write binary data to a file so that it can be read fast back?

I need to write a file format that writes out data to a file and can read it back.

It should be able to read the data back fairly fast, which should involve blitting a large block of data into a std::vector (since their storage is always implemented contiguously).

However, when writing the file, I have no idea how to enforce constraints on alignment and size of ints and other types.

How should this be done? I'm using gcc on buntu linux on Q6600 (x86).

Just as an example:

struct Vertex
{
  float point [3];
  float normal [3];
  float texcoord [2];
}

Later on, the data is stored in a std::vector<Vertex>. I have thought about using __attribute__ and packing / aligning it so that it will be more portable across different platforms.

edit: I've already made a specification, and I intend to use it. The largest bits of data are the Vertex and Indices, so those will be read as big blocks, and for example (one part of a larger spec): A VertexGroup is a group of vertices who share a characteristic. They can only hold one material at a time, so there should be many of them contained in a mesh.

<uint> thisid # Of this VertexGroup
<string> name
<uint> materialId # A material
<uint> vertexCount
for (vetexCount):
    <3xfloat> point
    <3xfloat> normal
    <2xfloat> texcoord
<uint> triangleCount
for (triangleCount):
    <3xuint> indices

Upvotes: 2

Views: 982

Answers (2)

Sanjaya R
Sanjaya R

Reputation: 6416

If its just POD ( plain old data ), w/o pointers, then you can just fwrite and fread. That assumes of course that you are absolutely going to read back the exact same format as before, on the same architecture.

Consider boost serialization.

Upvotes: 1

JSBձոգչ
JSBձոգչ

Reputation: 41378

This will depend on your compiler and platform. As far as I know, there is no way to enforce this in a completely cross-compiler and cross-platform manner, without defining lots of macros of your own.

However, both VC++ and GCC (the big two) support the #pragma pack directive, which will allow you to define the alignment and packing for your struct. See http://msdn.microsoft.com/en-us/library/2e70t5y1.aspx or http://gcc.gnu.org/onlinedocs/gcc/Structure_002dPacking-Pragmas.html.

With that in mind, you can use #pragma pack to define how your structure is aligned, then fread() or similar to simply blit the bytes from the file to memory. You may want to prefix the list with the list length so that you can allocate the memory for the whole list at once, then load the entire file using a single I/O call.

Upvotes: 2

Related Questions