CODEWITHSUNDEEP
c++visual-studio-2010
user1612986
user1612986

Reputation: 1415

c++ loading large amount of data at compile time

I have a C++ object which needs a huge amount of data to instantiate. For example:

  class object {
      public object() { 
          double a[] = { array with 1 million double element }; 
      /* rest of code here*/};
      private:
      /* code here*/    
  }

Now the data (i.e 1 million double numbers) is in a separate text file. The question: How can I put it after "double a[]" in an efficient way and eventually compile the code? I do not want to read the data at run time from a file. I want it compiled with the object. What can be a solution? Ideally I would like the data to sit in the separate text file as it presently resides and somehow also have an assignment like double a[] =..... above.

Is this possible? Thanks in advance!

Upvotes: 6

Views: 2414

Answers (3)

Yakk - Adam Nevraumont
Yakk - Adam Nevraumont

Reputation: 275200

One large problem with this design is that 1 million ints on the stack will probably blow the stack. What you probably want is to put the data on the data segment, or in some kind of resource that is stores in your binary file and can be loaded at run time. If you need more than one copy of the data, duplicate it into a std::vector at run time, so you know the data is on the free store (heap). Mayhap even use a shared_ptr to a std::array to reduce the chance of needless accidental duplication(or unique_ptr to reduce the chance of reference duplication).

4mb of data is not going to play all that well is all I am saying. And locality of reference on a 4mb array to your other variables is not going to be your biggest concern.

Depending in your compiled target platform and framework, there will be ways to stuff this kind of data into a binary resource. I've never done it for a multi-meg file, but here is the visual studio help on resource files: http://msdn.microsoft.com/en-us/library/7zxb70x7%28v=vs.80%29.aspx

Note that "the data being in the code" does not make it fundamentally faster to load (other than traversing the filesystem once to find it maybe). The OS still has to load the binary, and larger binaries take more time to load, and a big array of values will take up as much room in a binary as it does in a distinct file. The real advantage is that it isn't a file that can be "misplaced" relative to your executable, but resource fork/resource file/etc methods can deal with that.

As noted in the comments below, static const data (and global data) tends to be loaded into the data segment, which is distinct from both the heap (aka free store) and stack (aka automatic store). I forget what the standard calls it. I do know that a static local variable in a function will behave differently than a static or global non-local variable with regards to initialization order (global (static or not) data gets initialized fully prior to main starting, while static local is initialized the first time the function is called, if I remember correctly).

Upvotes: 1

SergV
SergV

Reputation: 1277

The answer of Luchian Grigore is quite correct. But compiler can have some limit on length of source code line. See for example https://stackoverflow.com/questions/10519738/source-line-length-limit

So try on your compiler. But I am afraid, more simple solution of your problem will be reading of huge data from file.

Upvotes: 0

Luchian Grigore
Luchian Grigore

Reputation: 258548

Something like:

class object
{
  public
  object(){ double a[] = { 
     #include "file.h"
  }; 
   /* rest of code here*/};
  private:
  /* code here*/    
}

The file has to be formatted correctly though - i.e. contain something like:

//file.h
23, 24, 40,
5, 1.1,

In general, you can use #include directives to paste content into files. I've seen virtual methods being pasted like that, if they were common for most derived classes. I personally don't really like this technique.

Upvotes: 10

Related Questions