serializing objects in C++ and storing as a blob type in mysql

Question

I am using mysql/C++ connector to connect to a mysql database. I have some complex data structures so I need to serialize those and save in the database.

I tried something like the following.

vector<int> vectorTest(10,100);
istream *blob = NULL;
ostringstream os;
int size_data = sizeof(vector<int>);

blob = new istringstream((char*)&vectorTest, istringstream::in | istringstream::binary);
string qry = "INSERT INTO vector(id,object) VALUES (?,?)";

prep_stmt = con->prepareStatement(qry);

prep_stmt->setInt(1,1);
prep_stmt->setBlob(2,blob);
prep_stmt->execute();

I just tried a small example here. However the vector object is not getting saved.

Alternatively can I can use the following approach.

ostringstream os;
int size_data = sizeof(vector<int>);
os.write((char*)&vectorTest, size_data);

However I don't know how to redirect the outputstream to an inputstream, because the setBlob() method needs an istream as the input parameter.

Can I know how to get any of this examples working ? If my approach is incorrect can anyone provide a code example or improve the given code segment ? Your immediate response is greatly appreciated.

Thanks

Answer 1

My solution to writing to a MySQL database was to use the Visitor design pattern and an abstract base class. I did not use the BLOB data structure, instead used fields (columns):

struct Field
{
    // Every field has a name.
    virtual const std::string    get_field_name(void) = 0;

    // Every field value can be converted to a string (except Blobs)
    virtual const std::string    get_value_as_string(void) = 0;

    // {Optional} Every field knows it's SQL type.
    // This is used in creating the table.
    virtual unsigned int         get_sql_type(void) = 0;

    // {Optional} Every field has a length
    virtual size_t               get_field_length(void) = 0;
};

I built a hierarchy including fields for numbers, bool, and strings. Given a Field pointer or reference, an SQL INSERT and SELECT statement can be generated.

A Record would be a container of fields. Just provide a for_each() method with a visitor:

struct Field_Functor
{
    virtual void    operator()(const Field& f) = 0;
};

struct Record
{
    void    for_each(Field_Functor& functor)
    {
      //...
      functor(field_container[i]);  // or something similar
    }
};

By using a more true Visitor design pattern, the SQL specifics are moved into the visitor. The visitor knows the field attributes due to the method called. This reduces the Field structure to having only get_field_name and get_value_as_string methods.

struct Field_Integer;

struct Visitor_Base
{
    virtual void process(const Field_Integer& fi) = 0;
    virtual void process(const Field_String& fs) = 0;
    virtual void process(const Field_Double& fd) = 0;
};


struct Field_With_Visitor
{
    virtual void    accept_visitor(Visitor_Base& vb) = 0;
};

struct Field_Integer
{
    void    accept_visitor(Visitor_Base& vb)
    {
        vb.process(*this);
    }
};

The record using the `Visitor_Base`:
struct Record_Using_Visitor
{
    void accept_visitor(Visitor_Base& vistor)
    {
        Field_Container::iterator   iter;
        for (iter = m_fields.begin();
             iter != m_fields.end();
             ++iter)
        {
            (*iter)->accept_visitor(rv);
        }
        return;
    }
};

My current hurdle is handling BLOB fields with MySQL C++ Connector and wxWidgets.

You may also want to add the tags: MySQL and database to your next questions.

Answer 2

You're going about this completely the wrong way. This isn't "serialization", in fact it's quite possibly the opposite of serialization -- it's just trying to write out a raw memory dump of a vector into the database. Imagine for a second that vector looked like something this:

struct vector_int {
  unsigned int num_elements;
  int* elements;
};

Where elements is a dynamically allocated array that holds the elements of the vector.

What you would end up writing out to your database is the value of num_elements and then the value of the pointer elements. The element data would not be written to the database, and if you were to load the pointer location back into a vector on a different run of your program, the location it points to would contain garbage. The same sort of thing will happen with std::vector since it contains dynamically allocated memory that will will be written out as pointer values in your case, and other internal state that may not be valid if reloaded.

The whole point of "serialization" is to avoid this. Serialization means turning a complex object like this into a sequence of bytes that contains all of the information necessary to reconstitute the original object. You need to iterate through the vector and write out each integer that's in it. And moreover, you need to devise a format where, when you read it back in, you can determine where one integer ends and the next begins.

For example, you might whitespace-delimit the ints, and write them out like this:

1413 1812 1 219 4884 -57 12 

And then when you read this blob back in you would have to parse this string back into seven separate integers and insert them into a newly-created vector.

Example code to write out:

vector<int> vectorTest(10,100);
ostringstream os;

for (vector<int>::const_iterator i = vectorTest.begin(); i != vectorTest.end(); ++i) 
{
  os << *i << " ";
}

// Then insert os.str() into the DB as your blob

Example code to read in:

// Say you have a blob string called "blob"

vector<int> vectorTest;
istringstream is(blob);
int n;

while(is >> n) {
  vectorTest.push_back(n);
}

Now, this isn't necessarily the most efficient approach, space-wise, since this turns your integers into strings before inserting them into the database, which will take much more space than if you had just inserted them as binary-coded integers. However, the code to write out and read in would be more complex in that case because you would have to concern yourself with how you pack the integers into a byte sequence and how you parse a byte sequence into a bunch of ints. The code above uses strings so that the standard library streams can make this part easy and give a more straightforward demonstration of what serialization entails.

Answer 3

boost has a serialization library (I have never used it tho)

or XML or JSON