Merging smaller chunks of data into one big section of memory

Question

I have code like this:

ByteArray ret;
ret.resize( MAX( body_left, tmp_read.size() ) );
while ( body_left > 0 ) {
    ByteArray::Write r = tmp_read.write();
    int rec = 0;
    err = connection->get_partial_data( r.ptr(), MIN( body_left, tmp_read.size() ), rec );
    if ( rec > 0 ) {
        ByteArray::Write w = ret.write();
        copymem( w.ptr(), r.ptr(), rec );
        body_left -= rec;
    }
}

I find it challenging to understand this code. A few questions:

Is ret.resize(MAX(body_left,tmp_read.size())); allocating the ByteArray of highest body_left or tmp_read.size()?

In ByteArray::Write r = tmp_read.write(); does r become a pointer to location in space that is going to be used to write data?

In ByteArray::Write w = ret.write();, does w become a pointer like r in the previous question?

Also, in this line:

copymem(w.ptr(),r.ptr(),rec);

As I understand this line, all of the data that is gathered under pointer r is copied to location under the pointer w. The problem is that they are different size, how to move pointer w.ptr() to keep data intact and in correct order? Or is w.ptr() is a pointer to function and this should not be a problem.

---

Extra context:

Method get_partial_data returns chunks of data - lets say 20, 20, and 10 bytes each. Variable ret is supposed to be 50 bytes long and have those chunks merged into one ByteArray.

Unfortunately I cannot find the definition of ByteArray in this project, so I guess it's part of another library (libGL maybe?).

I know this question is not very precise and I am making a leap of faith, but if anyone can assist me I would be grateful.

Original class and project this code was taken from:

https://github.com/okamstudio/godot/blob/master/core/io/http_client.cpp

Lines 503-516.

It's in different shape, as I already have applied dirty hack (that does not work to well).

Answer 1

To wrap everything I (we) have learnt so far:

Original code was as follows:

    ByteArray::Write r = tmp_read.write();
    int rec=0;
    err = connection->get_partial_data(r.ptr(),MIN(body_left,tmp_read.size()),rec);
    if (rec>0) {
        ByteArray ret;
        ret.resize(rec);
        ByteArray::Write w = ret.write();
        copymem(w.ptr(),r.ptr(),rec);
        body_left-=rec;
        if (body_left==0) {
            status=STATUS_CONNECTED;
        }
        return ret;
    }

Where copymem is a macro that looks as follows:

    #define copymem(m_to,m_from,m_count) \
    do { \
      unsigned char * _from=(unsigned char*)m_from; \
      unsigned char * _to=(unsigned char*)m_to; \
      int _count=m_count; \
      for (int _i=0;_i<_count;_i++) \
        _to[_i]=_from[_i]; \
    } while (0);

Using everything that remyable explained to me I dropped macro usage, because as I understand w.ptr() returns section of the memory and macro always starts coping from the first byte of that section. Macro do not support offset-ing nor passing pointers.

End result looks as follows:

    ByteArray ret;
    ret.resize(MAX(body_left,tmp_read.size()));
    ByteArray::Write w = ret.write();
    unsigned char * _to = (unsigned char*) w.ptr();
    int _offset = 0;
    while (body_left > 0) {
        ByteArray::Write r = tmp_read.write();
        int rec=0;
        err = connection->get_partial_data(r.ptr(),MIN(body_left,tmp_read.size()),rec);
        if (rec>0) {
            unsigned char * _from=(unsigned char*)r.ptr();
            for (int _i=0;_i<rec;_i++)
                _to[_offset+_i]=_from[_i];
            _offset += rec;
            body_left-=rec;
        }
    }
    if (body_left==0) {
        status=STATUS_CONNECTED;
    }
    return ret;

Anyone can confirm this is viable solution or even suggest improvement?

Update. I found that I can actually move w.ptr() by offset for a macro, alternative code looks as follow:

    ByteArray ret;
    ret.resize(MAX(body_left,tmp_read.size()));
    ByteArray::Write w = ret.write();
    int _offset = 0;
    while (body_left > 0) {
        ByteArray::Write r = tmp_read.write();
        int rec=0;
        err = connection->get_partial_data(r.ptr(),MIN(body_left,tmp_read.size()),rec);
        if (rec>0) {
            copymem(w.ptr()+_offset,r.ptr(),rec);
            body_left-=rec;
            _offset += rec;
        }
    }
    if (body_left==0) {
        status=STATUS_CONNECTED;
    }
    return ret;

Comments/opinions?

Answer 2

Is ret.resize(MAX(body_left,tmp_read.size())); allocating the ByteArray of highest body_left or tmp_read.size()?

MAX most likely is a macro that returns the greater of the two arguments. The line ret.resize(MAX(body_left,tmp_read.size())); ensures that ret is large enough for whatever writing operations that may occur.

In ByteArray::Write r = tmp_read.write(); does r become a pointer to location in space that is going to be used to write data?

In ByteArray::Write w = ret.write();, does w become a pointer like r in the previous question?

Write is a class defined on line 187. write() is a function defined on line 209 that returns a Write object, not a pointer. Therefore r and w are never pointers.

class Write {
  // ...
};

Write write() {
  Write w;
  // ...
  return w; 
}

Also, in this line:

copymem(w.ptr(),r.ptr(),rec);

As I understand this line, all of the data that is gathered under pointer r is copied to location under the pointer w. The problem is that they are different size, how to move pointer w.ptr() to keep data intact and in correct order? Or is w.ptr() is a pointer to function and this should not be a problem.

copymem is a macro defined on line 36.

#define copymem(m_to,m_from,m_count) \
do { \
  unsigned char * _from=(unsigned char*)m_from; \
  unsigned char * _to=(unsigned char*)m_to; \
  int _count=m_count; \
  for (int _i=0;_i<_count;_i++) \
    _to[_i]=_from[_i]; \
} while (0);

All this code appears to do is copy the contents of m_from to m_to. get_partial_data feeds the amount to read into rec, which is passed to copymem as m_count.