CODEWITHSUNDEEP
c++c++11stl
SteveH
SteveH

Reputation: 243

How to treat a std::vector like a C buffer efficiently?

With C buffer, I often do like this:

BYTE buffer[MAX_SIZE];
int dataSize = 0;

while (appRunning()) 
{
    dataSize += Receive(buffer + dataSize, MAX_SIZE - dataSize);

    int processedSize = ProcessBuffer(buffer, dataSize);
    ASSERT(processedSize <= dataSize);
    dataSize -= processedSize;
    memmove(buffer, buffer + processedSize, dataSize);
};

Is it possible to do so with a std::vector without losing much performance?

EDIT: I have found a way to replace a raw C buffer by a std::vector.

std::vector<BYTE> vbuf;
vbuf.reserve(MAX_SIZE); // allocated at once

while (appRunning()) 
{
    int pendingSize = GetPendingDataSize(); // from a socket
    if (pendingSize > vbuf.capacity())
        pendingSize = vbuf.capacity();

    vbuf.resize(pendingSize);
    int recvSize = Receive(vbuf.data(), vbuf.size());
    ASSERT(recvSize < vbuf.size());
    int processedSize = ProcessBuffer(vbuf.data(), vbuf.size());
    std::rotate(vbuf.begin(), vbuf.begin() + processedSize, vbuf.end());
    vbuf.resize(vbuf.size() - processedSize);
};

Actually, in my practical usage, receiving data and processing data may be done in multithread. So by using vector, I do not need to manage buffer's allocation, data size and buffer capacity manually. Compares to the C buffer, the performance penalty here is at vbuf.resize() calls. But I think that penalty is insignificant. Any better way is appreciated.

Upvotes: 6

Views: 4014

Answers (5)

benf
benf

Reputation: 1005

If you want behavior similar to C (that is, you want to guarantee the vector won't ever free or allocate more memory for the underlying vector), a nice solution is to use Boost's static_vector. It statically allocates the underlying buffer but otherwise it acts like a normal vector. 

boost::static_vector<BYTE, MAX_SIZE> buffer;

For this type of activity, however, you might also look into std::queue or boost::cirular_buffer and see if one of those fits your needs.

Upvotes: 0

Caleth
Caleth

Reputation: 62686

It's perfectly possible to replace it with std::vector, or equivalently std::array. You don't need to resize the vector, so don't.

std::vector<BYTE> buffer(MAX_SIZE);
BYTE * start = buffer.data();
int dataSize = 0;

while (appRunning()) 
{
    dataSize += Receive(start, MAX_SIZE - dataSize);

    int processedSize = ProcessBuffer(buffer.data(), dataSize);
    ASSERT(processedSize <= dataSize);
    dataSize -= processedSize;
    start = std::copy_n(buffer.data() + processedSize, dataSize, buffer.data());
};

Upvotes: 0

Maxim Egorushkin
Maxim Egorushkin

Reputation: 136246

When receiving messages over a TCP connection the last message in the buffer may be incomplete. After processing complete messages, people often just memmove that last incomplete message to the beginning of the buffer.

Another strategy is to use a "smart" ring-buffer to avoid that memmove and also avoid data wrapping over the ring buffer thus creating discontinuity. To make a "smart" ring-buffer allocate memory for the buffer using mmap and map the same region of pages twice with no gaps in between. This way reading past the end of the buffer continues reading it from the start, preventing the discontinuity inherent when using a regular ring-buffer.

Using std::vector for network buffers is less than ideal because resizing a vector initialises its elements that are later overwritten by recv call. That initialization is unnecessary for such buffers.

Upvotes: 2

Philipp Cla&#223;en
Philipp Cla&#223;en

Reputation: 43969

Since C++11, C arrays have been for the most parts replaced by std::array:

std::array<BYTE, MAX_SIZE> buffer;

It is only a thin wrapper around C arrays. So, from a performance perspective, you do not loose anything. There are nicer to work with, however.

But like C arrays, they only work for fixed-time arrays where the size is known at compile time. It is not possible to resize them. But if I read your question correctly, you do not need that extra flexibility, here.

Upvotes: 1

lgromanowski
lgromanowski

Reputation: 89

Writing from memory, please check if it works for you:

std::vector<BYTE> buffer;
buffer.reserve(MAX_SIZE);
int dataSize = 0;

while (appRunning()) 
{
    dataSize += Receive(&buffer[0] + dataSize, MAX_SIZE - dataSize);

    int processedSize = ProcessBuffer(&buffer[0], dataSize);
    ASSERT(processedSize <= dataSize);
    dataSize -= processedSize;
    memmove(&buffer[0], &buffer[0] + processedSize, dataSize);
};

Upvotes: -1

Related Questions