CODEWITHSUNDEEP
javascriptnode.jsbuffer
BJRINT
BJRINT

Reputation: 667

Write raw binary data to Buffer

I am working on a function that iterates over PCM data. I am getting chunks of data of varying size and I am currently handling this by buffer concatenation. Problem is, I am quite sure that this approach is a performance killer.

One of the simplest algorithm consists of chunking 500 chunks of 4800 bytes (= grain), and repeating them 3 times as such :

buf = <grain1, grain1, grain1, ..., grain500, grain500, grain500>

function(){
  // ...
  let buf = Buffer.alloc(0) // returned buffer, mutated

  // nGrains is defined somewhere else in the function
  // example: nGrains = 500
  for(let i=0;i<nGrains;i++){
    // a chunk of PCM DATA 
    // example: grain.byteLength = 4800 
    const grain = Buffer.from(this._getGrain())

    // example: nRepeats = 3
    for(let j=0;j<nRepeats;j++)
      buf = Buffer.concat([buf, grain])
  }

  return buf
}

I feel like these performance heavy operations (1500 mutating concatenations) could be avoided if there were some sort of way to directly write "raw data" from a given offset to a pre-size-allocated buffer. I made the following helper function that gave me HUGE performance improvements, but I feel like I am doing something wrong... 

function writeRaw(buf, rawBytes, offset) => {
  for(i=0;i<rawBytes.byteLength;i++){
    buf.writeUInt8(rawBytes.readUInt8(i), offset + i)
  }

  return buf
}

My function now looks like this:

function(){
  // ...
  const buf = Buffer.alloc(len) // returned buffer, immutable

  for(let i=0;i<nGrains;i++){
    const grain = Buffer.from(this._getGrain())

    for(let j=0;j<nRepeats;j++)
      writeRaw(buf, grain, (i * nRepeats + j) * grainSize)
  }

  return buf
}

My question is : Is there a cleaner way (or more standard way) to do this instead of iterating over bytes ? Buffer.write only seems to work for strings, although this would be ideal...

Upvotes: 0

Views: 426

Answers (1)

Jonas Wilms
Jonas Wilms

Reputation: 138234

There is Buffer.copy.

const buf = Buffer.alloc(len);

for(let i = 0; i < nGrains; i++){
  const grain = Buffer.from(this._getGrain());

 for(let j=0;j<nRepeats;j++)
   grain.copy(/*to*/ buf, /*at*/ (i * nRepeats + j) * grainSize);
}

You could also use Buffer.fill:

  const buf = Buffer.alloc(len);

  for(let i = 0; i < nGrains; i++) {
    const grain = Buffer.from(this._getGrain());
    buf.fill(grain, i * nRepeats * grainSize, (i + 1) * nRepeats * grainSize);
  }

Upvotes: 1

Related Questions