Splitting a byte[] into multiple byte[] arrays

Question

I am trying to "chunk" up the bytes of an image. This will allow me to upload a large image in bytes array. I have the image currently stored as one large byte[]. I would like to split the byte array into byte[]'s with a each exactly of 5 MB.

Answer 1

Based on the answer by Santosh Bhatt I created an extension function:

fun ByteArray.chunkedBy(chunkSize: Int): List<ByteArray> {
    val result: ArrayList<ByteArray> = ArrayList()
    if (chunkSize <= 0) {
        throw IllegalArgumentException("chunkSize must be larger than 0 but was $chunkSize")
    } else {
        for (chunk in this.indices step chunkSize) {
            result.add(this.copyOfRange(chunk, min(chunk + chunkSize, this.size)))
        }
    }

    return result
}

Use it like this: val chunks = byteArray.chunkedBy(CHUNKSIZE)

I hope somebody finds this useful.

Answer 2

These answers work fine but has optimization issues. They allocate extra space of the whole chunk size irrespective of the only actual bytes of memory the array needs to allocate for copying the data. Here is the solution for this problem.

private fun divideDataIntoChunks(source: ByteArray, chunkSize: Int): kotlin.collections.ArrayList<ByteArray> {
        val result: ArrayList<ByteArray> = ArrayList()
        if (chunkSize <= 0) {
            result.add(source)
        } else {
            for (chunk in source.indices step chunkSize) {
                result.add(source.copyOfRange(chunk, min(chunk+chunkSize,source.size)))
            }
        }
        return result
    }

Answer 3

You can use copyOfRange for that:

T[] copyOfRange (T[] original, 
            int from, 
            int to);

In your case, something like this:

Byte[] copyOfRange (original, 
            0, 
            5000000);

make sure you calculate the offset:

class test {

  // this is just for dummy data
  public static byte[] getTestBytes() {
    byte[] largeByteArray = new byte[50_000_000];
    for(int i = 0; i < 50_000_000; i ++) {
      largeByteArray[i] = 0;
    }
    return largeByteArray;
  }


  // this method splits your byte array into small portions 
  // and returns a list with those portions
  public static List<byte[]> byteToPortions(byte[] largeByteArray) {
    // create a list to keep the portions
    List<byte[]> byteArrayPortions = new ArrayList<>();

    // 5mb is about 5.000.000 bytes
    int sizePerPortion = 5_000_000;
    int offset = 0;

    // split the array
    while(offset < largeByteArray.length) {
      // into 5 mb portions
      byte[] portion = Arrays.copyOfRange(largeByteArray, offset, offset + sizePerPortion);

      // update the offset to increment the copied area
      offset += sizePerPortion;

      // add the byte array portions to the list
      byteArrayPortions.add(portion);
    }

    // return portions
    return byteArrayPortions;
  }

  // create your byte array, and split it to portions
  public static void main(String[] args) {
    byte[] largeByteArray = getTestBytes();

    List<byte[]> portions = byteToPortions(largeByteArray);

    // work with your portions
  }
}

---

Something cool: the value to does not have to be an index inside the array, it checks that for you without erroring and copies a subset that is valid to the intended array.

Answer 4

public static byte[][] divideArray(byte[] source, int chunksize) {


        byte[][] ret = new byte[(int) Math.ceil(source.length / (double) chunksize)][chunksize];

        int start = 0;

        int parts = 0;


        for (int i = 0; i < ret.length; i++) {
            if (start + chunksize > source.length) {
                System.arraycopy(source, start, ret[i], 0, source.length - start);
            } else {
                System.arraycopy(source, start, ret[i], 0, chunksize);
            }
            start += chunksize;
            parts++;
        }

        Log.d("Parts", parts + "");

        return ret;
    }

Call It by

divideArray(common.fullyReadFileToBytes(wallpaperDirectory), 5 * 1024 * 1024)