Grab a segment of an array in Java without creating a new array on heap

Question

I'm looking for a method in Java that will return a segment of an array. An example would be to get the byte array containing the 4th and 5th bytes of a byte array. I don't want to have to create a new byte array in the heap memory just to do that. Right now I have the following code:

doSomethingWithTwoBytes(byte[] twoByteArray);

void someMethod(byte[] bigArray)
{
      byte[] x = {bigArray[4], bigArray[5]};
      doSomethingWithTwoBytes(x);
}

I'd like to know if there was a way to just do doSomething(bigArray.getSubArray(4, 2)) where 4 is the offset and 2 is the length, for example.

Answer 1

One way is to wrap the array in java.nio.ByteBuffer, use the absolute put/get functions, and slice the buffer to work on a subarray.

For instance:

doSomething(ByteBuffer twoBytes) {
    byte b1 = twoBytes.get(0);
    byte b2 = twoBytes.get(1);
    ...
}

void someMethod(byte[] bigArray) {
      int offset = 4;
      int length = 2;
      doSomething(ByteBuffer.wrap(bigArray, offset, length).slice());
}

Note that you have to call both wrap() and slice(), since wrap() by itself only affects the relative put/get functions, not the absolute ones.

ByteBuffer can be a bit tricky to understand, but is most likely efficiently implemented, and well worth learning.

Answer 2

I needed to iterate through the end of an array and didn't want to copy the array. My approach was to make an Iterable over the array.

public static Iterable<String> sliceArray(final String[] array, 
                                          final int start) {
  return new Iterable<String>() {
    String[] values = array;
    int posn = start;

    @Override
    public Iterator<String> iterator() {
      return new Iterator<String>() {
        @Override
        public boolean hasNext() {
          return posn < values.length;
        }

        @Override
        public String next() {
          return values[posn++];
        }

        @Override
        public void remove() {
          throw new UnsupportedOperationException("No remove");
        }
      };
    }
  };
}

Answer 3

If you're seeking a pointer style aliasing approach, so that you don't even need to allocate space and copy the data then I believe you're out of luck.

System.arraycopy() will copy from your source to destination, and efficiency is claimed for this utility. You do need to allocate the destination array.

Answer 4

List.subList(int startIndex, int endIndex)

Answer 5

@unique72 answer as a simple function or line, you may need to replace Object, with the respective class type you wish to 'slice'. Two variants are given to suit various needs.

/// Extract out array from starting position onwards
public static Object[] sliceArray( Object[] inArr, int startPos ) {
    return Arrays.asList(inArr).subList(startPos, inArr.length).toArray();
}

/// Extract out array from starting position to ending position
public static Object[] sliceArray( Object[] inArr, int startPos, int endPos ) {
    return Arrays.asList(inArr).subList(startPos, endPos).toArray();
}

Answer 6

Arrays.asList(myArray) delegates to new ArrayList(myArray), which doesn't copy the array but just stores the reference. Using List.subList(start, end) after that makes a SubList which just references the original list (which still just references the array). No copying of the array or its contents, just wrapper creation, and all lists involved are backed by the original array. (I thought it'd be heavier.)

Answer 7

Disclaimer: This answer does not conform to the constraints of the question:

I don't want to have to create a new byte array in the heap memory just to do that.

(Honestly, I feel my answer is worthy of deletion. The answer by @unique72 is correct. Imma let this edit sit for a bit and then I shall delete this answer.)

---

I don't know of a way to do this directly with arrays without additional heap allocation, but the other answers using a sub-list wrapper have additional allocation for the wrapper only – but not the array – which would be useful in the case of a large array.

That said, if one is looking for brevity, the utility method Arrays.copyOfRange() was introduced in Java 6 (late 2006?):

byte [] a = new byte [] {0, 1, 2, 3, 4, 5, 6, 7};

// get a[4], a[5]

byte [] subArray = Arrays.copyOfRange(a, 4, 6);

Answer 8

This is a little more lightweight than Arrays.copyOfRange - no range or negative

public static final byte[] copy(byte[] data, int pos, int length )
{
    byte[] transplant = new byte[length];

    System.arraycopy(data, pos, transplant, 0, length);

    return transplant;
}

Answer 9

How about a thin List wrapper?

List<Byte> getSubArrayList(byte[] array, int offset, int size) {
   return new AbstractList<Byte>() {
      Byte get(int index) {
         if (index < 0 || index >= size) 
           throw new IndexOutOfBoundsException();
         return array[offset+index];
      }
      int size() {
         return size;
      }
   };
}

(Untested)

Answer 10

Java references always point to an object. The object has a header that amongst other things identifies the concrete type (so casts can fail with ClassCastException). For arrays, the start of the object also includes the length, the data then follows immediately after in memory (technically an implementation is free to do what it pleases, but it would be daft to do anything else). So, you can;t have a reference that points somewhere into an array.

In C pointers point anywhere and to anything, and you can point to the middle of an array. But you can't safely cast or find out how long the array is. In D the pointer contains an offset into the memory block and length (or equivalently a pointer to the end, I can't remember what the implementation actually does). This allows D to slice arrays. In C++ you would have two iterators pointing to the start and end, but C++ is a bit odd like that.

So getting back to Java, no you can't. As mentioned, NIO ByteBuffer allows you to wrap an array and then slice it, but gives an awkward interface. You can of course copy, which is probably very much faster than you would think. You could introduce your own String-like abstraction that allows you to slice an array (the current Sun implementation of String has a char[] reference plus a start offset and length, higher performance implementation just have the char[]). byte[] is low level, but any class-based abstraction you put on that is going to make an awful mess of the syntax, until JDK7 (perhaps).

Answer 11

Use java.nio.Buffer's. It's a lightweight wrapper for buffers of various primitive types and helps manage slicing, position, conversion, byte ordering, etc.

If your bytes originate from a Stream, the NIO Buffers can use "direct mode" which creates a buffer backed by native resources. This can improve performance in a lot of cases.

Answer 12

The Lists allow you to use and work with subList of something transparently. Primitive arrays would require you to keep track of some kind of offset - limit. ByteBuffers have similar options as I heard.

Edit: If you are in charge of the useful method, you could just define it with bounds (as done in many array related methods in java itself:

doUseful(byte[] arr, int start, int len) {
    // implementation here
}
doUseful(byte[] arr) {
    doUseful(arr, 0, arr.length);
}

It's not clear, however, if you work on the array elements themselves, e.g. you compute something and write back the result?

Answer 13

One option would be to pass the whole array and the start and end indices, and iterate between those instead of iterating over the whole array passed.

void method1(byte[] array) {
    method2(array,4,5);
}
void method2(byte[] smallarray,int start,int end) {
    for ( int i = start; i <= end; i++ ) {
        ....
    }
}

Answer 14

You could use the ArrayUtils.subarray in apache commons. Not perfect but a bit more intuitive than System.arraycopy. The downside is that it does introduce another dependency into your code.

Answer 15

I see the subList answer is already here, but here's code that demonstrates that it's a true sublist, not a copy:

public class SubListTest extends TestCase {
    public void testSubarray() throws Exception {
        Integer[] array = {1, 2, 3, 4, 5};
        List<Integer> list = Arrays.asList(array);
        List<Integer> subList = list.subList(2, 4);
        assertEquals(2, subList.size());
        assertEquals((Integer) 3, subList.get(0));
        list.set(2, 7);
        assertEquals((Integer) 7, subList.get(0));
    }
}

I don't believe there's a good way to do this directly with arrays, however.