CODEWITHSUNDEEP
javaarrays
Santosh Tiwari
Santosh Tiwari

Reputation: 1216

Get a slice of an array without creating a new copy

I am trying to find a solution/workaround for slicing extremely large arrays without creating new copies. Here is my problem.

Suppose I have a large array of double/int of size 100 million or more. I am storing many different arrays representing different things in a single extremely large array to significantly save on memory usage. Hence, instead of having 1 million arrays each of size 100, I have a single array of size 100 million. I store indices (start and stop) to keep track of my data.

I want to get thousands of slices with size 100. If I use the method Arrays.copyOfRange() to get slices, it defeats the purpose of putting everything in a single large array since each slice is a new copy eating up memory.

I have legacy code (in excess of 1 million lines written over the years by many people) that works with its own data (which are smaller arrays). It is not possible to modify the existing code to work with indices (begin, end) in a large array.

If I could somehow return the original array such that the returned array is a reference (or pretends to be) where index 0 is some arbitrary index in the original large array, it would be great.

In C/C++, I can easily return a pointer with a specific offset and length with which the calling code can work.

What are my options in Java?

Edit: I looked at the following similar question, but it does not contain a response to my question. How to get a sub array of array in Java, without copying data?

Upvotes: 10

Views: 6881

Answers (4)

zubergu
zubergu

Reputation: 3706

Can you create your own object that stores index, size and reference to the original array?

class CustomizedArray {
  int startIndex;
  int size;
  int[] originalArray;

  public CustomizedArray(int startIndex, int size, int[] originalArray) {
    this.startIndex = startIndex;
    this.size = size;
    this.originalArray = originalArray;
   }

   public int getIndex(int index) {
     int originalIndex = startIndex+index;
     if(index <0 || originalIndex >= startIndex+size) {
        throw new IndexOutOfBoundException();
     }
     return originalArray[originalIndex];


}

Then you can store CustomizedArray in some bigger structure.

Upvotes: 1

Erkan Haspulat
Erkan Haspulat

Reputation: 12562

How about creating a wrapper class that holds references to your original array and your start index, and using an instance of this wrapper to access your original array.

Below code might not be syntactically correct, but it should give you the idea.

public class ArraySlice(){
  private int startIndex;
  private int[] originalArray;
  //getters-setters

  public ArraySlice(int[] originalArray, int startIndex){
    //Initialize
  }

  public int get(int index){
    return originalArray[startIndex+index]
  }
}

Upvotes: 2

Ted Hopp
Ted Hopp

Reputation: 234847

For an array of int values, you can wrap in an IntBuffer. You can also wrap a slice of an array.

int[] largeArray = . . .

// create a slice containing the elements 100 through 149 (50 elements):
IntBuffer slice = IntBuffer.wrap(largeArray, 100, 50);

Upvotes: 3

Jean Logeart
Jean Logeart

Reputation: 53839

Your best option is to store the indexes of the slices in a separate structure, such as an array storing those indexes.

This way, you do not instantiate large arrays being a partition of the whole data array.

Upvotes: 1

Related Questions