CODEWITHSUNDEEP
javaarraysperformancejvmor-tools
Stephen Carman
Stephen Carman

Reputation: 997

High performance primitive array builder in Java

I currently use google-or tools to solve a max flow problem, so this has me create a few int[] arrays in java to pass into ortools. Now ortools is very fast and not an issue here but I'm open to performance minded alternatives here.

The problem lies mostly in building the arrays which takes a majority of the time as well as GC when the results are returned from which I chalk up to probably JNI overhead and not much I can do about that. The primitive arrays approach around the 5 - 7 million point mark and they are large enough to require them to be integers, short is not an option. Do I have any options or tricks or does anyone have any insight into how to most efficiently build these? Memory is not really an issue here I have enough that and for the most part I am open to any solution for the absolute bleeding edge performance, even if it requires a different representation of my data but this still must be able to be plugged into Ortools (unless You have an idea to replace it) but I welcome any suggestions at all regarding how to get the fastest array building out of this. Mind you I don't know the length of the arrays ahead of time, I don't do updates, deletes, only appends. I'm happy to provide anymore details. Thanks for any suggestions.

Upvotes: 0

Views: 6796

Answers (1)

maaartinus
maaartinus

Reputation: 46482

Too long for a comment.

If building the problem representation takes a lot of time when compared to solving, then you're doing something wrong. I guess, you're using something like

int[] appendTo(int[] array, int element) {
    int[] result = Arrays.copyOf(array, array.length + 1);
    result[result.length - 1] = element;
    return result;
}

which has a quadratic complexity. The solution is similar to what ArrayList does: Grow by some fixed factor and ignore trailing array elements. This mayn't be what you need at the end, but shrinking all arrays once (just before passing them to the library) is cheap.

You could use a class like

class MyIntArray {
   private int length;
   private int[] data = new data[4];

   // This does the final shrinking.
   public int[] toArray() {
       return Arrays.copyOf(array, length);
   }

   public MyIntArray append(int element) {
       if (array.length == length) {
           array = Arrays.copyOf(array, 2 * length);
       }
       array[length++] = element;
   }
}

or misuse the last element of an int[] for tracking the logical length (slightly more efficient, but very hacky).

There are various trade-offs, e.g., you could reduce your growth factor to 1.5 by using length + (length >> 1) instead of 2 * length, start with shorter or longer arrays, or even with an empty array (like ArrayList does; then you'd need to adapt the growth factor as well).

Upvotes: 4

Related Questions