CODEWITHSUNDEEP
javaperformanceoptimization
Artur
Artur

Reputation: 3282

Efficiently access fields by name

A little background as to what I'm trying to achieve:

I'm parsing JSON (over 15GB) and I must store it in memory so any wrappers and extra data is not welcomed, due to the framework and interfaces used within it I must provide functionality to access fields by name. By replacing some String with Enum, Integer with int, Double with double, etc. I'm able to shave about 90% of memory footprint (in comparison with Jackson).

I'm looking to efficiently access the fields at runtime in Java by their name. I'm aware of reflection, but for my case its performance is simply unacceptable, so I don't want to use it.

If it makes the problem easier to solve I'm not too bothered about setting the fields values. I also know at compile time the names of supported fields.

I don't want to store everything in a map i.e. Map<String,Object> due to the memory footprint of boxed object, but I don't mind returning them in a boxed form.

I'm sure this problem was encountered by others and I'm interested in any clever solutions - cleverer than tons of if ... else ... statements.

Let's say the interface to implement is:

public interface Accessor {
    Object get(String fieldName);
}

The Object returned by get can be of any type including enum. A naive implementation would be:

public class TestObject implements Accessor {

    public enum MyEnum {ONE, TWO, THREE};

    private final MyEnum myEnum;
    private final int myInt;
    private final double myDouble;
    private final String myString;

    public TestObject(MyEnum myEnum, int myInt, double myDouble, String myString) {
        this.myEnum = myEnum;
        this.myInt = myInt;
        this.myDouble = myDouble;
        this.myString = myString;
    }

    @Override
    public Object get(String fieldName) {
        if ("myEnum".equals(fieldName)) {
            return myEnum;
        } else if ("myInt".equals(fieldName)) {
            return myInt;
        } else if ("myDouble".equals(fieldName)) {
            return myDouble;
        } else if ("myString".equals(fieldName)) {
            return myString;
        } else {
            throw new UnsupportedOperationException(); // Or could simply return null
        }
    }

}

Upvotes: 2

Views: 183

Answers (2)

Richard Sitze
Richard Sitze

Reputation: 8463

What you want is a mapping from a fieldName to a value, the type of which is determined by the fieldName. You know the set of field names up-front, so this is an ideal task for an Enum.

If you don't like the idea of hard-coding each field as an enum, then the variation would be an enum-per-type (MY_FIELD1 becomes MY_ENUM), with a mapping from fieldName to this EnumType.

In the code below I'm making assumptions about the relationship between fieldName and TestObject. Specifically it looks like TestObject is presenting various types of the same value (surely where reasonable), as opposed to a separate value for each field name?

So, to the code:

Rewrite:

@Override
public Object get(String fieldName) {
    MyField field = MyField.mapNameToField(fieldName);
    if (field == null)
        throw new UnsupportedOperationException(); // Or could simply return null
    return field.getValue(this);
}

Given (something like):

enum MyField {
  MY_FIELD1("myField1") {
      public Object getValue(TestObject obj) { return obj.myEnum; }
  },
  MY_FIELD2("myField2") {
      public Object getValue(TestObject obj) { return obj.myInt; }
  },
  ...
  ;

  public abstract Object getValue(TestObject obj);
  public String getName() { return name; }

  public static MyField mapNameToField(String name) { return map.get(name); }

  static {
      map = new HashMap<String,MyField>();
      for(MyField value: values()) {
          map.put(value.getName(), value);
      }
  }

  private MyField(String fieldName) { name = fieldName; }

  private String name;
  private static Map<String, MyField> map;
}

Upvotes: 4

Julius Musseau
Julius Musseau

Reputation: 4155

I've never used this, but looks promising:

http://labs.carrotsearch.com/download/hppc/0.4.1/api/

"High Performance Primitive Collections (HPPC) library provides typical data structures (lists, stacks, maps) template-generated for all Java primitive types (byte, int, etc.) to conserve memory and boost performance."

In particular, the Object{Type}OpenHashMap classes might be what you're looking for:

  1. ObjectByteOpenHashMap
  2. ObjectCharOpenHashMap
  3. ObjectDoubleOpenHashMap
  4. ObjectFloatOpenHashMap
  5. ObjectIntOpenHashMap
  6. ObjectLongOpenHashMap
  7. ObjectShortOpenHashMap

I imagine you would have all 7 of these defined as fields (or whatever subset of them you like), and you would probe each one in turn to see if the key was present for that type of primitive value. E.g.,

if (byteMap.containsKey(key)) {
  return byteMap.lget(); // last value saved in a call to containsKey()
} else if (charMap.containsKey(key)) {
  return charMap.lget();
} else if {
  // and so on... 
}

Notice they have their own special lget() method call to optimize the containsKey() / get() usage pattern so typical with maps.

Upvotes: 0

Related Questions