SDN4 or neo4j-ogm performances issue

Question

I wrote some simple java code and I encountered some bad performances with SDN4 that I didn't have with SDN3. I suspect the find repositories methods depth parameter to not work exactly in the way it should be. Let me explain the problem:

Here are my java classes(it's just an example) in which I removed getters, setters, contructors, ...

First class is 'Element' :

@NodeEntity
public class Element {

@GraphId
private Long id;

private int age;

private String  uuid;

@Relationship(type = "HAS_VALUE", direction = Relationship.OUTGOING)
private Set<Value> values = new HashSet<Value>();

Second one is 'Attribute'

@NodeEntity
public class Attribute {
@GraphId
private Long id;

@Relationship(type = "HAS_PROPERTIES", direction = Relationship.OUTGOING)
private Set<HasInterProperties> properties;

The 'value' class allow my user to add a value on an Element for a specific attribute :

@RelationshipEntity(type = "HAS_VALUE")
public class Value {

@GraphId
private Long id;

@StartNode
Element element;

@EndNode
Attribute attribute;

private Integer value;

private String uuid;

public Value() {
}

public Value(Element element, Attribute attribute, Integer value) {
    this.element = element;
    this.attribute = attribute;
    this.value = value;
    this.element.getValues().add(this);
    this.uuid = UUID.randomUUID().toString();
}

'Element' classe really need to know its values but 'Attribute' class do not care at all about values.

An attribute has references on InternationalizedProperties class which is like that :

@NodeEntity
public class InternationalizedProperties {
@GraphId
private Long id;

private String name;

The relationship entity between an attribute and it InternationalizedProperties is like the following :

@RelationshipEntity(type = "HAS_PROPERTIES")
public class HasInterProperties {

@GraphId
private Long id;

@StartNode
private Attribute attribute;

@EndNode
private InternationalizedProperties properties;

private String locale;

I then created a little main method to create two attributes and 10000 elements. All my elements have a specific value for the first attribute but no values for the second one (no relation between them). Both attributes hav two differents internationalizedProperties. Here is a sample :

public static void main(String[] args) {
    ApplicationContext context = new ClassPathXmlApplicationContext("spring/*.xml");
     Session session = context.getBean(Session.class);
    session.query("START n=node(*) OPTIONAL MATCH n-[r]-() WHERE ID(n) <> 0 DELETE n,r", new HashMap<String, Object>());
    ElementRepository elementRepository = context.getBean(ElementRepository.class);
    AttributeRepository attributeRepository = context.getBean(AttributeRepository.class);
    InternationalizedPropertiesRepository internationalizedPropertiesRepository = context.getBean(InternationalizedPropertiesRepository.class);
    HasInterPropertiesRepository hasInterPropertiesRepository = context.getBean(HasInterPropertiesRepository.class);

    //Creation of an attribute object with two internationalized properties
    Attribute att = new Attribute();
    attributeRepository.save(att);
    InternationalizedProperties p1 = new InternationalizedProperties();
    p1.setName("bonjour");
    internationalizedPropertiesRepository.save(p1);
    InternationalizedProperties p2 = new InternationalizedProperties();
    p2.setName("hello");
    internationalizedPropertiesRepository.save(p2);
    hasInterPropertiesRepository.save(new HasInterProperties(att, p1, "fr"));
    hasInterPropertiesRepository.save(new HasInterProperties(att, p2, "en"));
    LOGGER.info("First attribut id is {}", att.getId());

    //Creation of 1000 elements having a differnt value on a same attribute

    for(int i = 0; i< 10000; i++) {
        Element elt = new Element();
        new Value(elt, att, i);
        elementRepository.save(elt);
        if(i%50 == 0) {
            LOGGER.info("{} elements created. Last element created with id {}", i+1, elt.getId());
        }
    }

    //Another attribut without any values from element.
    Attribute att2 = new Attribute();
    attributeRepository.save(att2);
    InternationalizedProperties p12 = new InternationalizedProperties();
    p12.setName("bonjour");
    internationalizedPropertiesRepository.save(p12);
    InternationalizedProperties p22 = new InternationalizedProperties();
    p22.setName("hello");
    internationalizedPropertiesRepository.save(p22);
    hasInterPropertiesRepository.save(new HasInterProperties(att2, p12, "fr"));
    hasInterPropertiesRepository.save(new HasInterProperties(att2, p22, "en"));
    LOGGER.info("Second attribut id is {}", att2.getId());

Finally, in another main method, I try to get several times the first attribute and the second one :

private static void getFirstAttribute(AttributeRepository attributeRepository) {
    StopWatch st = new StopWatch();
    st.start();
    Attribute attribute = attributeRepository.findOne(25283L, 1);
    LOGGER.info("time to get attribute (some element have values on it) is {}ms", st.getTime());
}

private static void getSecondAttribute(AttributeRepository attributeRepository) {
    StopWatch st = new StopWatch();
    st.start();
    Attribute attribute2 = attributeRepository.findOne(26286L, 1);
    LOGGER.info("time to get attribute (no element have values on it) is {}ms", st.getTime());
}

public static void main(String[] args) {
    ApplicationContext context = new ClassPathXmlApplicationContext("spring/*.xml");

    AttributeRepository attributeRepository = context.getBean(AttributeRepository.class);

    getFirstAttribute(attributeRepository);
    getSecondAttribute(attributeRepository);

    getFirstAttribute(attributeRepository);
    getSecondAttribute(attributeRepository);

    getFirstAttribute(attributeRepository);
    getSecondAttribute(attributeRepository);

    getFirstAttribute(attributeRepository);
    getSecondAttribute(attributeRepository);
}

Here are the logs of this execution :

time to get attribute (some element have values on it) is 2983ms
time to get attribute (no element have values on it) is 4ms     
time to get attribute (some element have values on it) is 1196ms
time to get attribute (no element have values on it) is 2ms     
time to get attribute (some element have values on it) is 1192ms
time to get attribute (no element have values on it) is 3ms     
time to get attribute (some element have values on it) is 1194ms
time to get attribute (no element have values on it) is 3ms     

Getting the second attribut (and its internationalized properties thanks to depth=1) is very quick but to get the first one remains very slow. I know that there are many relations (10000 exactly) which are pointing on the first attribute, but when I want to get an attribute with its internationalized properties I clearly do not want to get all the values which are pointing on it. (since Set is not specified on Attribute class).

That's why I think there is a performance problem here. Or may be I do something wrong ?

Thanks for your help

Answer 1

When loading data from the graph we don't currently analyse how your domain model is wired together, so we may potentially bring back related nodes that you do not require. These will then be discarded if they are not mappable in your domain, but if there are many of them, it could potentially impact response times.

There are two reasons for this approach.

• It is obviously much simpler to create generic queries to any depth,than it would be dynamically analyse your domain model to any arbitrary depth and generate on-the-fly custom queries; its also much simpler to analyse and prove the correctness of generic queries.

• We want to preserve the capability to support polymorphic domain models in the future, where we don't necessarily know what's in the database from one day to the next, but we want to adapt our domain model hydration according to what we find.

In this case I would suggest writing a custom query to load the Attribute objects, to ensure you don't bring back all the unwanted relationships.