Why Hibernate Search is slow? Even with few records

Question

I have a project that uses JSF 2, Spring, JPA 2 and Hibernate Search 4.5.0. This project is very simple but Hibernate Search is giving me some trouble.

The only part I use Hibernate Search is for indexing a Oracle View. This view, if I run a simple select count(*) from my_view gives me 5756 records, that is not much.

Well, the first thing I did, was the entity mapping with JPA / Hibernate Search:

@Entity
@Indexed
@Table(name = "my_view")
public class Person implements Serializable {

    private static final long serialVersionUID = 244555315052436669L;

    @Id
    @Column(name = "id", insertable = false, updatable = false)
    private Long id;

    @Field(store = Store.NO, index = Index.YES, analyze = Analyze.YES)
    @Column(name = "name", insertable = false, updatable = false)
    private String name;

    @Field(store = Store.NO, index = Index.YES, analyze = Analyze.YES)
    @Column(name = "email", insertable = false, updatable = false)
    private String email;

    @Field(store = Store.NO, index = Index.YES, analyze = Analyze.YES)
    @Column(name = "user", insertable = false, updatable = false)
    private String user;

    @Field(store = Store.YES, index = Index.YES, analyze = Analyze.YES)
    @Column(name = "phone", insertable = false, updatable = false)
    private String phone;

    //Getters and Setters ommited
}

After mapping my entity, I build a cronJob to index this entity. Take a look the method that I use to index my entity.

public void index() throws DAOException {
    FullTextEntityManager fullTextEntityManager = Search.getFullTextEntityManager(this.entityManager);

    try {
        fullTextEntityManager.createIndexer(Person.class)
            .purgeAllOnStart(Boolean.TRUE)
            .optimizeOnFinish(Boolean.TRUE)
            .startAndWait();
    }
    catch (InterruptedException e) {
        logger.error("Error creating index", e);
        throw new DAOException(e);
    }
}

And, after indexing my entity, I can search over many columns with one keyword. The problem is when I try to search without any keyword! My application has to return all records. But I thought, using Hibernate Search, this search would be extremely fast. My mistake!

The search takes more than 40 seconds to return! Here how I search without a term:

FullTextEntityManager fullTextEm = Search.getFullTextEntityManager(this.entityManager);
QueryBuilder qb = fullTextEm.getSearchFactory().buildQueryBuilder().forEntity(Person.class).get();
FullTextQuery fullTextQuery = fullTextEm.createFullTextQuery(qb.all().createQuery());
Sort sortField = new Sort(new SortField("name", SortField.STRING));
fullTextQuery.setSort(sortField);
return fullTextQuery.getResultList();

Answer 1

Problem solved with projections! Thanks to @chrylis, @RandomMooCow and @Sanne.

I had to modify some of my code. The first thing I did, was to change the way how to query the data to use projections.

FullTextEntityManager fullTextEm = Search.getFullTextEntityManager(this.entityManager);
QueryBuilder qb = fullTextEm.getSearchFactory().buildQueryBuilder().forEntity(Person.class).get();
FullTextQuery fullTextQuery = fullTextEm.createFullTextQuery(qb.all().createQuery());

// I added this line to use projections
fullTextQuery.setProjection("id", "name", "email", "user", "phone");

Sort sortField = new Sort(new SortField("name", SortField.STRING));
fullTextQuery.setSort(sortField);
return fullTextQuery.getResultList();

But, when I use projection, the query doesn't return a list of specific object (in my case: Person.class), but a list of Object[].class. So, I write a method to convert List<Object[]> to List<Person>. Here is the code:

private List<Person> toList(List<Object[]> objPeople) {
    List<Person> lstPeople = new LinkedList<Person>();
    Person Person = null;

    for(Object[] objPerson : objPeople) {
        Person p = new Person();
        p.setId(Long.parseLong(objPerson[0].toString()));
        p.setName(String.valueOf(objPerson[1] == null ? "" : objPerson[1]));
        p.setEmail(String.valueOf(objPerson[2] == null ? "" : objPerson[2]));
        p.setUser(String.valueOf(objPerson[3] == null ? "" : objPerson[3]));
        p.setPhone(String.valueOf(objPerson[4] == null ? "" : objPerson[4]));
        lstPeople.add(Person);
    }

    return lstPeople;
}

And finally, I changed my entity to store the values on the indexes, so Hibernate Search can retrieve the data from indexes and convert to objects. My entity:

@Entity
@Indexed
@Table(name = "my_view")
public class Person implements Serializable {

    private static final long serialVersionUID = 244555315052436669L;

    @Id
    @Field(store = Store.YES, index = Index.YES, analyze = Analyze.YES)
    @Column(name = "id", insertable = false, updatable = false)
    private Long id;

    @Field(store = Store.YES, index = Index.YES, analyze = Analyze.YES)
    @Column(name = "name", insertable = false, updatable = false)
    private String name;

    @Field(store = Store.YES, index = Index.YES, analyze = Analyze.YES)
    @Column(name = "email", insertable = false, updatable = false)
    private String email;

    @Field(store = Store.YES, index = Index.YES, analyze = Analyze.YES)
    @Column(name = "user", insertable = false, updatable = false)
    private String user;

    @Field(store = Store.YES, index = Index.YES, analyze = Analyze.YES)
    @Column(name = "phone", insertable = false, updatable = false)
    private String phone;

    //Getters and Setters ommited
}

And it worked!

Answer 2

The Query execution is probably extremely fast (as searching without a keyword is very efficient), but the next step of execution is to load all the results in memory from the database and loading 5000+ elements from the database is never going to be fast, unless you look into caching of entities and queries.

If you need to load everything, that's not a full-text query but better performed by a standard relational query. Whatever the query type -unless you use projections- you might also want to investigate the loading strategies that your model is causing. Often you can get a significant boost by adjusting fetch strategies of some relation, and/or enabling second level caching.