Graph, facet, full-text: best of all worlds?

Question

Lucene, Elasticsearch: faceted search, full-text search, great!

Neo4j, maybe Titan: graph search awesomeness!

But what are good strategies for queries that combine facets, full-text and graphs? I would not want to materialize graph search results and carry these over into a faceted/full-text query. How to combine these worlds?

---

Here is an example:

• Facets – We have "items" with facets such as modified date, status, item type (and much more; although fields/facets of these items are even configurable, a reasonable, average item would have around 20 facets).
• Graphs – These "items" are related to each other (we talk a lot about "upstream" and "downstream" items/relationships), they are related to users in various ways, they have associated comments (which in turn will have more users associated).

Now we would like to do queries such as:

• Show me all the "items" that are x steps away from a given list of items (a graph concern), that are in status such-and-such and that are modified in the last 30 days (faceted concerns).
• Show me all the users that are connected to a given "item", meaning that they modified or commented on the given item, or any of its related items up to x steps away from the given item, provided that these related items were in a certain status, and modified after the given item was modified.

---

You could model graphs in Lucene, but that seems a big no-starter. (Prove me wrong!)

You could add facets to Neo4j, but that seems a performance hazard that it is just not optimized for. (But maybe you have different experiences.)

And maybe there is a golden bullet to this problem.

Answer 1

Well, unsurprisingly, it depends. Apologies if this comes across as a somewhat generic answer, but it may be the best you'll get without a lot more specifics on the nature of your challenge.

You need to first outline what your query use cases are, and what the associated data volumes are. If this were a vanilla database and we were trying to optimize your query to make it as quick as possible, we'd first try to identify which portions of your query are the most selective. We'd then push those towards execution first, leaving the less selective bits of your query for execution later. This has the advantage of quickly cutting down on the volume of potential results we're dealing with.

So let's stipulate that you need to do faceted search, full-text search, and graph search. If you can look at your data and what kinds of facets/text/graph patterns you're looking for, in general do the most exclusive/selective thing first.

As far as modeling graphs in lucene, I suppose this could work, but only if the faceted search bits are your biggest need, and the graph bits are fairly primitive. Working with things like shortest path in lucene sounds painful (although I've long since learned not to claim that things are impossible, since almost nothing's impossible if you have enough hours)

Hybrids are also possible (i.e. not using lucene in neo4j, but using them separately and inter-linking them). I hesitate to suggest that without more detail, because it's a powerful (but complicated and costly) way to go.