CODEWITHSUNDEEP
graphpaginationmarklogicmarklogic-8nosql
BartoszMiller
BartoszMiller

Reputation: 1227

Partial document read and pagination by property in Marklogic

I'm working on medical system which is using Marklogic. Some time ago I asked a question (How to query a graph of documents of different types at once in Marklogic?) which resulted in our decision to use one large Patient document which holds most of the information for patient. For the purpose of the example I'm presenting a simplified Patient document which contains an array of Appointments.

Simplified patient document

{
    "firstName": "Chauncey", 
    "lastName": "Hays", 
    "memberId": "KLKB XMQ 77", 
    "dateOfBirth": "1971-09-11", 
    "gender": "MALE", 


    "appointments": [
        { 
            "name": "Follow Up", 
            "start": "2017-06-08T23:13:00Z", 
            "end": "2017-06-09T00:43:00Z" 
        },
        {
            "name": "Follow Up", 
            "start": "2017-06-09T23:13:00Z", 
            "end": "2017-06-10T00:43:00Z" 
        }
    ]

}

One of our use cases is to return paginated appointments (property of Patient document) filtered by patient data e.g. find a page of appointments for all MALE patients. We have been struggling with it for a while with no luck. One of the solutions that I see (probably slightly over-engineered) is to return a page of Patients (page of Patients may result in larger number of Appointments than the page size) and filter out appointments on the server side.

My main question is: is it feasible to filter by Patient data (gender=MALE), do partial read (read Appointments) and paginate results by Appointments (property of Patient document) in Marklogic? Any comments greatly appreciated.

Upvotes: 0

Views: 154

Answers (1)

Sam Mefford
Sam Mefford

Reputation: 2475

You're right, that for your use case structured query option "searchable-expression" won't work because your query filter is outside the searchable-expression. Nevertheless, here's what I tried that matched 0 results:

String query = "{ \"search\": {" +
  "\"query\": {" +
  "  \"value-query\": {" +
  "    \"json-property\": \"gender\"," +
  "    \"text\": \"MALE\"" +
  "  }" +
  "}," +
  "\"options\": {" +
  "  \"searchable-expression\": {" +
  "    \"text\": \"appointments\"" +
  "  }" +
  "}}}";
QueryManager queryMgr = client.newQueryManager();
RawCombinedQueryDefinition queryDef =
  queryMgr.newRawCombinedQueryDefinitionAs(Format.JSON, query);
queryMgr.search(queryDef, new SearchHandle());

And while extract-document-data works, it's like you described, where you're really paging over docs, not the extracted piece. Nevertheless, here's what I tried that matched correctly and returned the desired results, but didn't offer a way to paginate as desired:

String query = "{ \"search\": {" +
  "\"query\": {" +
  "  \"value-query\": {" +
  "    \"json-property\": \"gender\"," +
  "    \"text\": [\"MALE\"]" +
  "  }" +
  "}," +
  "\"options\": {" +
  "  \"extract-document-data\": {" +
  "    \"extract-path\": \"/appointments\"" +
  "  }" +
  "}}}";
QueryManager queryMgr = client.newQueryManager();
RawCombinedQueryDefinition queryDef =
  queryMgr.newRawCombinedQueryDefinitionAs(Format.JSON, query);
DocumentManager docMgr = client.newDocumentManager();
DocumentPage page = docMgr.search(queryDef, 1);

So I think you'll need to resort to a server-side script, ideally in the form of a resource extension. Or you can do an invoke. Install this script as "/ext/pageAppointments.xqy" in your modules database:

  declare variable $start as xs:long external;
  declare variable $pageLength as xs:long external;
  let $appointments :=
    cts:search(collection(), cts:json-property-value-query("gender", "MALE"))/appointments
  return $appointments[$start to ($start + $pageLength - 1)]

Then run this in Java:

ServerEvaluationCall call = client.newServerEval()
  .modulePath("/ext/pageAppointments.xqy")
  .addVariable("start", 1)
  .addVariable("pageLength", 10);
EvalResultIterator results = call.eval();
try {
  for ( EvalResult result : results ) {
    JsonNode appointment = result.getAs(JsonNode.class);
    System.out.println(appointment);
  }
} finally { results.close(); }

Upvotes: 3

Related Questions