CODEWITHSUNDEEP
javamarklogic
kdm06
kdm06

Reputation: 189

MarkLogic 8 Java API - add element range index programatically

I'm using Search Query By Example using RawQueryByExampleDefinition from the ML Java API. Before we upgraded to version 8, passing in a dynamic options like the below works (focus on the element name):

<options xmlns="http://marklogic.com/appservices/search">
  <sort-order collation="http://marklogic.com/collation/en/S1/EO/CU/MO" type="xs:string" direction="ascending">
    <element ns="" name="dynamicElement"/>
  </sort-order>
</options>

The above works for ML 7 without any element range index configured. However, for ML 8, this gives me an error and after digging into the cause, it appears that creating an element range index is now required, else ML Java API will throw an error.

My problem is the elements ingested into ML from my API are not really predetermined or schemaless, which means they (client) can insert any JSON format. Is there a way to create an element range index dynamically or programmatically so that the system will first check the passed in "schemaless" data and add element range indexes on the fly based on the passed in JSON structure or elements?

Upvotes: 1

Views: 343

Answers (2)

Sam Mefford
Sam Mefford

Reputation: 2475

I think it's worth pointing out that you can also create a field range index which is based on a field which can use XPath to match a flexible list of element names and json-properties at any depth. That way your application can reference the field while the field configuration on your server can represent your latest list of elements you want included.

Then your options would become:

<options xmlns="http://marklogic.com/appservices/search">
  <sort-order collation="http://marklogic.com/collation/en/S1/EO/CU/MO"
    type="xs:string" direction="ascending">
    <field name="dynamicField"/>
  </sort-order>
</options>

Upvotes: 1

mblakele
mblakele

Reputation: 7842

That sounds like a bug fix to me. You should always have an appropriate range index for any query sort options at scale. If scale isn't important, do the sorting in Java.

You can create range indexes using the admin API: https://docs.marklogic.com/admin:database-add-range-element-index and related functions. You can create as many range indexes as you like.

It might work to expose a REST API that allows users to create a range index on demand. However in your situation I'd do some careful analysis first, because that's likely to lead to unbounded growth in disk and memory utilization. You may also have to think about possible conflicts between elements with the same name but different syntactical meanings, calling for different collations etc. This is what XML namespaces are for: ns1:title vs ns2:title. But JSON doesn't have namespaces.

Upvotes: 1

Related Questions