CODEWITHSUNDEEP
xquerymarklogicmarklogic-dhf
Shalini
Shalini

Reputation: 348

How to embed custom code in MarkLogic Data Hub Framework?

I have created an Entity and have created input and harmonize flow. I can able to see generated XQuery files.

Now i have a requirement where i need to do some (if-else) on my raw data and based on the conditions i need to Push some of the data to my FINAL Database and some Data will remain in STAGING itself (That should not go into FINAL).

I am confused on which files (main.xqy,headers.xqy etc) i need to do code changes so if i run my Harmonize Flow then entire thing should work in one go.

Upvotes: 0

Views: 238

Answers (1)

Elijah Bernstein-Cooper
Elijah Bernstein-Cooper

Reputation: 515

Each of the harmonization flow plugins in the MarkLogic Operational Data Hub Framework are intended to be customized. There are five plugins, collector.xqy, content.xqy, header.xqy, triples.xqy, and writer.xqy. The simplest harmonization follows something like this:

  1. Identify which documents in the staging database need to be processed in the collector plugin
  2. Transform the documents from step 1. in the content plugin (add the if/else logic)
  3. Write the harmonized documents from step 2. to the final database using the writer plugin.

Here are summaries of each of the plugins from the ODH Wiki:

Collector

Select IDs of documents in the staging database to be processed.

Content

Perform transformation of input data into a normalized or canonical format to store in the final document or documents. You can add custom transformation code here.

Header

A headers plugin is responsible for extracting header items from the content. You can add metadata or augment the content in the header section here.

Triples

A triples plugin is responsible for extracting semantic triples from the source content. You can control the embedded triples in the envelope document.

Writer

A writer plugin is responsible for writing the final envelope to the database. You can control the output permissions, URI, collections etc. of the harmonized document with this module.

Upvotes: 2

Related Questions