CODEWITHSUNDEEP
jsonspringspring-bootspring-batchmarklogic
user1575601
user1575601

Reputation: 425

Spring Batch/Boot converting csv to multiple json and write to Marklogic database

I have requirement where reader being is used to convert a file to a composite object, I need to take different objects inside that object and write it as separate json file. That means for single line of csv file, there will be multiple jsons files created and that needs to be written to Marklogic database. I have used multiple item writer to convert a file to single output file, but now I need to split each line to multiple line and write same to marklogic database. Any idea how single line can be split to multiple files and written to Marklogic database.

example of composite object created out of Item reader, below is just an example not the actual issue scenario: 

    Person{
        HomeAddress homeadd;
        OfficeAddress officeAdd;
    }

A single line of csv represents home add and office add. I need in output two json files/objects (one for each type of add) written to Marklogic database. Thanks

Upvotes: 0

Views: 1223

Answers (1)

David Ennis -CleverLlamas.com
David Ennis -CleverLlamas.com

Reputation: 7770

If you were using MLCP to process the CSV to one record per line of CSV, then you could also define a transform rule on the input and hijack that process to parse/insert the additional documents.

You could also use a post-commit trigger and after the initial insert, process the documents into the required pieces. If this is high-volume, then you may decide to do this via Corb2.

You could pre-process the CSV into multiple csv files suitable for immidiate ingestion.

Considering all of the options above you could use the data movement SDK to author your solution: https://developer.marklogic.com/learn/data-movement-sdk (or even the MLCP/Hadoop related libraries)

Upvotes: 1

Related Questions