CODEWITHSUNDEEP
wso2integration
user5568021
user5568021

Reputation: 17

WSO2 integration approach

We are planning to use WSO2 in the scenario where we are receiving a file from people soft application with customer/user details (file size: approx 2GB) and that need to be inserted in SAP success factor. We want to use the WSO2 product for integration between People soft and SAP. We were looking into some of the WSO2 product to achieve this, like DSS for streaming file or batch processing and ESB to use for our scenario.

Can DSS/ESB help in our scenario for streaming the data from file and call SAP webservice to create user?

Is there any approach in WSo2 Product to read row by row from source (here txt /csv file) and then do transformation and then call webservice to create data in target system. Please advise.

Upvotes: 0

Views: 120

Answers (1)

Jean-Michel
Jean-Michel

Reputation: 5946

You can read a file in streaming mode with WSO2 ESB (with a "VFS" proxy) and use smooks mediator with a smooks config file describing your csv structure.

You will find different samples over the net, one of them : http://vvratha.blogspot.fr/2014/05/processing-large-text-file-using-smooks.html

In most of the samples, when smooks split the content of your huge file in small parts, routing fragments in JMS or in other files, you will find a "highWaterMark" in the config with an attribute named "mark" with a huge value : you absolutly need to replace this value with -1 to avoid poor performance

Upvotes: 1

Related Questions