CODEWITHSUNDEEP
oracle-databasesoaosboracle-fusion-middlewarebigdata
NickName
NickName

Reputation: 313

Handling Big Data with OSB Proxy

I have created a OSB Proxy Service(Messaging Service) which loading the data with a MFL file.

The format of data is:

1/1/2007;00:11:00;2.500;0.000;242.880;10.200;0.000;0.000;0.000;
1/1/2007;00:12:00;2.494;0.000;242.570;10.200;0.000;0.000;0.000;

All the data records are : 2075259

The total size of file(.txt or .data) is : 130MB.

Which is best way to handling all these data in order to inserted to an OSB Proxy and transformed all the data in a simple xml file?

I have tested with a small size of records(5000) and it works as expected but how i should insert all this data in the proxy?

The MFL transformation is a valid idea or i should create a FileAdapter Proxy which will received the data from a dbtable?

Please for your suggestion

Thank you in advance.

Upvotes: 1

Views: 398

Answers (1)

jmhostalet
jmhostalet

Reputation: 4639

ESBs are efficient at handling messages in the order of KBs, not MBs, although this is very subjective and depends a lot on the number of concurrent requests, transactions per second, sizing of hardware etcetera. As Trent points out in a comment, you could implement a claim check pattern and delegate the file transformation to an external utility, such as perl or similar.

Upvotes: 2

Related Questions