Loading Multiple files Parallelly in ODI12c

Question

I am working in ODI12c Project. I have a Scenario in which i need to load multiple files in a single table Parallelly. I tried through file List and using Loop in ODI package,but its loading data in serial i.e. 1st file then 2nd file etc. Please suggest how should i load data Parallelly.

Answer 1

Is your target table partitioned? The bottleneck you might be having could be due to the target destination being the same physical storage. Partitioning the target (or similar techniques) might help

Answer 2

I have an another approach to achieve this:

1. Create a simple package for loading the file.
2. Create a another package which is calling the another one.
3. Load all the files name in a oracle table like:

File_id Name

1 File1

2 File2

Now create a increment variable with pull the name of 1st file and then 2 . Now call the file loading package inside another package & make it asynchronous.

Pass the increment variable which is having the file name into model created for loading file ( resource Name).

Each time when variable is getting incremented it's gonna pull the next file and create a different session for each file loading . there might be a gap of 1 sec for each file loading because of increment of variables,i think that would be ok.

I hope you got the way in case any help you can add comment.

Answer 3

If you want to load multiple files in odi in one go you can merge all those files in one single file and load it in a single go. I'm considering all files have same structure. if you want to load them separately then creating separate model for each file will be headache and re usability of code will also compromised. You can run scenarios in asynchronous mode to made them run independent . I believe this is my best answer till now.