Talend tMap Set Default Value for Rejected Inner Joins and connect them with the main data flow

Question

i've got the following problem.

I have several tMaps, each has a lookup and at the end all the data is written in a db. The following mockup shall illustrate it:

There can be values in the main data stream which are not found in the lookup tables. For this values there is a rejected path which catches them from the specific tMap.

Requirements:

In case of a rejected inner join the looked up value shall be set to a default value (for example 0, which could be done in the schema of the tMap) and after that these "corrected" records should be added to the "normal" main data flow and process the next lookup. The tUnite component is not able to handle this cases because it can not exist in a data flow loop.

Does anybody got an idea how to solve this problem?

Cheers.

Answer 1

The answer was so easy that i didn't got it in the first conception. I just have to change the join model from inner to left-join so all the formal rejected values will have a null value in it. Afterwards i can check the columns in the tmap and set them on a default value if they are null.

row1.id == null ? 0 : row1.id

Cheers.

Answer 2

If I understand correctly what you are trying to accomplish you will have to have staging files or staging tables on the database. Once you get the rejected rows, write them on a file or table. The accepted files will go also to a staging table(different than the rejected). Then you can union both tables or files by reading them. The key point is having a staging structure. I attach a picture what how would it be. In the picture the staging structure is a mysql table.

Let me know if it helps!