CODEWITHSUNDEEP
talend
Guillermo Teixeira
Guillermo Teixeira

Reputation: 305

How to check if the stream of rows has ended

Is there a way for me to know if the stream of rows has ended? That is, if the job is on the last row? What im trying to do is for every 10 rows do something, my problem are the last rows, for example in 115 rows, the last 5 wont happen but i need them to.

Upvotes: 0

Views: 202

Answers (1)

Ibrahim Mezouar
Ibrahim Mezouar

Reputation: 4051

There is no built-in functionality in Talend which tells you if you're on the last row. You can work around this using one of the following:

  • Get the row count beforehand. For instance, if you have a file, you can use tFileRowCount to count the number of rows, then when you process your file, you use a variable for your current row number, and so you can tell if you've reached the last row. If your data come from a database, you could either issue a query that returns the total number of rows beforehand, or modify your main query to return the total number of rows in an additional column and use that (using ranking functions).

  • Do some processing after the subjob has ended: There may be situations where you need a special processing for the last row, you can achieve this by getting the last row processed by the previous subjob (which you have already saved, for instance, by putting a tSetGlobalVar after your target, when your subjob is done, your variable contains the last written value).

Edit

For your use case, what you could do is first store the result of the API call in memory using tHashOutput, then read it with a tHashInput in order to process it, and you'll know then how many rows you have retrieved using tHashOutput's global variable tHashOuput_X_NB_LINE.

Upvotes: 1

Related Questions