CODEWITHSUNDEEP
pentahokettlepentaho-spoonpdi
sachin
sachin

Reputation: 1

Not able to get system data at the end of transformation step

I wants to log transformation start time and end time into table. But I am getting error Field [start_date] is required and couldn't be found!.

Following steps I did.

Step 1 : Get Transformation name and system date from Get System Data as Transformation Start_Date.

Step 2 : Use Table Input to get count of records in table A.

Step 3 : Use Filter to check if table A is empty (Count = 0), if empty then copy of data from table B to Table A.

Step 4 : IF empty then control goes to Table Input to select all data from table B.

Step 5 : Use Table Output To insert data from Table Input.

Step 6 : Get system date from Get System Data as transformation End_date.

Step 7 : Use Table Output step to insert data into log table, Into this step I am inserting Transformation name,Start Date and End Date.

Can someone let me know where I am wrong. I am not able to get Start Date at the end of transformation. Following is the Diagram.

Transformation Diagram

Upvotes: 0

Views: 1109

Answers (1)

xkavator
xkavator

Reputation: 386

Table Input step ignores records that were generated before. In your diagram "Get_Transformation_name_and_start_time" generates a single row that is passed to the next step (the Table Input one) and then it's not propagated any further.

You can use a single "Get System Info" step at the end of your transformation to obtain start/end date (in your diagram that would be Get_Transformation_end_time 2). To get transformation start date you can use "with system date (fixed)" value. It will return the system time, determined at the start of the transformation, that will be common for all rows. You can use "system date (variable)" as an end timestamp (in case of more than one record you'll have to take max of these values).

It's probably worth looking at standard Pentaho logging options: http://wiki.pentaho.com/display/EAI/.08+Transformation+Settings#.08TransformationSettings-Logging , you can set-up a DB conection and a table that will store transformation execution data "out of the box".

Upvotes: 0

Related Questions