CODEWITHSUNDEEP
etlpentahokettlepentaho-spoonpentaho-data-integration
Prakhar Gajbhiye
Prakhar Gajbhiye

Reputation: 80

Pentaho Kettle Variable Update

In light words the problem could be stated as "I am setting an environment variable in one transformation t1.ktr and using the variable in another transformation t2.ktr. I want to update the variable every 15 mins without actually stopping the t2.ktr i.e. during the execution of t2.ktr . How can I achieve that ?

To give you an overview :
I am making rest api call using HTTP POST step in my transformation (multi-threaded). To make this rest api call I need to pass a certain Token which gets expired in every 15 mins. I am getting this token via another API call. So in one transformation lets say token.ktr I am getting the token and storing it in environment variable TOKEN via Set Variable step and in next transformation lets call it rest.ktr I am getting this variable via Get Variable step and using it in HTTP POST call.

For 15 min I get proper responses, but after that I get error responses since token gets expired.

Let me know if further clarification is needed.

Upvotes: 1

Views: 1344

Answers (1)

Cristian Curti
Cristian Curti

Reputation: 1054

I've had a similar case that i produced a sequence of KTRs in a Job with Evaluations. I achieved this with a Job and 2 KTRs

DISCLAIMER - For my case, i have to make several HTTP GET's, not just a single long one, so i can easily loop from one GET to another and check the Expiration of the Token bettween HTTP GETs, this is how i achieved it.

Step 1:

Start your KTR that has the Auth API Call. In this KTR you'll also use a formula step, creating a DateTime row using the NOW() function, this will get the timestamp of when you called the Authorization. This KTR will end by setting the Auth and this timestamp as variables in the Parent job.

KTR1 - Example enter image description here

Step 2:

In the Job you will call this KTR first, and right after it, you'll use a Set Variables step, i named this variable Expiration, and here you'll set it to OK, signaling Token not expired. Next you'll call the KTR that makes the HTTP GET, using the Token. The result of this KTR will be Success (not expired token, success on HTTP GET, moves on), or Fail (Token expired, set Expiration to "Expired").

JOB Example enter image description here

Step 3:

After the HTTP GET you need to check to result, whether or not it succeeded, or if the token expired mid way, so you would need to renew token and continue the loop, or end the loop if no more HTTP GETs are needed.

Again, this is for a very especific use case, and i'm sure other people can do better, but, this is my take on the issue, it works for me.

Upvotes: 2

Related Questions