CODEWITHSUNDEEP
talendlogfile
Ajay
Ajay

Reputation: 1

How to read log file and load to snowflake table in talend

which talend component is used to read log file. log file is Talend cloud generated log file

read log file and load to snowflake table using talend job. not able to identify which component will help to read and parse log file & normalize the data

Upvotes: 0

Views: 144

Answers (1)

Emile Dadou
Emile Dadou

Reputation: 554

first I don't think there is a special component that read this kind of file, and you should add what you already tried.

It's something that I had to do and it's a bother to parse and use the file that you download from the TMC.

I worked around it to have a file (logs) that are easy to use. Talend / TMC has a OpenAPI that can be used for this kind work, here is the documentation :

https://api.eu.cloud.talend.com/tmc/swagger/swagger-ui.html#/executions%20%3A%3A%20logs/getTaskExecutionLog

And with it you can directly get thet logs of a job executions in a json format.

So you can use a component like tRest to get the data via API and then use a tExtractJSONFields to get the data :

enter image description here

And here is how to access the data :

enter image description here

Hope this helps

Upvotes: 0

Related Questions