CODEWITHSUNDEEP
error-handlingapache-nifi
Apple
Apple

Reputation: 1

how to capture errors of any processor of the workflow within the processor group in NiFi

I have a flow within a processor group. so whenever there is an error need to capture those errors. Firstly, let me explain my flow. through getfile ill fetch the excel file from nifi server, then using updateattribute processor i capture UUID value using this condition from filename (${filename:substringAfterLast('_'):substringBefore('.')}). then using convertexceltoCSVProcessor file gets converted from excel to csv. then using convertrecord processor csvfile gets converted to json format. then using splitjson processor the records gets split. Now using evaluatjsonpath processor. attributes gets mapped.and in replacetext processor i write insert query statement and then in putsql processor using DBCP connection i send the records to the desired table in postgresql. And the success queue of putsql processor is connected to one executesql processor.In executesql i write SQLselect query like this: insert into xplorer.discovery_template_upload_status (id,file_name,job_status,is_successful) values('${UUID_1}','${filename}','Completed','Yes') ON CONFLICT (id) DO UPDATE SET file_name='${filename}',job_status= 'Completed',is_successful='Yes'. and the failure,retry queue of putsql is connected another executesql processor and i wrote SQLselect query like this :insert into xplorer.discovery_template_upload_status (id,file_name,job_status,is_successful) values('${UUID_1}','${filename}','Completed','No') ON CONFLICT (id) DO UPDATE SET file_name='${filename}',job_status= 'Completed',is_successful='No'. so this is how i created a flow. now whenever there is any failure or error trigged at any of processor. need to capture those errors within the processor group only. how to achieve this. I hope your answer will solve my requirement.

tried using S2S (site to site reporting task) which is on root level to capture logs but I need for this particular processor group.

Upvotes: 0

Views: 154

Answers (1)

mattyb
mattyb

Reputation: 12083

Apache NiFi (as of this writing, current version 1.23.1) does not currently have the capability to filter in the Bulletin reporting task but I have written it up as an improvement under NIFI-11974. I think the bulletin data contains the process group information (bulletinGroupId), you could use QueryRecord to filter the records after you bring in all the bulletins via the reporting task, or you could add the nifi-sql-reporting-nar and use QueryNiFiReportingTask to search the BULLETIN table using SQL filtering on your process group ID (it should be in the bulletinGroupId column), then only the bulletins you're interested in should come into the Input Port.

Upvotes: 0

Related Questions