CODEWITHSUNDEEP
apache-nifi
Archon
Archon

Reputation: 1477

Apache NiFi - How to add/pass attributes to a Processor, not a flow file

My Purpose

Execute a sql and write result(flow file) using my own schema to a file directly.

Please see the explanation blow.

Solution 1 (use 4 processors)

  1. ExecuteSql and the records has auto-generated(embedded) avro schema.
  2. ConvertRecord: The Record Reader just use embedded avro schema and the Record Writer use my own schema from HortonworkSchemaRegistry, so pass attributes - 'schema.name' and 'schema.version' - by using UpdateAttribute.
  3. It works.

enter image description here

Solution 2 (use ExecuteSqlRecord)

It may like this:

enter image description here

ExecuteSqlRecord has Record Writer

enter image description here

And the Record Writer get avro schema from HortonworkSchemaRegistry using 'schema.name' and 'schema.version' attributes

enter image description here

But ExecuteSqlRecord not support user-define-attributes

enter image description here

So

Upvotes: 0

Views: 3586

Answers (1)

notNull
notNull

Reputation: 31540

As for now, Users cannot add new properties to ExecuteSQL* processors.

Below are the ways you can try

  1. Using GenerateFlowFile processor

    • Add schema.name attribute with some value.

Flow:

1.GenerateFlowFile //add schema.name attribute with value.
2.ExecuteSQLRecord
2.PutFile

(or)

  1. By hard code schema.name value in RecordWriter controller service. in this case you don't need GenerateFlowFile processor.

Flow:

1.ExecuteSQLRecord //hardcode schema.name property value
2.PutFile

Upvotes: 2

Related Questions