CODEWITHSUNDEEP
apache-nifi
salvob
salvob

Reputation: 1370

NiFi 1.5.0 : GenerateTableFetch order results

Is it possible getting a dataset from a Database and order them? I have a very big table (72milion of rows) and I need to get them in a parallel way (as the GenerateTableFetch does) but ordered by a column, in my case "url" ?

GenerateTableFetch setup

Upvotes: 0

Views: 423

Answers (1)

mattyb
mattyb

Reputation: 12093

If you provide a Maximum-value Column, then the queries will be ordered on that column (for incremental fetching). You may be able to try adding your ORDER BY clause to the "Additional WHERE clause" section.

As of NiFi 1.7.0 (not yet released at the time of this writing) you can specify arbitrary queries in QueryDatabaseTable (via NIFI-1706), please feel free to write a corresponding Jira for GenerateTableFetch. I'm not sure about the logistics of adding it to GTF, but if it works for QDT, I don't see why we can't add it to GTF.

Upvotes: 2

Related Questions