CODEWITHSUNDEEP
apache-nifi
Ram
Ram

Reputation: 35

Validate CSV schema for CSV validation

I am converting a JSON to csv and while pushing it to putfile I want to validate my csv before pushing. How should I give Schema in ValidateCSV processor to do that.

Below is the sample JSON and need validate it before pushing it to putfile.

[
  {
    "name": "Tony",
    "age": 20,
    "regdate": "2022-07-01 02:15:15",
    "due_date": "2021-05-01 03:30:33",
    "start_date": "2021-05-01 03:30:33"
  },
  {
    "name": "Steve",
    "age": 21,
    "regdate": "2022-03-01 05:22:15",
    "due_date": "2022-03-01 05:22:15",
    "start_date": "2022-04-01 02:30:33"
  },
  {
    "name": "Peter",
    "age": 23,
    "regdate": "2021-08-06 02:20:15",
    "due_date": "2022-01-03 05:30:33",
    "start_date": "2022-01-03 05:30:33"
  }
]

I have given schema in JSON and CSV values but below is the error

enter image description here

my schema is name,age,start_date,regdate,due_date

Suggest a valid schema for me to proceed further.

Upvotes: 0

Views: 726

Answers (1)

tonykoval
tonykoval

Reputation: 1247

Check the documentation of the ValidateCsv processor:

The cell processors cannot be nested (except with Optional which gives the possibility to define a CellProcessor for values that could be null) and must be defined in a comma-delimited string as the Schema property.

Input to processor (after ConvertRecord Json -> CSV):

name,age,regdate,due_date,start_date
Tony,20,2022-07-01 02:15:15,2021-05-01 03:30:33,2021-05-01 03:30:33
Steve,21,2022-03-01 05:22:15,2022-03-01 05:22:15,2022-04-01 02:30:33
Peter,23,2021-08-06 02:20:15,2022-01-03 05:30:33,2022-01-03 05:30:33

For your use case probably schema:

StrNotNullOrEmpty(),ParseInt(),ParseDate("yyyy-MM-dd HH:mm:ss"),ParseDate("yyyy-MM-dd HH:mm:ss"),ParseDate("yyyy-MM-dd HH:mm:ss")

Upvotes: 1

Related Questions