Ashok
Ashok

Reputation: 1

ApcheBeam Dataflow(Java) - How Avoid writing empty file into GCS if record count is zero

I am trying to write CSV file into GCS, it is working fine with the data, if rows count is zero still empty file been written into bucket, I want to avoid writing empty file into GCS. How to avoid writing empty file into GCS if there are no rows.

I was referring same as old question How to write a Beam condition based on the size of a PCollection

Below is my sample code.

  public WriteFilesResult<String> writeToGCS(
     PCollection<Row> rows, String fileName String filePath) {
     return rows.apply(
        "output".concat(fileName),CsvIO.writeRows(filePath, CSVFormat.DEFAULT));

}

Upvotes: 0

Views: 13

Answers (0)

Related Questions