CODEWITHSUNDEEP
google-cloud-dataflow
G B
G B

Reputation: 755

Are cloud dataflow job outputs transactional?

Assuming I don't know the job status that was supposed to generate some output files (in cloud store), can I assume that if some output files exist they contain all of the job's output? Or it's possible that partial output is visible?

Thanks, G

Upvotes: 0

Views: 71

Answers (1)

Marián Dvorský
Marián Dvorský

Reputation: 474

It is possible that only a subset of the files is visible, but the visible files are complete (cannot grow or change).

The filenames contain the total number of files (output-XXXXX-of-NNNNN), so once you have one file, you know how many more to expect.

Upvotes: 2

Related Questions