CODEWITHSUNDEEP
javahadoophdfs
abagshaw
abagshaw

Reputation: 6582

HDFS - put race condition

If I use copyFromLocalFile() to put a file in HDFS, with overwrite set to false, what will happen if a second process calls copyFromLocalFile() writing to the same location on HDFS also with overwrite set to false before the first put completes?

If the second call will throw an IOException is there a way I can determine that the exception was due to an existing put operation currently in progress and not because of network failure or some other issue?

Upvotes: 1

Views: 292

Answers (1)

xkrogen
xkrogen

Reputation: 654

HDFS allows for atomic creation, so the second write will fail.

(source: personal experience as an active HDFS developer)

The second call will throw a FileAlreadyExistsException as documented by ClientProtocol in its Javadoc.

Upvotes: 2

Related Questions