CODEWITHSUNDEEP
perlunixrenameaix
pete1450
pete1450

Reputation: 135

Is there a moment where a file "doesn't exist" during a rename?

We have a black-box third-party Java program that takes input files from a location and makes PDFs. It puts a manifest file in the same location every time for each input which necessitates us feeding the file in a controlled manner. Does the manifest (or .xen/.que) still exist? Don't feed an input file.

We're getting VERY rare (one out of tens of thousands of files) instances of our feed script not finding anything, feeding a file, and the resulting error when the manifest is overwritten and things don't match up. I wrote a perl script that does nothing but print the time down to 100-thousandths, glob anything in the directory that we care about, and print it. Below you can see .xen and .que files where .xen is the input and .que is a renamed version of it to indicate processing.

My question then is this: How is the lack of files at 94.26493 possible? Does the OS hide a file while it is renaming? We're getting our problem when the feed program looks for files at that moment so my planned hack is to check for files twice; hopefully slow enough to catch either end of the rename. I should also point out that once 2 files show up on a line, that is where the feed program has put another file in. It is not the same file as before the rename.

1421417394.26392/gpfs/fsdd/projects/corr_esch/corr_esch.d.xen
1421417394.26416/gpfs/fsdd/projects/corr_esch/corr_esch.d.xen
1421417394.26442/gpfs/fsdd/projects/corr_esch/corr_esch.d.xen
1421417394.26468/gpfs/fsdd/projects/corr_esch/corr_esch.d.xen
1421417394.26493
1421417394.26907/gpfs/fsdd/projects/corr_esch/corr_esch.d.xen.que_142_1421417394265
1421417394.27426/gpfs/fsdd/projects/corr_esch/corr_esch.d.xen /gpfs/fsdd/projects/corr_esch/corr_esch.d.xen.que_142_1421417394265
1421417394.27456/gpfs/fsdd/projects/corr_esch/corr_esch.d.xen /gpfs/fsdd/projects/corr_esch/corr_esch.d.xen.que_142_1421417394265
1421417394.27486/gpfs/fsdd/projects/corr_esch/corr_esch.d.xen /gpfs/fsdd/projects/corr_esch/corr_esch.d.xen.que_142_1421417394265
1421417394.27528/gpfs/fsdd/projects/corr_esch/corr_esch.d.xen /gpfs/fsdd/projects/corr_esch/corr_esch.d.xen.que_142_1421417394265

Upvotes: 11

Views: 415

Answers (1)

cjm
cjm

Reputation: 62109

The actual guarantee in POSIX is that if you rename a to b and b already exists, there will be no point in time during the rename when b does not exist. It will refer either to the previously existing b or the new b formerly called a.

If b does not already exist (which appears to be the case in your example), then the guarantee doesn't apply. It is possible that there's a moment when neither a nor b exists (it depends on how the particular filesystem works). It's also possible that there's a moment when both a and b exist (and refer to the same file).

Your proposed solution of checking twice with a short delay is probably the simplest approach.

Upvotes: 9

Related Questions