CODEWITHSUNDEEP
linuxbashunixfind
David542
David542

Reputation: 110163

What accounts for increase in time to execute -type f in find

I have done the following commands, on a combination of Volumes which contains about 1M files.

$ sudo find "$FULFILLMENT" "$ARCH1" "$ARCH2" "$MASTERING"
Tue Jan 29 15:04:05 PST 2013
Tue Jan 29 15:22:46 PST 2013
18m41s


$ sudo find "$FULFILLMENT" "$ARCH1" "$ARCH2" "$MASTERING" -type f
Tue Jan 29 15:24:06 PST 2013
Tue Jan 29 15:49:10 PST 2013
25m4s

Why does using -type f take so much longer than the normal find command? I would think it would be quicker, since it didn't have to iterate through all the files (perhaps 20% less files). What accounts for the increase in time?

Upvotes: 1

Views: 101

Answers (2)

Austin Phillips
Austin Phillips

Reputation: 15756

Without the -type f parameter, find will not stat each file, but use the readdir system call to get directory entries, reducing the number of system calls, context switches, inode lookups etc.

It's possible for find to use a query optimizer in this specific case. Look up the -O2 option in the man page for find. Your query would become:

$ sudo find -O2 "$FULFILLMENT" "$ARCH1" "$ARCH2" "$MASTERING" -type f

This should have the same performance of the find without the -type f since it will try to use the file type information returned by the readdir system call if available rather than requiring a separate stat call for every file.

Further background:

Not all file systems support returning the file type information in the readdir call, and not all versions of find will be compiled with support for this extended type information.

If find --version reports the D_TYPE feature, then your find supports reading this additional type information. File system types ext2, ext3, ext4 support the D_TYPE field.

Upvotes: 4

Ed Heal
Ed Heal

Reputation: 59997

It would have to iterate through the directory and do a stat on them to find out if they are files (not directories, symbolic links etc).

PS: I am making some assumptions over the values "$FULFILLMENT" "$ARCH1" "$ARCH2" "$MASTERING"

Upvotes: 1

Related Questions