CODEWITHSUNDEEP
linuxbufferstat
phoxis
phoxis

Reputation: 61950

Can 'stat' ing a large number of files cause performance effect due to practically flushing the cache?

Can repetitive sequential stating on each file cause the buffer cache or the slab objects to evict old entries and/or grow the slab causing noticeable performance difference, except the disk I/O? For example the buffer cache or the slab for Linux is in a state where they have the objects loaded which are mostly active. 'stat' ing files on the disk (say all of them) will bring the inode and dentry objects on the cache. Because they are large in number I think they may evict the older entries and fill the cache up. Once this is done, the old entries which were evicted, will have to be reloaded from disk when the corresponding application accesses it.

My question is does this kind of stating for example repetitive du usage on a huge number of files make a noticeable effect on the system? That depends on the size of the cache, I understand, my main objective is to understand if repetitive 'du' on a huge number of files will effect the performance of the system noticeably. Also indicate if any other indirect effects can be seen.

Upvotes: 3

Views: 95

Answers (1)

Alexander Dzyoba
Alexander Dzyoba

Reputation: 4209

IMHO, that depends on cache replacement policy. If it's FIFO, then yes - duing will replace older entries with newer. But if it's LRU, which is much more common, then, i think, it's unlikely to happen - I (want to) believe that I/O on files have higher priority for LRU time/reference counting then stating. Or maybe that caches are completely separate. Anyway, some thoughtful kernel source code reading are strongly recommended.

Upvotes: 1

Related Questions