CODEWITHSUNDEEP
linuxperformancetestingcachingfile-io
Paweł Hajdan
Paweł Hajdan

Reputation: 18542

How to evict file from system cache on Linux?

When running performance tests file system cache hit or miss can significantly influence test results. Therefore generally before running such tests used files are evicted from system cache. How to do that on Linux?

Clarification: If possible, the solution should not require root privileges.

Upvotes: 6

Views: 4817

Answers (5)

user143174
user143174

Reputation: 61

There is a command line utility by Eric Wong that makes it easy to invoke posix_fadvise:

http://git.bogomips.org/cgit/pcu.git/tree/README

It's then as simple as

$ pcu-fadvise -a dontneed filename-to-evict

Upvotes: 6

James
James

Reputation: 999

If you can put the test data in a separate filesystem then mounting the filesystem afresh for the test will give you empty caches.

If you list the test fileystem in /etc/fstab with the "user" option then you can mount it for the test without being superuser

Upvotes: -1

Paweł Hajdan
Paweł Hajdan

Reputation: 18542

Ha, I have the answer:

#include <unistd.h>
#include <fcntl.h>
int main(int argc, char *argv[]) {
  int fd;
  fd = open(argv[1], O_RDONLY);
  fdatasync(fd);
  posix_fadvise(fd, 0,0,POSIX_FADV_DONTNEED);
  close(fd);
  return 0;
}

This is from http://insights.oetiker.ch/linux/fadvise.html

Upvotes: 10

DGentry
DGentry

Reputation: 16258

Regarding use of O_DIRECT: that would perturb the results in another way. The kernel will attempt to DMA the filesystem data directly into your read() buffer, so it can be handed up to your application without any additional copy being done. Without O_DIRECT the kernel DMAs the file data into the page cache, and copies it from the page cache to your read() buffer.

This is fine if your app is really going to use O_DIRECT in production. If you run performance tests with O_DIRECT and then remove O_DIRECT for production, your performance test will be unrealistic.

Upvotes: 0

Mecki
Mecki

Reputation: 132869

As a superuser you can do the following:

To free pagecache:

  • echo 1 > /proc/sys/vm/drop_caches

To free dentries and inodes:

  • echo 2 > /proc/sys/vm/drop_caches

To free pagecache, dentries and inodes:

  • echo 3 > /proc/sys/vm/drop_caches

This operation will not "lose" any data (caches are written out to disk before their data is dropped), however, to really make sure all cache is cleaned, you should sync first. E.g. all caches should be cleared if you run

sync; echo 3 > /proc/sys/vm/drop_caches

As I said, only a superuser (root) may do so.

Upvotes: 10

Related Questions