CODEWITHSUNDEEP
linuxnfs
Tomoya Kabe
Tomoya Kabe

Reputation: 1217

NFS cache-cleaning command?

I have a trouble with NFS client-side attribute caching. I'm using some servers, one is an NFS server and the others are NFS client servers.

All servers are Debian(lenny, 2.6.26-2-amd64 of Linux) and versions are following.

 % dpkg -l | grep nfs
ii  libnfsidmap2                        0.20-1                     An nfs idmapping library
ii  nfs-common                          1:1.1.2-6lenny1            NFS support files common to client and server
ii  nfs-kernel-server                   1:1.1.2-6lenny1            support for NFS kernel server

In the NFS server, /etc/exports is written as following:

/export-path   192.168.0.0/255.255.255.0(async,rw,no_subtree_check)

In the NFS clients, /etc/fstab is written as following:

server:/export-path     /mountpoint   nfs rw,hard,intr,rsize=8192,async 0 0

As you can see, "async" option is used for multi-clients access performance. However, sometimes this can cause false-caching errors.

Since I am maintaining many servers (and I have not so strong permission to change the mount options), I don't want to modify /etc/exports nor /etc/fstab. I think it is sufficient if I have a command-line tool that "cleans" NFS client-side attribute cache with a user permission.

Please let me know if there such commands.

Thanks,

(Appended)

I mean by "false-caching errors",

 % ls -l /data/1/kabe/foo                  
ls: cannot access /data/1/kabe/foo: No such file or directory
 % ssh another-server 'touch /data/1/kabe/foo' 
 % ls -l /data/1/kabe/foo
ls: cannot access /data/1/kabe/foo: No such file or directory

Sometimes such cases happen. The problem is not a file content but file attributes(=dentries information) since NFS says it guarantees Close-to-Open consistency.

Upvotes: 24

Views: 70040

Answers (5)

Erik Aronesty
Erik Aronesty

Reputation: 12915

Within a given process, calling opendir() and closedir() on the parent directory of a file invalidates the NFS cache. I used this while programming a job scheduler. Very, very helpful. Try it!

This is the line number of the relevant code (showing the use in context): https://github.com/earonesty/grun/blob/master/grun#L820

It was the only way I could fix the issue of job #1 completing and job #2, which needed some output files, firing off in a context where those files were visible,

Upvotes: 16

Jeff Taylor
Jeff Taylor

Reputation: 326

You're seeing the effects of NFS's attribute cache. See man nfs, and check out DATA AND METADATA COHERENCE.

NFS by default caches attributes for a minimum of 30 seconds (acregmin and acdirmin) and a maximum of 60 seconds (acregmax and acdirmax). You can override all of these together with actimeo, or disable the attribute cache entirely with noac. With the noac mount option, the behaviour described by the OP goes away, but hits performance.

lookupcache=positive is useful if you're just looking for the appearance of new files, but deletions will still go through the attribute cache.

Upvotes: 2

ArtemGr
ArtemGr

Reputation: 12547

AFAIK, the sync and async options aren't the source of attribute caching. Async allows the server to delay saving data to server filesystem, e.g. it affects the write durability in case of NFS server failures, but if the NFS server is stable then async does not affect the NFS clients.

There is a lookupcache=positive NFS mount option that might be used to prevent negative lookup caching, e.g. the NFS returning "No such file or directory" when the file actually exists on the server. See Directory entry caching in man nfs.

Upvotes: 7

Bin
Bin

Reputation: 13

clear /var/lib/nfs/rmtab file on nfs server.

The below commands are used to clear memory related problems. and it is very dangerous too. soem times it will crash ur application hosted on the box

# sync

# To free pagecache
echo 1 > /proc/sys/vm/drop_caches

# To free dentries and inodes
echo 2 > /proc/sys/vm/drop_caches

# To free pagecache, dentries and inodes
echo 3 > /proc/sys/vm/drop_caches

Upvotes: -2

Mr_Pink
Mr_Pink

Reputation: 109417

Depending on what you mean by "false-caching errors", running sync may get you what you need. This will flush all filesystem buffers.

If needed, you can also clear out the VM caches in the kernel using /proc/sys/vm/drop_caches. 

# To free pagecache
echo 1 > /proc/sys/vm/drop_caches

# To free dentries and inodes
echo 2 > /proc/sys/vm/drop_caches

# To free pagecache, dentries and inodes
echo 3 > /proc/sys/vm/drop_caches

Upvotes: 27

Related Questions