CODEWITHSUNDEEP
linuxcoredump
Underhill
Underhill

Reputation: 442

Core dump file appears then disappears

I have a program with a fairly large memory footprint, about 30GB, that core dumps.

While the program is dumping (top showing it as D - Defunct) the dump file is visible and readable. It's in the expected directory, with the expected file name format. Everything is on local disk. I can even load the file in gdb. Obviously the dump takes a few minutes on a process this size.

The file gets up to about 13GB, then the process exits ... and the core file disappears! Poof.

Platform: Redhat EL6 (2.6.32-220.4.2.el6.x86_64)

The file system plenty of space, over 500GB free.

$ df .
Filesystem           1K-blocks      Used Available Use% Mounted on
blahblahblah         834006872 270024424 521617316  35% /data

ulimit isn't showing any applicable restrictions:

$ ulimit -a
core file size          (blocks, -c) unlimited
file size               (blocks, -f) unlimited
<snipped>

Edit: Smaller core dumps don't disappear. Including for this same program. This seems to be specific to enormous core dumps.

tune2fs 1.41.12 (17-May-2010)
<snip>
Block size:               4096
Fragment size:            4096
Reserved GDT blocks:      1020
Blocks per group:         32768
Fragments per group:      32768
Inodes per group:         8192
Inode blocks per group:   512
<snip>

Upvotes: 3

Views: 645

Answers (1)

Vitaly Isaev
Vitaly Isaev

Reputation: 5805

Assuming that you're using the ext2/ext3/ext4, you could be faced with a restriction on the maximum file size for your file system. This page (warning: in Russian) tells that the lower limit for the range of valid values for a maximum file size in ext* file system equals to 16 GB (the exact value depends on your drive's cluster size). Therefore we can not exclude the fact that you have gone beyond that limit.

There are plenty of tools that may help to find out which process had wiped your file: lsof, inotify, auditd. They all are supported in your kernel and they all are different, and it hard to say now which one of them will fit you best.

Upvotes: 1

Related Questions