Why does apache mod_perl process become a zombie?

Question

Occasionally mod_perl apache process is marked "defunct" in "top" utility, that is becomes a zombie process. Is it a correct behavior? Do I have to worry about it?

Our Perl script is very simple, it does not spawn any child processes. The zombie process disappears pretty quickly. Apache2, Ubuntu.

Our apache config is here: apache_config.txt

Here is a snap-shot of top.

    PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
19525 www-data  20   0 55972  25m 4684 S 10.3  2.4   0:00.32 apache2
19486 www-data  20   0 52792  21m 4120 S  1.7  2.1   0:00.05 apache2
19538 www-data  20   0 52792  21m 4120 S  1.3  2.1   0:00.04 apache2
19539 www-data  20   0     0    0    0 Z  0.7  0.0   0:00.03 apache2 <defunct>
19481 www-data  20   0 52860  21m 4016 S  0.3  2.1   0:00.05 apache2
19521 www-data  20   0 52804  21m 3824 S  0.3  2.1   0:00.08 apache2

These are CPAN modules I use

CGI();
XML::LibXML();
DateTime;
DateTime::TimeZone;
Benchmark();
Data::Dump();
Devel::StackTrace();
DBD::mysql();
DBI();
LWP();
LWP::UserAgent();
HTTP::Request();
HTTP::Response();
URI::Heuristic();
MD5();
IO::String();
DateTime::Format::HTTP();
Math::BigInt();
Digest::SHA1();


top:
26252 www-data  20   0     0    0    0 Z  0.3  0.0   0:00.22 apache2 <defunct>

access.log with pid logged as the first parameter:
26252 85.124.207.173 - - [26/Dec/2009:22:16:42 +0300] "GET /cgi-bin/wimo/server/index.pl?location=gn:2761369&request=forecast&client_part=app&ver=2_0b191&client=desktop&license_type=free&auto_id=125CC6B6DAA HTTP/1.1" 200 826 0 "-" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; Trident/4.0; GTB6.3; Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1) ; .NET CLR 2.0.50727; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729)"

3 different zombie processes logged by server-status

Srv     PID     Acc     M   CPU     SS  Req ConnChild   Slot    Client      VHost           Request
32-0    1300    0/0/45  _   0.00    0   0   0.0 0.00    2.29    127.0.0.1   weather_server  OPTIONS * HTTP/1.0
100-0   1254    1/7/41  C   0.22    0   0   0.0 0.00    1.51    127.0.0.1   weather_server  OPTIONS * HTTP/1.0
29-0    1299    0/12/78 _   0.31    0   2   0.0 0.78    2.37    [my ip was here]    weather_server  GET /server-status HTTP/1.1

Answer 1

I see that too with my very simple mod_perl 2 module. I do not fork anything, just write a string to client socket and then return OK. And still a defunct process appears at my CentOS 5.5 Linux VM and then goes away. Here is my source code and you can test it with "telnet yourhost 843" and pressing ENTER:

package SocketPolicy;

# Run: semanage port -a -t http_port_t -p tcp 843
# And add following lines to the httpd.conf
# Listen 843
# <VirtualHost _default_:843>
#       PerlModule                   SocketPolicy
#       PerlProcessConnectionHandler SocketPolicy
# </VirtualHost>

use strict;
use warnings FATAL => 'all';
use APR::Const(-compile => 'SO_NONBLOCK');
use APR::Socket();
use Apache2::ServerRec();
use Apache2::Connection();
use Apache2::Const(-compile => qw(OK DECLINED));

use constant POLICY =>
qq{<?xml version="1.0"?>
<!DOCTYPE cross-domain-policy SYSTEM
"http://www.adobe.com/xml/dtds/cross-domain-policy.dtd">

<cross-domain-policy>
<allow-access-from domain="*" to-ports="8080"/>
</cross-domain-policy>
\0};

sub handler {
        my $conn   = shift;
        my $socket = $conn->client_socket();
        my $offset = 0;

        # set the socket to the blocking mode
        $socket->opt_set(APR::Const::SO_NONBLOCK => 0);

        do {
                my $nbytes = $socket->send(substr(POLICY, $offset), length(POLICY) - $offset);
                # client connection closed or interrupted
                return Apache2::Const::DECLINED unless $nbytes;
                $offset += $nbytes;
        } while ($offset < length(POLICY));

        my $slog = $conn->base_server()->log();
        $slog->warn('served socket policy to: ', $conn->remote_ip());
        return Apache2::Const::OK;
}

1;

Answer 2

My first suspicion is that you really are forking but maybe you don't realize it. Is it possible for to include your code? Remember that any system or `` calls are forking. This could easily be happening inside a CPAN module without you realizing. There is some useful information about mod_perl and forking (including how zombies are created and how to avoid them) here.

Update: try adding this to your config:

# Monitor apache server status
ExtendedStatus On
<VirtualHost 127.0.0.1:80>
    <Location /server-status>
        SetHandler server-status
        Order deny,allow
        Deny from all
        Allow from 127.0.0.1
    </Location>
</VirtualHost>

And then change the Allow from to be your IP, then you can visit http://yourdomain.com/server-status and get a page of summary information on apache. Try doing this when you see one of the zombies and look to see what apache thinks that process is doing.