CODEWITHSUNDEEP
c++clinuxshellsh
A R
A R

Reputation: 2803

Error while running a shell script : "pipe call failed"

I am executing a shell script in an infinte loop. Script is meant to get the connected devices. But when it runs for some time, it will stop executing after displaying an error message pipe call failed.

This is the line where I am getting this error

arp -n -i eth1 | grep "?" | awk '{print $4}' > out.txt

Am I doing anything wrong in redirecting this output to a file? Is there a way to handle file descriptors in a shell script?

EDIT: Here is my script

HOME_NETWORK_INTERFACE=eth1
echo "Generating list of MoCA device MACs..."
if [ $# -lt 1 ]; then
        echo "Error! Insufficient arguments. Format is $0 <output file path>"
        exit 1
fi
arp -n -i $HOME_NETWORK_INTERFACE | grep "?" | awk '{print $4}' > $1
echo "Done"

EDIT :

I found that this issue is due to the usage of "system" command in cpp application. If this script is executed using "system", say 1000 times, i will get this error. Does any one have any idea about this? Thanks in advance

Upvotes: 2

Views: 1875

Answers (4)

prisco.napoli
prisco.napoli

Reputation: 616

The error message "pipe call failed" means that there is no more free memory to allocate. As suggested by Dale, check the limits of your system using ulimit -a and verify in which way the system command is called: sequentially or in parallel.

Pipes have a size of 64KB starting form the kernel 2.6.11. Moreover, the system() command executes a command by calling /bin/sh -c command and returns after the command has been completed. Is not very efficient to call the system command 1000 times, so my advice is to add a while cycle inside your script to avoid weak performance (you can provide the number of iterations as a second argument).

Finally, it is a good practice to add a shebang (#!/bin/sh) as the first line of the script if it is intended to be run as an executable. This is not strictly necessary, because if the shebang is missing, the default shell (e.g. /bin/sh) is used.

#!/bin/sh

HOME_NETWORK_INTERFACE=eth0
echo "Generating list of MoCA device MACs..."
if [ $# -lt 1 ]; then
        echo "Error! Insufficient arguments. Format is $0 <output file path>"
        exit 1
fi

X=0
while [ $X -lt $2 ]
do
    arp -n -i $HOME_NETWORK_INTERFACE | grep "?" | awk '{print $4}' > $1
    X=`expr $X + 1`
done
echo "Done"

Upvotes: 2

A R
A R

Reputation: 2803

Finally i found the issue. I am using socket communication in my application. Some how some sockets were not closing properly.so after some time file descriptors will exhaust which is leading to the issue "pipe call failed" because pipe also uses descriptors.

Upvotes: 0

unbesiegbar
unbesiegbar

Reputation: 461

"alvits" has exactly pointed out the issue with using system command. Usually we try to implement the equivalent "C" code rather than using system command so as to avoid memory spikes.

It seems that you are using a while loop into you C++ code and calling this script. Just try if you can run the script in while loop (resulting into only a single system call), redirect the o/p to a file and the read the file continuously for the desired result.

Upvotes: 2

Dale_Reagan
Dale_Reagan

Reputation: 2061

I don't see any obvious issues with your script; the error messages does 'sound like' a system resource problem (and these values may vary considerably between systems.) Some suggestions:

  • Consider using 'fgrep' when special characters are involved and you need to negate possible shell variable munging (don't think that's a factor here, but just in case...)

  • Check the 'limits' for the user/process you are running using ulimit -a

  • you could be 'consuming' more open files (guessing) than allowed OR it could be that the 'system' is having the issue (instead of the 'user process'); the 'real' problem could be some other process running at the same time is consuming the resource.

  • If your system/user 'limits' are ok and Since you state that this is an 'infinite loop' then I will guess that you are spawning multiple processes instead of running only one. Once you reach the 'hard limit' for pipe or file-related resources the script fails and you get a 'system error'.

    $ ulimit -a | awk '{printf "%3d | %s\n", NR, $0}'

1 | core file size (blocks, -c) 0
2 | data seg size (kbytes, -d) unlimited
3 | scheduling priority (-e) 0
4 | file size (blocks, -f) unlimited
5 | pending signals (-i) 46232
6 | max locked memory (kbytes, -l) 64
7 | max memory size (kbytes, -m) unlimited
8 | open files (-n) 1024
9 | pipe size (512 bytes, -p) 8
10 | POSIX message queues (bytes, -q) 819200
11 | real-time priority (-r) 0
12 | stack size (kbytes, -s) 10240
13 | cpu time (seconds, -t) unlimited
14 | max user processes (-u) 1024
15 | virtual memory (kbytes, -v) unlimited
16 | file locks (-x) unlimited

Upvotes: 2

Related Questions