CODEWITHSUNDEEP
linuxshellpipe
Kir Chou
Kir Chou

Reputation: 3080

Max Pipe Commands of Linux

I wrote a simple shell recently. This problem came up while I implemented the pipe.

I knew the maximum length of command line argument in Ubuntu is 2097152. (By this)

#define MAX_CMD_LEN 2097152

But I wanted to know is there a maximum number of pipe commands? For example: (the number of n)

ps -aux |　grep "a.out" | awk '{print $5}' | top | ... | cat
1       |  2            | 3                | 4   | ... | n

Upvotes: 2

Views: 4098

Answers (2)

Basile Starynkevitch
Basile Starynkevitch

Reputation: 1

There are not strict limits on the length of the pipeline (that is the number of commands composing it). However, the maximum number of processes can be limited with setrlimit(2) RLIMIT_NPROC and each system has some absolute limit. Likewise, the number of file descriptors can be limited with RLIMIT_NOFILE

Also, your shell will call pipe(2), and that syscall can fail (and this is practically limiting).

BTW, the maximal command line should better not be a hardwired limit (it can be changed somehow). The limitation is when execve(2) fails. And you could use sysconf(_SC_ARG_MAX) -see sysconf(3)- to query something related.

At last proc(5) could be used to query some related thresholds, e.g. using /proc/sys/fs/pipe-max-size, /proc/sys/kernel/core_pipe_limit, etc.

The bottom line is that you should not build in any wired-in limits in your shell. You should just manage resources and handle failure (of syscalls(2) and of standard functions like malloc(3) ...). Avoid defining arbitrary builtin limits like your MAX_CMD_LEN (BTW, the exact figure is not the same on my Debian/Sid/x86-64 running a kernel 3.17 which I did compile myself).

Upvotes: 1

Charles Duffy
Charles Duffy

Reputation: 295373

No, there is not such a limit. For a single process table entry, ARG_MAX applies (to combined environment and argv length). However, a pipeline spans separate process table entries, and each piece connects only to the process before it (via stdin) and the process after it (via stdout). There's no single place the entire pipeline needs to be stored or tracked as a unit inside the operating system itself (as opposed to the constructing shell).

Upvotes: 5

Related Questions