CODEWITHSUNDEEP
macosshellcommand-linexargs
Itachi
Itachi

Reputation: 6080

How to suspend the main command when piping the output to another delay command

I have two custom scripts to implement their own tasks, one for outputting some URLs (pretend as cat command below) and another for receiving a URL to parse it via network requests (pretend as sleep command below).

Here is the prototype:

Case 1:

cat urls.txt | xargs -I{} sleep 1 && echo "END: {}"

The output is END: {} and the sleep works.

Case 2:

cat urls.txt | xargs -I{} echo "BEGIN: {}" && sleep 1 && echo "END: {}"

The output is 

BEGIN: https://www.example.com/1
BEGIN: https://www.example.com/2
BEGIN: https://www.example.com/3
END: {}

but it seems only sleep 1 second.

Q1: I'm a little confused, why are these outputs?

Q2: Are there any solutions to execute the full pipelined xargs delay command for every cat line output?

Upvotes: 0

Views: 100

Answers (2)

Itachi
Itachi

Reputation: 6080

Thanks for shellter and UtLox's reminder, I found the xargs is the key.

Here is my finding, the shell/zsh interpreter splits the sleep 5 and echo END: {} as another serial of commands, so xargs didn't receive my expected two && inline commands as one utility command and replace the {} with value in the END expression. This could be proved by xargs -t.

cat urls.txt | xargs -I{} -t echo "BEGIN: {}" && sleep 1 && echo "END: {}"

Inspired by UtLox's the answer, I found I could join my expectation with sh -c in xargs.

cat urls.txt | xargs -I{} -P 5 sh -c 'echo "BEGIN: {}" && sleep 1 && echo "END: {}"'

For the -P 5, it makes the utility commmand ran with max specified subprocesses in parallel mode to make use of most bandwide resources.

Done!

Upvotes: 1

UtLox
UtLox

Reputation: 4164

You can put the commands into a separate script:

worker.sh

#!/bin/bash
echo "BEGIN: $*" && sleep 1 && echo "END: $*"

set execute permission:

chmod +x worker.sh

and call it with xargs:

cat urls.txt | xargs -I{} ./worker.sh {}

output

BEGIN: https://www.example.com/1
END: https://www.example.com/1
BEGIN: https://www.example.com/2
END: https://www.example.com/2
BEGIN: https://www.example.com/3
END: https://www.example.com/3

Between BEGIN and END the script sleep for one second.

Upvotes: 2

Related Questions