Peter Butkovic
Peter Butkovic

Reputation: 12129

pass wget output (streamed shell script) to bash, but with extra arguments

I'd like to download the specific file via wget, pass it as a bash script and in one shot also provide arguments for it.

In my case, the script is stored at: https://raw.githubusercontent.com/vishnubob/wait-for-it/master/wait-for-it.sh

I've tried:

wget -O - https://raw.githubusercontent.com/vishnubob/wait-for-it/master/wait-for-it.sh | bash

but it ends with:

Error: you need to provide a host and port to test.
Usage:
    bash host:port [-s] [-t timeout] [-- command args]
    -h HOST | --host=HOST       Host or IP under test
    -p PORT | --port=PORT       TCP port under test
                            Alternatively, you specify the host and port as host:port
    -s | --strict               Only execute subcommand if the test succeeds
    -q | --quiet                Don't output any status messages
    -t TIMEOUT | --timeout=TIMEOUT
                            Timeout in seconds, zero for no timeout
    -- COMMAND ARGS             Execute command with args after the test finishes

as I'd also need to pass arguments to this bash script (hostname and port to check in my specific case), namely I'd need to run something like:

wait-for-it.sh localhost:8181

UPDATE: I'd love to have the solution without local save (=> pipe to bash only please)

Upvotes: 5

Views: 2480

Answers (1)

agc
agc

Reputation: 8406

For a script that's not recursive, it's easy:

 # pipe source code to `bash`, run code with args *foo* and *bar*
 <stream with source code> | bash -s - foo bar 

But the script wait-for-it.sh contains $0, and is somewhat recursive, (it calls itself), and which makes it incompatible with streaming, since:

  1. the stream doesn't have a filename where random access seeks work,
  2. there's no way to change $0 to the name of the stream.

A bash piping function gets around that:

strm2fnct(){ 
s=${1:-self$$}
sed "1i $s"'() {
s/\$0/'"$s"'/
/timeout/{s/'"$s"'[^&]*/bash -c "&" /};
$a \} ; export -f '"$s; $s"' "$@"'
}

Usage for this Q:

f='https://raw.githubusercontent.com/vishnubob/wait-for-it/master/wait-for-it.sh'
wget -O - "$f" | strm2fnct ${f##*/} | bash -s - 'localhost:8181'

Output:

--2017-05-21 21:21:49--  https://raw.githubusercontent.com/vishnubob/wait-for-it/master/wait-for-it.sh
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.36.133
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.36.133|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 4070 (4.0K) [text/plain]
Saving to: ‘STDOUT’

-   100%[===========================================>]   3.97K  --.-KB/s    in 0.1s    

2017-05-21 21:21:50 (29.8 KB/s) - written to stdout [4070/4070]

wait-for-it.sh: waiting 15 seconds for localhost:8181
wait-for-it.sh: timeout occurred after waiting 15 seconds for localhost:8181

Method.

Although bash doesn't save streaming data, it remembers functions. Therefore strm2fnct:

  • wraps the entire stream (comments and all) inside of an ad hoc shell function; example:

    strm2fnct <<< "echo hello world"
    

    Output:

    self6196() {
    echo hello world
    } ; export -f self6196; self6196 "$@"
    
  • this ad hoc function gets a quasi-random name by default, (really it's the string "self" followed by the PID), or it can be passed a name, e.g. strm2fnct foobar names the ad hoc function foobar();

  • replaces every instance of $0 with the ad hoc name, except that...
  • these timeout commands in wait-for-it.sh require further edits:

    grep -n '^ *timeout' wait-for-it.sh 
    56:        timeout $BUSYTIMEFLAG $TIMEOUT $0 --quiet --child --host=$HOST --port=$PORT --timeout=$TIMEOUT &
    58:        timeout $BUSYTIMEFLAG $TIMEOUT $0 --child --host=$HOST --port=$PORT --timeout=$TIMEOUT &
    

    ...because timeout can't see shell functions, so the ad hoc function needs to be exported, and called by bash -c, and its arguments need to be quoted. See what changes by running:

    diff wait-for-it.sh <( strm2fnct wait-for-it.sh < wait-for-it.sh )
    

Upvotes: 5

Related Questions