CODEWITHSUNDEEP
bashawk
Sven
Sven

Reputation: 53

Append to a string in bash

I'm trying to get a download URL using curl and awk and want to append something to that URL afterwards.

Here some snipped of my code:

IMAGE=$(curl -I -s https://downloads.raspberrypi.org/raspbian_lite_latest | awk '/Location/ {print $2}')
CHECKSUM="$IMAGE.sha256"

echo $IMAGE
echo $CHECKSUM

What I'm getting is that it is somehow replacing parts at the beginning.

https://downloads.raspberrypi.org/raspbian_lite/images/raspbian_lite-2018-11-15/2018-11-13-raspbian-stretch-lite.zip
.sha256/downloads.raspberrypi.org/raspbian_lite/images/raspbian_lite-2018-11-15/2018-11-13-raspbian-stretch-lite.zip

I'm a bit helpless, because the following works as expected:

A="https""://abc.org/a_b/a.zip" # looks weird, but full URLs are not allowed here
B="$A.sha256"
echo $B

What am I doing wrong?

Upvotes: 4

Views: 125

Answers (3)

James Brown
James Brown

Reputation: 37394

Since you are using bash you can use substring replacement, ie. replace the \r in IMAGEvar:

$ CHECKSUM="${IMAGE/$'\r'/}.sha256"
$ echo $CHECKSUM 
https://downloads.raspberrypi.org/raspbian_lite/images/raspbian_lite-2018-11-15/2018-11-13-raspbian-stretch-lite.zip.sha256

or prepare for it in the awk part by setting the record separator RS:

... | awk -v RS="\r?\n" '/Location/ {print $2}'

Tested with gawk, mawk and original-awk. Surprisingly busybox awk removed it by itself:

$ echo -e \\r | busybox awk '{print $1}' | hexdump -C
00000000  0a                                                |.|

but for example:

$ echo -e \\r | gawk '{print $1}' | hexdump -C
00000000  0d 0a                                             |..|

Upvotes: 1

Ondrej K.
Ondrej K.

Reputation: 9664

The problem apparently is, that your $IMAGE contains / ends in a trailing '\r(carriage return). So you've actually appended ".sha256" as you expected to"something\r.sha256" which when being echoed means.... something, cursor back to the beginning of the line, .sha256. Long story short, strip that '\r`. E.g:

IMAGE=$(curl -I -s https://downloads.raspberrypi.org/raspbian_lite_latest | awk '/Location/ {sub(/\r$/, "", $2); print $2}')

Upvotes: 2

Jan Gassen
Jan Gassen

Reputation: 3534

When you hexdump your string, you see that is uses windows line endings (with carriage return):

echo $IMAGE | hexdump -C
00000000  68 74 74 70 73 3a 2f 2f  64 6f 77 6e 6c 6f 61 64  |https://download|
00000010  73 2e 72 61 73 70 62 65  72 72 79 70 69 2e 6f 72  |s.raspberrypi.or|
00000020  67 2f 72 61 73 70 62 69  61 6e 5f 6c 69 74 65 2f  |g/raspbian_lite/|
00000030  69 6d 61 67 65 73 2f 72  61 73 70 62 69 61 6e 5f  |images/raspbian_|
00000040  6c 69 74 65 2d 32 30 31  38 2d 31 31 2d 31 35 2f  |lite-2018-11-15/|
00000050  32 30 31 38 2d 31 31 2d  31 33 2d 72 61 73 70 62  |2018-11-13-raspb|
00000060  69 61 6e 2d 73 74 72 65  74 63 68 2d 6c 69 74 65  |ian-stretch-lite|
00000070  2e 7a 69 70 0d 0a                                 |.zip..|
00000076

To fix that, use

IMAGE=$(curl -I -s https://downloads.raspberrypi.org/raspbian_lite_latest | awk '/Location/ {print $2}' | tr -d "\r")

Upvotes: 3

Related Questions