CODEWITHSUNDEEP
sed
Nikolai Prokoschenko
Nikolai Prokoschenko

Reputation: 8765

How do I push `sed` matches to the shell call in the replacement pattern?

I need to replace several URLs in a text file with some content dependent on the URL itself. Let's say for simplicity it's the first line of the document at the URL.

What I'm trying is this:

sed "s/^URL=\(.*\)/TITLE=$(curl -s \1 | head -n 1)/" file.txt

This doesn't work, since \1 is not set. However, the shell is getting called. Can I somehow push the sed match variables to that subprocess?

Upvotes: 1

Views: 2777

Answers (4)

blackghost
blackghost

Reputation: 1825

Late reply, but making sure people don't get thrown off by the answers here -- this can be done in gnu sed using the e command. The following, for example, decrements a number at the beginning of a line:

echo "444 foo" | sed "s/\([0-9]*\)\(.*\)/expr \1 - 1 | tr -d '\n'; echo \"\2\";/e"

will produce:

443 foo

Upvotes: 0

e40
e40

Reputation: 404

The accept answer is just plain wrong. Proof:

Make an executable script foo.sh:

#! /bin/bash

echo $* 1>&2

Now run it:

$ echo foo | sed -e "s/\\(foo\\)/$(./foo.sh \\1)/"
\1
$

The $(...) is expanded before sed is run.

Upvotes: 4

Dennis Williamson
Dennis Williamson

Reputation: 360315

Try this:

sed "s/^URL=\(.*\)/\1/" file.txt | while read url; do sed "s@URL=\($url\)@TITLE=$(curl -s $url | head -n 1)@" file.txt; done

If there are duplicate URLs in the original file, then there will be n^2 of them in the output. The @ as a delimiter depends on the URLs not including that character.

Upvotes: 1

leonbloy
leonbloy

Reputation: 75976

So you are trying to call an external command from inside the replacement pattern of a sed substitution. I dont' think it can be done, the $... inside a pattern just allows you to use an already existent (constant) shell variable.

I'd go with Perl, see the /e option in the search-replace operator (s/.../.../e).

UPDATE: I was wrong, sed plays nicely with the shell, and it allows you do to that. But, then, the backlash in \1 should be escaped. Try instead:

sed "s/^URL=\(.*\)/TITLE=$(curl -s \\1 | head -n 1)/" file.txt

Upvotes: 3

Related Questions