CODEWITHSUNDEEP
bashsedldapbase64
jgrocha
jgrocha

Reputation: 3052

using sed to replace some 64 based encoded strings (calling external echo + base64)

I'm using ldapsearch to query AD servers. Some results are 64 encoded (always with :: before), like:

company: Medidata
company: Milestone
company:: QWRQIFNlcnZpw6dvcw==
company: ROFF

Manually, I can decode those encoded strings with:

echo QWRQIFNlcnZpw6dvcw== | base64 -d

But I'm not able to convert all strings using sed (to replace only the encoded strings).

This works, but does do the replacement. It is used just for debug.

cat output.txt | sed "s/:: \(.\+\)/: `echo \\\1`/"
company: Medidata
company: Milestone
company: QWRQIFNlcnZpw6dvcw==
company: ROFF

What I would like to do is:

cat output.txt | sed "s/:: \(.\+\)/: `echo \\\1 | base64 -d`/"
base64: invalid input
company: Medidata
company: Milestone
company: 
company: ROFF

base64 complains about the input, but it looks good to me.

What am I doing wrong?

Upvotes: 0

Views: 1004

Answers (2)

pynexj
pynexj

Reputation: 20768

With GNU sed you can use s///e:

$ cat file
company: Medidata
company: Milestone
company:: QWRQIFNlcnZpw6dvcw==
company: ROFF
$ cat file | gsed -E "s/(.*):: (.*)/printf %s '\\1: '; echo '\\2' | base64 -d/"
company: Medidata
company: Milestone
printf %s 'company: '; echo 'QWRQIFNlcnZpw6dvcw==' | base64 -d
company: ROFF
$ cat file | gsed -E "s/(.*):: (.*)/printf %s '\\1: '; echo '\\2' | base64 -d/e"
company: Medidata
company: Milestone
company: AdP Serviços
company: ROFF

The doc for s///e:

This command allows one to pipe input from a shell command into pattern space. If a substitution was made, the command that is found in pattern space is executed and pattern space is replaced with its output. A trailing newline is suppressed; results are undefined if the command to be executed contains a NUL character. This is a GNU sed extension.

Upvotes: 4

Socowi
Socowi

Reputation: 27255

With sed "s/:: \(.\+\)/: `echo \\\1`/" and sed "s/:: \(.\+\)/: `echo \\\1 | base64 -d`/" the subshell `...` is expanded before sed even runs. Therefore base64 complains that it cannot decode the literal string \1.

You have to instruct sed (which has a different syntax than bash) to call an external program. Standard sed cannot call external programs. See How to embed a shell command into a sed expression?. But GNU sed can:

sed -E "s/(.*:): (.+)/printf %s '\1 '; echo '\2' | base64 -d/e"

This assumes that the line you are replacing does not contain any '. In your case, this shouldn't be a problem I think.

Upvotes: 3

Related Questions