CODEWITHSUNDEEP
bashawksed
Bash learner
Bash learner

Reputation: 31

How to convert multiple parameters URLs into single parameter URLs in bash

$ cat urls.txt
http://example.com/test/test/test?apple=&bat=&cat=&dog=
https://test.com/test/test/test?aa=&bb=&cc=
http://target.com/test/test?hmm=

I want output like below 👇🏻 , how can i do that in bash ( single line command )

$ cat urls.txt
http://example.com/test/test/test?apple=
http://example.com/test/test/test?bat=
http://example.com/test/test/test?cat=
http://example.com/test/test/test?dog=
https://test.com/test/test/test?aa=
https://test.com/test/test/test?bb=
https://test.com/test/test/test?cc=
http://target.com/test/test?hmm=

Upvotes: 0

Views: 72

Answers (4)

potong
potong

Reputation: 58401

This might work for you (GNU sed):

sed -E 's/(([^?]+\?)[^=]+=)&/\1\n\2/;P;D' file

Replace each & by a newline and the substring before the first parameter, print/delete the first line and repeat.

Upvotes: 0

Carlos Pascual
Carlos Pascual

Reputation: 1126

With GNU awk using gensub():

awk '{print gensub(/^(https?:)(.*)(\?[[:alpha:]]+=)(.*)/,"\\1\\2\\3","g")}' file
http://example.com/test/test/test?apple=
https://test.com/test/test/test?aa=
http://target.com/test/test?hmm=
  • gensub() for specifying components of the regexp in the replacement text, using parentheses in the regexp to mark the components (four here). We print only 3 of them: "\\1\\2\\3" .

Upvotes: 0

Randy Movit
Randy Movit

Reputation: 1

I try use sed but it is complex. if use perl like this:

perl -pe 'if(/(.*\?)/){$url=$1;s#&#\n$url#g;}' url.txt

it works well.

Upvotes: 0

Renaud Pacalet
Renaud Pacalet

Reputation: 29040

With GNU awk:

$ awk -F'?|=&|=' '{for(i=2;i<NF;i++) print $1 "?" $i "="}' urls.txt 
http://example.com/test/test/test?apple=
http://example.com/test/test/test?bat=
http://example.com/test/test/test?cat=
http://example.com/test/test/test?dog=
https://test.com/test/test/test?aa=
https://test.com/test/test/test?bb=
https://test.com/test/test/test?cc=
http://target.com/test/test?hmm=

Upvotes: 1

Related Questions