CODEWITHSUNDEEP
linuxbashscriptingshell
Salman Arshad
Salman Arshad

Reputation: 272306

Simple regular expression parsing in bash

I want to parse a log file (log.txt) which contains rows similar to these:

2010-10-19 07:56:14 URL:http://www.website.com/page.php?ID=26 [13676] -> "www.website.com/page.php?ID=26" [1]
2010-10-19 07:56:14 URL:http://www.website.com/page.php?ID=44 [14152] -> "www.website.com/page.php?ID=44" [1]
2010-10-19 07:56:14 URL:http://www.website.com/page.php?ID=13 [13681] -> "www.website.com/page.php?ID=13" [1]
2010-10-19 07:56:14 ERROR:Something bad happened
2010-10-19 07:56:14 ERROR:Something really bad happened
2010-10-19 07:56:15 URL:http://www.website.com/page.php?ID=14 [12627] -> "www.website.com/page.php?ID=14" [1]
2010-10-19 07:56:14 ERROR:Page not found
2010-10-19 07:56:15 URL:http://www.website.com/page.php?ID=29 [13694] -> "www.website.com/page.php?ID=29" [1]

As you might have guessed:

1) I need to extract this portion from each row:

2010-10-19 07:56:15 URL:http://www.website.com/page.php?ID=29 [13694] -> "www.website.com/page.php?ID=29" [1]
------------------------^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

2) This portion goes to another file (log.html) like this:

<a href="http://www.website.com/page.php?ID=29">http://www.website.com/page.php?ID=29</a>

I need to do this via bash script, which will run on a *nix platform. I have no idea about shell programming so detailed script will be much appreciated, pointers to bash programming reference will do.

Upvotes: 3

Views: 6491

Answers (5)

codaddict
codaddict

Reputation: 455350

Something like this:

while read line
do
        URL=$(echo $line | egrep -o 'URL:[^ ]+' | sed  's/^URL://')     
        if [ -n "$URL" ]; then
                echo "<a href=\"$URL\">$URL</a>" >> output.txt
        fi  
done < input.txt

Upvotes: 1

ghostdog74
ghostdog74

Reputation: 342819

Here's a bash solution

#!/bin/bash
exec 4<"log.txt"
while read -r line<&4
do
  case "$line" in
    *URL:* )
      url="${line#*URL:}"
      url=${url%% [*}
      echo "<a href=\"${url}\">${url}</a>"
  esac
done
exec 4<&-

Upvotes: 5

Zsolt Botykai
Zsolt Botykai

Reputation: 51663

What about sed:

sed -n 's/.*URL:\([^ ]\+\) .*/<a href="\1">\1<\/a>/;/<a href/p' logfile

(Please note: you can address the URL part more properly, e.g. by the length of the date string in front of it, but I was just lazy.)

Upvotes: 1

mouviciel
mouviciel

Reputation: 67869

This should work:

sed -n 's%^.* URL:\(.*\) \[[0-9]*\] -> .*$%<a href="\1">\1</a>%p' log.txt

Upvotes: 2

a&#39;r
a&#39;r

Reputation: 37019

Here's a small awk script that should do what you need.

awk '/URL:/ { sub(/^URL:/,"", $3); printf "<a href=\"%s"\">%s</a>\n", $3, $3; }'

Upvotes: 2

Related Questions